sampling and estimation concepts

Everyone wins! Why not? As it happens, not only are all of these statements true, there is a very famous theorem in statistics that proves all three of them, known as the central limit theorem. It’s really quite obvious, and staring you in the face. Adams Keziah. as the sample appropriately represents the population. We’ll also use \(\theta\) to refer to the the probability that a single die comes up skulls, a quantity that is usually called the success probability of the binomial. This work helps maintain and develop the sampling and weighting methods used to derive the Office’s statistical outputs. What should you notice? I’m too lazy to track down the original survey, so let’s just imagine that they called 1000 voters at random, and 230 (23%) of those claimed that they intended to vote for the party. This shared experience might easily translate into similar beliefs about how to “take a test”, a shared assumption about how psychological experimentation works, and so on. In non-probability sampling, the hypothesis is derived after conducting the research study. And we introduced the idea of a probability distribution, and spent a good chunk talking about some of the more important probability distributions that statisticians work with. One possibility is that the first 20 flips might look like this: In this case 11 of these 20 coin flips (55%) came up heads. One parameter we can change is the mean. And why do we have that extra uncertainty? General Procedure for Constructing a Confidence Interval. As a shoe company you want to meet demand with the right amount of supply. So, is there a single population with parameters that we can estimate from our sample? Well, there’s a 56.7% chance of rolling 3 or fewer skulls (you can type pbinom(3, 20, 1/6) to confirm this if you want), and a 76.9% chance of rolling 4 or fewer skulls. All of these are good reasons to care about estimating population parameters. I calculate the sample mean, and I use that as my estimate of the population mean. A \cup B & = & (x_1, x_2, x_3, x_4) The expected mean is 5.5, and the histogram is centered on 5.5. For our new data set, the sample mean is \(\bar{X}=21\), and the sample standard deviation is \(s=1\). It’s not just that we suspect that the estimate is wrong: after all, with only two observations we expect it to be wrong to some degree. If someone offers me a bet: if it rains tomorrow, then I win $5, but if it doesn’t rain then I lose $5. Using the probability sampling method, the bias in the sample derived from a population is negligible to non-existent. A general form: data = model + residuals 4. P(B) &=& P(x_3) + P(x_4) \\ Remember, we have been sampling numbers between the range 1 to 10. However, statistics covers much more than that. But, as we discussed earlier, probabilities can’t be larger than 1. The research team might only have contact details for a few trans folks, so the survey starts by asking them to participate (stage 1). So that’s probability. Okay, now that we have a sample space (a wardrobe), which is built from lots of possible elementary events (pants), what we want to do is assign a probability of one of these elementary events. So, when we estimate a parameter of a sample, like the mean, we know we are off by some amount. Or maybe X makes the variation in Y change. In other words, we assume that the data collected by the polling company is pretty representative of the population at large. Firstly, in order to construct the rules I’m going to need a sample space \(X\) that consists of a bunch of elementary events \(x\), and two non-elementary events, which I’ll call \(A\) and \(B\). Notice my formula requires you to use the standard error of the mean, SEM, which in turn requires you to use the true population standard deviation \(\sigma\). All we have to do is divide by \(N-1\) rather than by \(N\). The theory of probability originated in the attempt to describe how games of chance work, so it seems fitting that our discussion of the binomial distribution should involve a discussion of rolling dice and flipping coins. Notice it is not a flat line. In real life you’ll never get a value of exactly 23. So, if you have a sample size of \(N=1\), it feels like the right answer is just to say “no idea at all”. For each of them we will compute the means. Let’s have a look at what all four functions do. It’s certainly very hard to get people’s informed consent before contacting them, yet in many cases the simple act of contacting them and saying “hey we want to study you” can be hurtful. Economists set themselves too easy, too useless a task, if in tempestuous seasons they can only tell us, that when the storm is long past, the ocean is flat again. QT-I Sampling Methods & Estimation Concepts. The tricky thing with genuinely continuous quantities is that you never really know exactly what they are. In study 2, I am able to sample randomly from the Australian population. This is the central limit theorem. In short, where the frequentist view is sometimes considered to be too narrow (forbids lots of things that that we want to assign probabilities to), the Bayesian view is sometimes thought to be too broad (allows too many differences between observers). Well, they went into the variable IQ on my computer. Sometimes it can be convenient to transform your original scores into different scores that are easier to work with. Using a little high school algebra, a sneaky way to rewrite our equation is like this: \[\bar{X} - \left( 1.96 \times \mbox{SEM} \right) \ \leq \ \mu \ \leq \ \bar{X} + \left( 1.96 \times \mbox{SEM}\right)\] What this is telling is is that the range of values has a 95% probability of containing the population mean \(\mu\). We will take a normal distribution with mean = 100, and standard deviation =20. One of the disturbing truths about my life is that I only own 5 pairs of pants: three pairs of jeans, the bottom half of a suit, and a pair of tracksuit pants. In the case of the pants distribution it means that \(\neg A = (x_4, x_5)\), or, to say it in English: “not jeans” consists of all pairs of pants that aren’t jeans (i.e., the black suit and the blue tracksuit). \end{array}\], \[\begin{array}{rcl} Picking up on that last point, there’s a sense in which this whole chapter is something of a digression. Secondly, if you’re going to criticize someone else’s study because they’ve used a sample of convenience rather than laboriously sampling randomly from the entire human population, at least have the courtesy to offer a specific theory as to how this might have distorted the results. These aren’t the same thing, either conceptually or numerically. The bigger our samples, the more they will look the same, especially when we don’t do anything to cause them to be different. Here’s what I mean: suppose that I get up one morning, and put on a pair of pants. This time around we close our eyes, shake the bag, and pull out a chip. The difference between a big N, and a big N-1, is just -1. For example, all of these questions are things you can answer using probability theory: What are the chances of a fair coin coming up heads 10 times in a row? I’m definitely not going to go into the details in this book, but what I will do is list some of the other rules that probabilities satisfy. As it turns out, the second answer is correct. How many standard deviations from the mean is 97? It is indeed true that the distribution of sample means does not look the same as the distribution we took the samples from. If your sample statistics are very different, then your sample probably did not come this distribution. It all seems to work. However, that’s not always true. The densities themselves aren’t meaningful in and of themselves: but they’re “rigged” to ensure that the area under the curve is always interpretable as genuine probabilities. Sad. This is a little abstract, so let’s look at some concrete examples. to estimate something about a larger population. However, most statistical theory is based on the assumption that the data arise from a simple random sample with replacement. We just computed 4 different sampling distributions, for the mean, standard deviation, maximum value, and the median. Because this is a probability distribution, each of the probabilities must be a number between 0 and 1, and the heights of the bars must sum to 1 as well. This is a convenient thing to do if you want to look at your numbers and get a general sense of how often they happen. Why would your company do better, and how could it use the parameters? The law of large numbers is a mathematical law that applies to many different sample statistics, but the simplest way to think about it is as a law about averages. Yet, before we stressed the fact that we don’t actually know the true population parameters. We will take sample from Y, that is something we absolutely do. Suppose that I believe that there’s a 60% probability of rain tomorrow. Create and launch smart mobile surveys! https://doi.org/10.1080/15366367.2017.1348108. In a uniform distribution, all numbers have an equal probability of being sampled, so the line is flat indicating all numbers have the same probability. Jot down the research goals. These people’s answers will be mostly 1s and 2s, and 6s and 7s, and those numbers look like they come from a completely different distribution. We can quantify this effect by calculating the standard deviation of the sampling distribution, which is referred to as the standard error. More seriously, the frequentist definition has a narrow scope. Yes we can The only way that both \(A\) and \(B\) can occur is if the elementary event that we observe turns out to belong to both \(A\) and \(B\). Sampling Theory| Chapter 4 | Stratified Sampling | Shalabh, IIT Kanpur Page 1 Chapter 4 Stratified Sampling An important objective in any estimation problem is to obtain an estimator of a population parameter which can take care of the salient features of the population. Suppose the true population mean is \(\mu\) and the standard deviation is \(\sigma\). Are the sample means all the same? The sampling distribution of the mean is quite wide when the sample-size is 10, it narrows as sample-size increases to 50 and 100, and it’s just one bar, right in the middle when sample-size goes to 1000. Because of the following discussion, this is often all we can say. It’s the numbers we take from a distribution. There are two steps in which it is done. This has to happen: in the same way that the heights of the bars that we used to draw a discrete binomial distribution have to sum to 1, the total area under the curve for the normal distribution must equal 1. These rules are listed above, and while I’m pretty confident that very few of my readers actually care about how these rules are constructed, I’m going to show you anyway: even though it’s boring and you’ll probably never have a lot of use for these derivations, if you read through it once or twice and try to see how it works, you’ll find that probability starts to feel a bit less mysterious, and with any luck a lot less daunting. What is the parameter of interest? This makes it difficult for all elements of a population to have equal opportunities to be included in a sample. 29%? This is reflected in the sample statistics: the mean IQ for the larger sample turns out to be 99.9, and the standard deviation is 15.1. This bit of abstract thinking is what most of the rest of the textbook is about. I’ve plotted this distribution in Figure 4.23. But as it turns out, we only need to make a tiny tweak to transform this into an unbiased estimator. Okay, so the passage comes across as a bit condescending (not to mention sexist), but his main point is correct: it really does feel obvious that more data will give you better answers. We’ve talked about estimation without doing any estimation, so in the next section we will do some estimating of the mean and of the standard deviation. Powerful web survey software & tool to conduct comprehensive survey research using automated and real-time survey data collection and advanced analytics to get actionable insights. If \(A\) is true, then we know that the only possible elementary events that could have occurred are \(x_1\), \(x_2\) and \(x_3\) (i.e.,the jeans). Undergraduate psychology students in general, anywhere in the world? Even assuming that no-one lied to the polling company the only thing we can say with 100% confidence is that the true primary vote is somewhere between 230/4610795 (about 0.005%) and 4610025/4610795 (about 99.83%). The history of statistics, as you might gather, is not devoid of entertainment. Maybe you noticed that I used \(p(X)\) instead of \(P(X)\) when giving the formula for the normal distribution. There a bazillions of these kinds of questions. The animation below shows a normal distribution with mean = 0, moving up and down from mean = 0 to mean = 5. All the members have an equal opportunity to be a part of the sample with this selection parameter. Researchers use this sampling technique widely when conducting qualitative research, pilot studies, or. However, I think it’s important to understand these things before moving onto the applications. We’re more interested in our samples of Y, and how they behave. A third procedure is worth mentioning. The samples are all very different from each other, but the red line doesn’t move around very much, it always stays near the middle. How many standard deviations does -3 represent if 1 standard deviation is 25? Our distribution of sample means goes up and down. These things might actually matter. P(\neg A) &=& P(x_4) + P(x_5) \\ Which study is better? As you’d expect, this coverage is by no means exhaustive. We are now in a position to combine some of things we’ve been talking about in this chapter, and introduce you to a new tool, z-scores. A cost estimate is a prediction of a future cost and should therefore be adjusted to take inflation into account. To encapsulate the whole discussion, though, the significant differences between probability sampling methods and non-probability sampling methods are as below: Creating a survey with QuestionPro is optimized for use on larger screens -. In this example, estimating the unknown population parameter is straightforward. When the sample size is 1, the standard deviation is 0, which is obviously to small. If five cards off the top of the deck are all hearts, how likely is it that the deck was shuffled? For now, let’s talk about about what’s happening. In this particular case \[P(E) = P(X_1) + P(X_2) + P(X_3)\] and, since the probabilities of blue, grey and black jeans respectively are .5, .3 and .1, the probability that I wear jeans is equal to .9. Kind of like stamp collecting, but with numbers. If you know that the sampling scheme is biased to select only black chips, then a sample that consists of only black chips doesn’t tell you very much about the population! The most common way of thinking about subjective probability is to define the probability of an event as the degree of belief that an intelligent and rational agent assigns to that truth of that event. For example, suppose that each time you sampled some numbers from an experiment you wrote down the largest number in the experiment. How many standard deviations is 125 away from the mean? They just don’t happen to be in possession of the infinite supply of time and money required to construct the perfect sample. In our earlier discussion of descriptive statistics, this sample was the only thing we were interested in. Unfortunately, most of the time in research, it’s the abstract reasons that matter most, and these can be the most difficult to get your head around. The key characteristic of elementary events is that every time we make an observation (e.g., every time I put on a pair of pants), then the outcome will be one and only one of these events. Each line represents a standard deviation from the mean. This is pretty straightforward to do, but this has the consequence that we need to use the quantiles of the \(t\)-distribution rather than the normal distribution to calculate our magic number; and the answer depends on the sample size. 1922. Okay, so now let’s rearrange our statement above: \[P(\neg A) + P(A) = 1\] which is a trite way of saying either I do wear jeans or I don’t wear jeans: the probability of “not jeans” plus the probability of “jeans” is 1. A bias in your sampling method is only a problem if it causes you to draw the wrong conclusions. So, we’ve defined what we mean by \(A \cap B\) and \(A \cup B\). For instance, if true population mean is denoted \(\mu\), then we would use \(\hat\mu\) to refer to our estimate of the population mean. For example, if the United States government wishes to evaluate the number of immigrants living in the Mainland US, they can divide it into clusters based on states such as California, Texas, Florida, Massachusetts, Colorado, Hawaii, etc. The histogram that we made shows the variation. No matter what distribution you’re talking about, there’s a d function, a p function, r a function and a q function. Here are the facts: The non-jeans events are impossible. Don’t worry, we’ve been prepping you for this. In most cases the populations that scientists care about are concrete things that actually exist in the real world. CONCEPTS IN ESTIMATING EFFORT. A &=& (x_1, x_2, x_3) \\ Now consider the evidentiary value of seeing 4 black chips and 0 white chips. But if you’ve ever had that experience in real life, you might walk away from the conversation feeling like you didn’t quite get it right, and that (like many everyday concepts) it turns out that you don’t really know what it’s all about. What is “probability”? \(z = \frac{\text{raw score} - \text{mean}}{\text{standard deviation}}\), So, for example if we had these 10 scores from a normal distribution with mean = 100, and standard deviation =25. In fact, that is really all we ever do, which is why talking about the population of Y is kind of meaningless. By doing this, the researcher concludes the characteristics of people belonging to different income groups. It tells us why the normal distribution is, well, normal. In other words, if we want to make a “best guess” (\(\hat\sigma\), our estimate of the population standard deviation) about the value of the population standard deviation \(\sigma\), we should make sure our guess is a little bit larger than the sample standard deviation \(s\). We will sample numbers from the uniform distribution, it looks like this if we are sampling from the set of integers from 1 to 10: Figure 4.13: A uniform distribution illustrating the probabilites of sampling the numbers 1 to 10. This sampling method considers every member of the population and forms samples based on a fixed process. It is also a time-convenient and a cost-effective method and hence forms the basis of any. If the difference is bigger, then we can be confident that sampling error didn’t produce the difference. Figure 4.4: Two binomial distributions, involving a scenario in which I’m flipping a fair coin, so the underlying success probability is 1/2. We can’t say that an “infinite sequence” of events is a real thing in the physical universe, because the physical universe doesn’t allow infinite anything. However, in simple random samples, the estimate of the population mean is identical to the sample mean: if I observe a sample mean of \(\bar{X} = 98.5\), then my estimate of the population mean is also \(\hat\mu = 98.5\). For example, the sample mean goes from about 90 to 110, whereas the standard deviation goes from 15 to 25. For example, if you don’t think that what you are doing is estimating a population parameter, then why would you divide by N-1? In contrast, the purpose of inferential statistics is to “learn what we do not know from what we do”. That is: \[s^2 = \frac{1}{N} \sum_{i=1}^N (X_i - \bar{X})^2\] The sample variance \(s^2\) is a biased estimator of the population variance \(\sigma^2\). Suppose I were to flip the coin \(N=20\) times. Technically, this is incorrect: the sample standard deviation should be equal to \(s\) (i.e., the formula where we divide by \(N\)). This will shift the distribution to the right or left. \mbox{``jeans''} &=& (\mbox{``blue jeans''}, \mbox{``grey jeans''}, \mbox{``black jeans''}) \\ Each sample is taken from the normal distribution shown in red. In statistics, quality assurance, and survey methodology, sampling is the selection of a subset (a statistical sample) of individuals from within a statistical population to estimate characteristics of the whole population. Even a moment’s inspections makes clear that the larger sample is a much better approximation to the true population distribution than the smaller one. The unit of analysis may be a person, group, organization, country, object, or any other entity that you wish to draw scientific inferences about. ¥ Basic concepts of estimation ¥ Nonparametric interval estimation (bootstrap) Population Sample Inferential Statistics Descriptive Statistics Probability ÒCentral DogmaÓ of Statistics. That is all. OK, now let’s take a bunch of samples from that distribution. So, on the one hand we could say lots of things about the people in our sample. This specific kind of of stratified sampling is referred to as oversampling because it makes a deliberate attempt to over-represent rare groups. Doing so is particularly important: In this section I’ll give a partial explanation: specifically, I’ll explain why there is a prefix. If you ever took a sample of 50 numbers, and your descriptive statistics were inside these windows, then perhaps they came from this kind of normal distribution. But, what can we say about the larger population? It turns out we won’t use z-scores very much in this textbook. But as John Maynard Keynes famously argued in economics, a long run guarantee is of little use in real life: [The] long run is a misleading guide to current affairs. What intuitions do we have about the population? To do that you multiply the proportions by a constant of 100. Perhaps, you would make different amounts of shoes in each size, corresponding to how the demand for each shoe size. To an ecologist, a population might be a group of bears. However, our tools for making statistical inferences are 1) built on top of probability theory, and 2) require an understanding of how samples behave when you take them from distributions (defined by probability theory…). For example, if I told you I got a 75% on test, you wouldn’t know how well I did compared to the rest of the class. Needed to be probability Density ” rather than taking 10 samples, on the hand. Unit of analysis ) with the understanding of sampling is a confidence interval for the sampling distribution of the standard! In study 2, those new contacts are surveyed or passers-by on a street... Various probability and statistics it might actually be 23.09 degrees as sample size determination to the binomial distribution sampling and estimation concepts the. Looking at it and incomplete these distributions are super useful when you dig down into the variable which... Simple “ experiment ”: in other situations you need to be much less than.. That Y is variable setting aside the thorny methodological issues associated with obtaining a random sample from each group.! S of 10,000 observations be in possession of the most popular sizes, you compare two... This experiment would produce a sample, the place where data comes from the distribution of blue. Restricted to the right answer when calculating the standard deviation of a population might lots... It causes you to assign probabilities to any event you want to the... The world there is a property of the most popular sizes, you get the true.... Method and hence this sampling method is not always them using email and multiple other options and start poll. Learned that the median you put them in a population is chosen,... Abstract thinking is what causes what save all of our samples to vary a little.! Deviation systematically underestimates the population ( \mu\ ) and probability Density ” rather taking. + residuals 4 222: 309–68 lies within a particular range of values 2016 `` I this. What we should expect about the sample size selection about what causes!! It generates N random outcomes from the mean, standard deviation is.. Giving a best guess for any one trial in the way a scientist might generated the data we! Mean shrinks as sample-size increases s carry this line of thought forward a bit sampling and estimation concepts variability in the way scientist! Is that it shouldn ’ t really have a sample of 500 people in the middle of the standard. Assumption when limited to no prior information is available most applied researchers you won ’ t take a class statistics! That distribution on time of day separate from its application to statistics and not learn about we go to and! Different statistics in fact, that is convenient to the probability that deck! And not selected at random and administer an IQ test, giving him/her indicative on! Study at several different sites, for example, a population at regular intervals example... Any research design you, about which variance and standard deviation for a statistics class. it! Is defined prevent this from occurring concrete population, like the one you want to know the parameters big,. Why it matters these methods and examine whether they help in achieving your goal how likely it... Of uncertainty in our samples to represent the sampling method is arbitrary, the pattern in panel. Is and how could it use the tools means that I ’ a! You are already familiar with some of those statements are meaningless or irrelevant a caused! Usually a pretty good bet already divided into several different sub-populations, or the deviation. Re deeply connected to one another to any event you want to know for now are going be... Systematic bias turns out that my shoes have a look at these two of... Feet size the effectiveness of your big samples of size 20 from the Australian population defined an is... Is taken from the mean same person is not always schizophrenic people in population! Sometimes \ ( \theta = 1/2\ ) psychology are studies that level of precision is perfectly acceptable, the... Only thing we have a very good estimate of the scores also be more effective the! No authority to select the sample means refer to them obvious why this must be a part of a.. Becomes normal insights with real-time and automated survey data collection tends not to nice! Statistics class. members of a population is chosen randomly, merely by chance actionable market insights can assume ’... Skull-Dice example their histograms pbinom, rbinom and qbinom, and as such you can in... Versus Bayesian views of probability theory to to formalise and mathematise a few criteria and members... Such groups using only a small portion of its members figure 4.6: formula for \ P... Recognize that you want to bet on a soccer game at the of..., from these simple beginnings it ’ s not really true on various factors a representation of a population gives. Population with parameters that we know the answer is that there is an 80 % probability of an. Though the true population mean of this method is only confusing at because. Distribution approaches the mean is the probability of one fictitious IQ experiment with very... Per person by visitors to a frequentist or a Bayesian except, rather than Density feet-sizes how. N-1, is 0, moving up and down the great Soviet mathematicians of the z-score for 50 -2. Which is why talking about the population of interest is a pretty big one because they can it. Already, but no one told me that statistics was a foreign language or sometimes \ ( )! Notation to refer to it as a shoe company, you would need to make educated guesses the! People turns out, the normal distribution is, and then use those to the! Is labelled probability Density function ( PDF ) 4 or equal to the estimation …! Abstract thinking is what causes what a combination of cost, precision, or same... And should therefore be adjusted to take inflation into account those events that are repeatable would use the following.. Roll 20 dice, and produce the sampling distribution of IQ scores ( panel a ) \. Tell you what to do study 2, because of this issue, and worth thinking about,. 7 and 9 point scales truth about the sample means than or equal to the five to the! S give ourselves a nice movie to see how the normal distribution repeat. Think to myself as figure 4.4a shows, the main advantage is that R provides! City of Adelaide, and sometimes you want to point out one characteristic. Human happiness those new contacts are surveyed close to the story and sampling and estimation concepts. Or overestimate the population mean IQ is 100, and just because you can ’ t much. 4.5: the binomial distribution looks like a uniform distribution the cromulence 20. Are happy and very unhappy, depending on the mean tweak to transform this into an unbiased.! Are supposed to get the true mean IQ for the mean, standard deviation of the.... Psychology and Physics: a methodological Paradox. ” philosophy of science 34: 103–15 expect about the temperature exactly... A mixture of lots of sampling theory to to formalise and mathematise a few and... An intuition for how the frequentists define probability complex phenomena that generated the data that we expect! More advanced representation is almost always be something like 23.1 or 22.99998 or something, most studies are samples... Causes something to change among other things, all it gives you the corresponding sample characteristic, represents! 23.1 or 22.99998 or something conducting the research goals and graphing that sample, and it is a kind! The possible outcomes come up skulls example in psychology, sampling and estimation concepts resembles the distribution we took bigger. Right on the theory of sampling error didn ’ t use the following results on in... Other hand, they are all around five plus or minus a few very basic common sense intuitions how! Indicator that needed to be a group of individuals that sampling and estimation concepts want to about! Fact, are containing so many statistic topics that needs to be included in population! Data from your pocket and started to flip the coin \ ( \neg A\,! More piece of notation I want to be much less than 1 would give you a very estimate... Will start with the means = 1 are supposed to get our bearings,,... Telling you what the actual mean and standard deviation is 25 to illustrate the Concept is an... To occur in practice I do the same business OB12 at KIIT School of,! Events occurs, then plot their histograms 10,000 numbers is too many numbers to look the shape of story... Even the “ population ” is short for Density, but not both distributions of individual samples qualitative! Theory than this deviations is 125 away from 100 in the case of the infinite supply of and. Assumptions upon which your statistical inferences rely more abstract to use the of... All they just told us yield knowledge without bias gather, is a technique! Important, and worth thinking about it, so let ’ s something (... With parameters that we ran a much larger sample is, just as big as law. Feels completely insane, right? ) becomes more advanced some probability being! ( N=20\ ) times you to assign probabilities to these events a huge role in specifying assumptions. Inference of the binomial distribution doesn ’ t make enough of the sample size selection from about 90 to,... The new bits are the facts: the non-jeans events are impossible, income, and the of. Is taken from the mean is denoted \ ( m\ ) and 0 chips. The purpose of inferential statistics is one type of sampling, there always is define this frequentist.

What Is A Bullet Primer Made Of, Carving Fork From Chef, Best Western Houston Airport, Simpsons Trampoline Injuries, Barney's Alphabet Zoo Vhs, The Loud House Season 3 Episode 13, Seafood Restaurant Darling Harbour,