1 Population and Sample

statistical inference

sampling distribution

Author

Rodolfo Lourenzutti

Learning objectives

Define and distinguish population, sample, parameter, statistic, estimator, and estimate, and identify each in a real study.
Distinguish between the population distribution and the sample distribution, two conceptually distinct objects that are easy to confuse.
Explain what sampling variability is and why it is unavoidable, the problem that the next chapter tackles head-on.

Introduction

Imagine you are a quality-control manager at Apple’s iPhone manufacturing plant. Apple sources its displays from external suppliers, and every incoming shipment must meet strict durability standards before it goes into production. A new shipment of \(50{,}000\) screens has just arrived. To verify quality, you need to determine the average pressure at which a screen cracks; if the average falls below the required threshold, the shipment goes back to the supplier.

**Screen durability testing**: Applying pressure until the screen cracks.

Exercise 1.1 How would you approach this problem?

The most straightforward answer to Exercise 1.1 is to test every screen. If you did, you would know exactly the cracking pressure of every unit and could answer any durability question with perfect certainty. There is just one problem: testing a screen destroys it. Test all \(50{,}000\) and you have no screens left to assemble iPhones, resulting in a flawless quality report, and no product to ship.

So, how can we learn (at least approximately) about the average pressure at which the screens crack without testing all the screens?

A reasonable solution is to test a portion of the screens (i.e., a sample) and use what we learn from that sample to make an informed decision about the whole shipment. This, learning about a large population from a smaller, manageable sample, is the essence of statistical inference.

But going down this path raises important questions:

How should you select the screens to be tested?
How many screens is enough?
How confident can you be that what you observed in, say, \(300\) screens reflects the behaviour of all \(50{,}000\)?
If a colleague independently tested a different \(300\) screens, would they reach the same conclusion?

These questions are of fundamental importance in any statistical study, and they are not just technical details, because statistical inference is not about making educated guesses; it is about quantifying uncertainty in a rigorous, principled way so that decisions can be made with a known level of confidence.

In this chapter, the basic concepts of statistical inference. By the end, you will understand not just which statistics to compute, but why they work and how much to trust them; the very same reasoning that allows a polling firm to estimate the views of 40 million Canadian voters from a survey of \(1{,}500\) people, or a clinical trial of \(300\) patients to reveal whether a new drug actually works.

1.1 Population and Parameters

In statistical inference, we generalize the information from a sample to the entire population. In everyday conversation, words like “population,” “sample,” and “inference” are used quite loosely. To be able to statistically generalize our results, we must be absolutely clear about the boundaries of our study: Who are we studying? What are we measuring?

In statistics, giving these concepts exact boundaries is what allows us to make safe, reliable calculations. Let’s establish these core building blocks, always keeping our shipment of screens in mind.

1.1.1 Who are we studying? The target population.

The first thing to nail down is the group we care about. In the screens problem, Apple’s decision is about the shipment that just arrived: all \(50{,}000\) screens sitting in the warehouse. We call this group the target population:

Definition 1.1 (Target Population) The complete group of all individuals or items that we are interested in studying.

The boundary of the target population matters more than it might seem. It is specifically this shipment, not every screen the supplier has ever made, and not next month’s delivery. A conclusion about this population does not automatically transfer to any other. Defining the target population carefully is the first, and often most overlooked, step of any study.

As you can see in the example above, a vague population boundary doesn’t just make your calculations messy: it can lead to dangerously misleading business decisions.

1.1.2 What do we measure? The variable of interest.

Once we know who we are studying, we specify what we want to learn about each element. In the screens problem, we want to know the pressure at which each screen cracks. We call this the variable of interest:

Definition 1.2 (Variable of Interest) The characteristic or measurement we wish to study.

In our problem, the variable is crack pressure (in psi). Screen #1 has its own crack pressure, screen #2 has a different one, and so on for all \(50{,}000\) screens. In summary, each element in the population has its own value for the variable of interest.

Note that crack pressure is a numerical variable, i.e., it is a number, and taking averages or comparing pressures makes sense. Not every variable works this way. If we were instead asking each screen “did it pass or fail quality control?”, the variable would be categorical: it places each screen into one of two groups rather than assigning it a number.

Common Mistake: Count vs. Variable

Students often think: “If we count that 480 screens passed, ‘480’ is a number, so shouldn’t this be a numerical variable?”

Remember to always look at the individual level. The variable is what you record for one single screen. If you walk up to Screen #42, its value is simply a category: "pass" or "fail". The fact that we later count or average these categories doesn’t change the nature of the variable itself. If the individual raw data consists of labels/words, the variable is categorical.

The type of variable matters because it determines which summaries and statistical methods are appropriate. We will see both types throughout the course.

Exercise 1.2 For each of the following scenarios, identify the type of the variable of interest at the individual level.

(a) An agricultural scientist measures the weight (in grams) of individual apples harvested from an orchard.

viewof answer_sd_var_a = Inputs.radio(
  ["Numerical — weight is measured on a continuous numerical scale, and averages or differences are meaningful.",
   "Categorical — weight just places apples into groups like small, medium, or large."],
  {label: "Variable type: "}
)

(b) A biology department records the natural hair color (e.g., black, brown, blonde, red) of students enrolled in an introductory course.

viewof answer_sd_var_b = Inputs.radio(
  ["Numerical — we can count how many students have each hair color.",
   "Categorical — hair color is a descriptive label, placing each student into a category."],
  {label: "Variable type: "}
)

viewof answer_sd_var_c = Inputs.radio(
  ["Numerical — phone numbers consist entirely of digits.",
   "Categorical — phone numbers act as labels/identifiers; performing mathematical operations (like averaging) on them makes no sense."],
  {label: "Variable type: "}
)

1.1.3 What if we could measure everything? Population distribution and parameters.

If we could somehow afford to measure the crack pressure of every screen in the shipment (without destroying them), we would have a complete list of \(50{,}000\) numbers, one per screen. This complete picture is called the population distribution:

Definition 1.3 (Population Distribution) The collection of values of the variable of interest across the entire population.

(Disclaimer: this is not a formal definition of “distribution”, but it will serve us well throughout the course.)

With the population distribution in hand, we could answer any question about the shipment:

What fraction of screens crack below Apple’s \(750\) psi threshold? If it is more than \(1\%\), Apple returns the shipment.
What is the average crack pressure across all \(50{,}000\) screens?
What pressure can \(99.9\%\) of screens withstand?
- This could be useful for warranty purposes: if we know that \(99.9\%\) of screens survive at least \(X\) psi, we can offer a warranty covering any screen that cracks under \(X\) psi.

Unfortunately, the population distribution is precisely what we cannot observe directly. In this case, measuring crack pressure requires applying pressure until the screen breaks. It is a destructive test. Measure all \(50{,}000\) and you have zero screens left to put in iPhones.

This is not just a quirk of the screens problem. In virtually every real study, the population distribution is unobservable, because measuring the entire population is too expensive, too slow, ethically impossible, or, as here, physically destructive. The whole point of statistical inference is to learn something reliable about the population distribution from a small, observable piece of it.

For learning purposes, let’s play pretend. Suppose we have access to the entire shipment of \(50{,}000\) screens’ crack pressure data. In practice, we would never have access to this; but having the ground truth here lets us study how well our statistical methods work.

A histogram lets us see the overall shape: where the values concentrate, how much spread there is, and whether the distribution is symmetric or skewed.

We can see that the distribution is right-skewed (i.e., a longer tail on the right), meaning some screens are exceptionally strong, and that most screens survive well above the \(750\) psi threshold. But a small fraction (just left of the red line) do not.

Making sense of a list with \(50{,}000\) values is not easy, so having a list of \(50{,}000\) numbers is not useful in itself. What matters are specific numerical summaries that let us answer our questions, like the fraction of screens that crack below Apple’s \(750\) psi threshold, or the average crack pressure across the shipment. These numerical summaries of the population distribution are called parameters.

Definition 1.4 (Parameter) A numerical summary of the population distribution.

Parameters describe the population as a whole; they are fixed (constants) but usually unknown. Common parameters include the proportion \(p\), the mean \(\mu\), the median \(Q_{0.5}\), and the standard deviation \(\sigma\). The right choice depends on the question. For example, for the screens shipment:

“What fraction of screens fail Apple’s threshold?” → proportion \(p\)
- the fraction of all \(50{,}000\) screens with crack pressure below \(750\) psi.
“What is the typical crack pressure?” → mean \(\mu\) or median \(Q_{0.5}\)
- these are central tendency measures; the median might be more informative here because the distribution is right-skewed, and a few extremely strong screens could inflate the mean making it misrepresent a typical screen. But more importantly, it gives us a very useful interpretation: “Half of the screens crack below \(Q_{0.5}\) psi, and half above.” The mean would be harder to interpret in this context.
“How consistent is the manufacturing process?” → standard deviation \(\sigma\)
- a small \(\sigma\) means screens are uniform; a large \(\sigma\) means quality varies widely. It is related to the width of the distribution: a wider distribution has a larger \(\sigma\).
“What pressure can \(99.9\%\) of screens withstand?” → quantile \(Q_{0.001}\)
- the \(0.1\)th percentile, useful for setting warranty thresholds.

Parameters are much easier to communicate than a raw list of \(50{,}000\) numbers. Compare:

“I’m returning this shipment because \(3.2\%\) of screens crack under \(750\) psi, and our standard requires no more than \(1\%\).”

versus

“I’m returning this shipment. Here are the \(50{,}000\) crack-pressure values I collected. I don’t like them.”

The first message is immediately clear, yet it doesn’t mention the raw values at all. The second is useless, even though it contains all the data. At some point, we need to summarize the population distribution to make informed decisions.

Variable vs. parameter: a common confusion

Students very frequently confuse variables for parameters and vice-versa. The variable is what you measure on each individual screen: screen #3,471 has a crack pressure of \(803.2\) psi; screen #12,847 has \(941.7\) psi. Every screen has its own value. The parameter is a single number that summarizes the entire population: \(p = 0.032\) is the fraction of all \(50{,}000\) screens that fail. One lives at the level of the individual; the other lives at the level of the population.

1.1.3.1 Exercises

Exercise 1.3 A streaming platform wants to understand whether its users are engaging enough with the service. The business team asks: does the average daily watch time across all active subscribers exceed \(45\) minutes?

(a) What is the variable of interest, and is it numerical or categorical?

viewof answer_sd_1_1_a = Inputs.radio(
  ["The average daily watch time across all subscribers (numerical)",
   "Each subscriber's daily watch time, in minutes (numerical)",
   "Whether each subscriber watched more than 45 minutes that day (categorical)",
   "The total number of minutes streamed platform-wide (numerical)"],
  {label: "Variable of interest: "}
)

(b) What is the parameter of interest, and what symbol do we use for it?

viewof answer_sd_1_1_b = Inputs.radio(
  ["The watch time of the subscriber who watches the most (no standard symbol)",
   "The average daily watch time across all active subscribers (μ)",
   "The proportion of subscribers who exceed 45 minutes (p)",
   "The standard deviation of watch times (σ)"],
  {label: "Parameter: "}
)

Exercise 1.4 What proportion of screens in the shipment crack at or below \(750\) psi? Is this above or below Apple’s \(1\%\) maximum?

Exercise 1.5 What pressure can \(99.9\%\) of the screens in the shipment withstand? (That is, find the pressure such that only \(0.1\%\) of screens crack at or below it.)

Exercise 1.6 Samsung’s sales representative pushes back:

“Look the average crack pressure for this shipment is around \(1{,}000\) psi, well above your \(750\) psi threshold. There’s no way this shipment fails your standard.”

Is the representative’s argument convincing?

viewof answer_sd_1_4 = Inputs.radio(
  ["Yes: a mean of 1,000 psi is far above 750 psi, so the failure rate must be negligible.",
   "No: even with a high average, the population can have many screens below 750 psi.",
   "It depends: we need to know the standard deviation before deciding.",
   "Yes: the mean is the only parameter that matters for threshold decisions."],
  {label: "Your answer: "}
)

1.2 Sample

In the previous section, we established what we want to learn: the target population, the variable of interest, and the parameters that concisely describe the population distribution. But, since we cannot measure the variable of interest for every individual in the population, we collect data on a subset of the population: a sample. In this section, we introduce some new concepts: what a sample is, how to draw one to avoid systematic bias, how to summarize the data it contains, and how those summaries connect back to the parameters we actually care about. Let’s start with the definition of a sample.

Definition 1.5 (Sample) A subset of the population.

What we hope for is that this subset will represent the population well, but this is not always the case.

Imagine you are making a soup and want to know if it has enough salt. You don’t drink the whole pot; you taste a single spoonful (a sample) and extrapolate your finding to the entire pot (the population). If you stirred the pot well before tasting, the spoonful will be a great representation of the whole pot of soup.
Now imagine you are cooking a basket of French fries. You take a single piece to see if you have added enough salt. But, purely by random chance, you grab a fry on top of the basket that got too much salt on top. You conclude incorrectly that the whole basket is too salty. Here, you drew a random sample that doesn’t represent the population well (but it is still a sample)!

**Soup tasting**: Homogeneous mixing represents the population well.

**French fries**: Outliers in solid mixtures can lead to unrepresentative samples.

The french fries example illustrates this: a sample is not necessarily representative. The word “sample” simply means a subset of the population, and carries no guarantee of quality. But the situation is even trickier than that. Since we do not know the population distribution, we can never be certain whether our sample is a “good” one (i.e., represents the population well) or a “bad” one (unrepresentative).

Because we cannot assess the quality of individual samples, how we draw our samples becomes crucial. We need to develop good sampling methods that are reliable and allow us to measure our uncertainty in a principled way.

1.2.1 How do we sample? Simple Random Sampling

In statistics, there are many possible sampling strategies, each with its own advantages and disadvantages. Some of the most common include:

Simple random sampling: elements are selected one by one, ensuring that at each step of the drawing process, every remaining element of the population has an equal chance of being selected next.
Stratified sampling: the population is divided into subgroups (strata), and a random sample is drawn from each.
Cluster sampling: the population is divided into clusters, and clusters are randomly selected.

All of these methods share one key ingredient that makes them effective: randomness.

Randomness is the magical element of statistics, and it may feel counterintuitive at first. How can randomness be a good thing? Isn’t it better to be precise and deliberate? Well, to start with, randomness prevents the hand-picking that introduces bias in sample selection by ensuring that no individual or subgroup is systematically favoured or excluded. In addition, it gives us the mathematical tools to quantify how uncertain our estimates are, something no non-random method can do in a principled way.

In this course, we focus exclusively on simple random sampling (SRS). Don’t let the name fool you. SRS is a widely used method in practice, and it is surprisingly good given how “simple” it is. There are two types of SRS: with replacement and without replacement.

To draw a simple random sample of size \(n\) without replacement from a population, you:

List all elements of the population.
Select one element at random (all elements have the same probability of being selected).
Record the selected element’s value and remove it from the population.
Repeat steps 2 and 3 exactly \(n\) times.

To draw a simple random sample of size \(n\) with replacement, you follow the same procedure but do not remove the selected element from the population in Step 3. Let’s explore the difference between the two methods in the next exercise.

Exercise 1.7 (Explore: With vs. Without Replacement) Suppose we have a population consisting of a bag of \(N = 8\) colored, numbered balls. We want to select a random sample of size \(n = 6\). Using the interactive simulator below, draw a sample under both schemes and observe how the two methods behave.

Instructions:

Click the “Draw next ball” button to draw balls one by one.
Watch the “Population (bag)” on the left (With Replacement) and the right (Without Replacement).
Draw all 6 balls, then answer the questions below.

mutable rwr_draw_state = ({
  sampleWith:    [],
  sampleWithout: [],
  remaining:     [0,1,2,3,4,5,6,7],
  nDrawn:        0,
  lastWith:      null,
  lastWithout:   null
})

{
  const MAX_DRAWS = 6;
  const state  = rwr_draw_state;
  const COLORS = [
    "#e05252","#e5834e","#e0be45","#6fc94f",
    "#4eb8e5","#4e78e5","#8b4ee5","#c44eb8"
  ];

  function ballHTML(idx, { grayed = false, highlighted = false, size = 40 } = {}) {
    const opacity = grayed ? 0.18 : 1.0;
    const border  = highlighted ? "3px solid #1a1a1a" : "2px solid rgba(0,0,0,0.12)";
    const shadow  = highlighted
      ? "0 0 0 3px rgba(255,255,255,0.8), 0 0 12px rgba(0,0,0,0.5)"
      : "0 2px 4px rgba(0,0,0,0.18)";
    return (
      `<span style="display:inline-flex;align-items:center;justify-content:center;` +
      `width:${size}px;height:${size}px;border-radius:50%;background:${COLORS[idx]};` +
      `color:white;font-weight:700;font-size:${size <= 32 ? 12 : 14}px;border:${border};` +
      `box-shadow:${shadow};opacity:${opacity};margin:3px;` +
      `text-shadow:0 1px 2px rgba(0,0,0,0.25);flex-shrink:0;">${idx + 1}</span>`
    );
  }

  const bagWithHTML = [0,1,2,3,4,5,6,7]
    .map(i => ballHTML(i, { highlighted: i === state.lastWith }))
    .join("");

  const bagWithoutHTML = [0,1,2,3,4,5,6,7]
    .map(i => ballHTML(i, {
      grayed:      !state.remaining.includes(i),
      highlighted: i === state.lastWithout
    }))
    .join("");

  const noDrawsMsg =
    `<span style="color:#bbb;font-size:0.82em;line-height:36px;">No balls drawn yet</span>`;

  const sampleWithHTML = state.sampleWith.length === 0 ? noDrawsMsg
    : state.sampleWith.map((i, pos) =>
        ballHTML(i, { size: 36, highlighted: pos === state.sampleWith.length - 1 })
      ).join("");

  const sampleWithoutHTML = state.sampleWithout.length === 0 ? noDrawsMsg
    : state.sampleWithout.map((i, pos) =>
        ballHTML(i, { size: 36, highlighted: pos === state.sampleWithout.length - 1 })
      ).join("");

  const hasDuplicate  = state.sampleWith.length > new Set(state.sampleWith).size;
  const duplicateBall = hasDuplicate
    ? [...state.sampleWith].find((v, i, arr) => arr.indexOf(v) !== i)
    : null;
  const canDraw = state.nDrawn < MAX_DRAWS && state.remaining.length > 0;

  const activeBtn = (color) =>
    `padding:8px 18px;background:${color};color:white;border:none;border-radius:6px;` +
    `cursor:pointer;font-size:13.5px;font-weight:600;`;
  const disabledBtn =
    "padding:8px 18px;background:#adb5bd;color:white;border:none;border-radius:6px;" +
    "cursor:not-allowed;font-size:13.5px;font-weight:600;";
  const bagBox =
    "display:flex;flex-wrap:wrap;justify-content:center;align-items:center;" +
    "padding:8px;border:1px dashed #d0d0d0;border-radius:8px;min-height:54px;" +
    "background:var(--bs-light-bg-subtle,#f8f9fa);";
  const dupMsg = hasDuplicate
    ? `<div style="text-align:center;color:#c0392b;font-size:0.88em;font-weight:600;margin-top:10px;">` +
      `⚠ Ball ${duplicateBall + 1} was drawn more than once: this can only happen with replacement!</div>`
    : "";
  const limitMsg = (!canDraw && state.nDrawn >= MAX_DRAWS)
    ? `<div style="text-align:center;color:#888;font-size:0.83em;margin-top:6px;">` +
      `Maximum draws reached. Click Reset to start over.</div>`
    : "";

  const widget = html`<div style="margin:16px 0;">
    <div style="display:flex;margin-bottom:12px;align-items:stretch;">

      <div style="flex:1;padding:16px;border:2px solid #3498db;border-radius:12px;margin:0 8px;background:var(--bs-body-bg,#fff);min-width:0;">
        <h4 style="text-align:center;margin:0 0 12px;color:#3498db;font-size:0.97em;">
          Sampling <em>With</em> Replacement
        </h4>
        <div style="font-size:0.78em;color:#888;margin-bottom:4px;text-align:center;">Population (bag)</div>
        <div style="${bagBox}">${bagWithHTML}</div>
        <div style="text-align:center;font-size:1.5em;color:#bbb;line-height:1.8;">↓</div>
        <div style="font-size:0.78em;color:#888;margin-bottom:4px;text-align:center;">
          Sample drawn so far (n&nbsp;=&nbsp;${state.nDrawn})
        </div>
        <div style="${bagBox};min-height:48px;">${sampleWithHTML}</div>
        <div style="font-size:0.75em;color:#3498db;margin-top:8px;text-align:center;">
          Each ball is <strong>returned</strong> to the bag before the next draw.
        </div>
      </div>

      <div style="flex:1;padding:16px;border:2px solid #e67e22;border-radius:12px;margin:0 8px;background:var(--bs-body-bg,#fff);min-width:0;">
        <h4 style="text-align:center;margin:0 0 12px;color:#e67e22;font-size:0.97em;">
          Sampling <em>Without</em> Replacement
        </h4>
        <div style="font-size:0.78em;color:#888;margin-bottom:4px;text-align:center;">Population (bag)</div>
        <div style="${bagBox}">${bagWithoutHTML}</div>
        <div style="text-align:center;font-size:1.5em;color:#bbb;line-height:1.8;">↓</div>
        <div style="font-size:0.78em;color:#888;margin-bottom:4px;text-align:center;">
          Sample drawn so far (n&nbsp;=&nbsp;${state.nDrawn})
        </div>
        <div style="${bagBox};min-height:48px;">${sampleWithoutHTML}</div>
        <div style="font-size:0.75em;color:#e67e22;margin-top:8px;text-align:center;">
          Each ball is <strong>removed</strong> from the bag after being drawn.
        </div>
      </div>

    </div>
    <div style="display:flex;justify-content:center;align-items:center;gap:14px;flex-wrap:wrap;">
      <button id="rwr-draw-btn" style="${canDraw ? activeBtn('#0284c7') : disabledBtn}">
        Draw next ball
      </button>
      <span style="font-size:13px;color:#666;">${state.nDrawn} / ${MAX_DRAWS} draws</span>
      <button id="rwr-reset-btn" style="${activeBtn('#6c757d')}">↺ Reset</button>
    </div>
    ${dupMsg}
    ${limitMsg}
  </div>`;

  widget.querySelector("#rwr-draw-btn").addEventListener("click", () => {
    if (!canDraw) return;
    const s          = rwr_draw_state;
    const idxWith    = Math.floor(Math.random() * 8);
    const rem        = s.remaining;
    const pickPos    = Math.floor(Math.random() * rem.length);
    const idxWithout = rem[pickPos];
    mutable rwr_draw_state = {
      sampleWith:    [...s.sampleWith, idxWith],
      sampleWithout: [...s.sampleWithout, idxWithout],
      remaining:     rem.filter((_, i) => i !== pickPos),
      nDrawn:        s.nDrawn + 1,
      lastWith:      idxWith,
      lastWithout:   idxWithout
    };
  });

  widget.querySelector("#rwr-reset-btn").addEventListener("click", () => {
    mutable rwr_draw_state = {
      sampleWith:    [],
      sampleWithout: [],
      remaining:     [0,1,2,3,4,5,6,7],
      nDrawn:        0,
      lastWith:      null,
      lastWithout:   null
    };
  });

  return widget;
}

(a) Run the simulation a few times. What is a key consequence of sampling with replacement that can never occur when sampling without replacement?

viewof answer_srs_sim_a = Inputs.radio(
  ["You can draw the exact same individual multiple times, leading to duplicate values in your sample.",
   "You can never draw the exact same individual twice.",
   "The population size decreases after each draw.",
   "The sample size is allowed to exceed the population size."],
  {label: "Consequence: "}
)

(b) Based on your observations, which sampling method is more efficient for gathering new information about a population?

viewof answer_srs_sim_b = Inputs.radio(
  ["Sampling with replacement, because keeping the population unchanged preserves the true probabilities.",
   "Sampling without replacement, because each drawn individual is guaranteed to be a new, unique member of the population.",
   "They are equally efficient because both are random."],
  {label: "More efficient method: "}
)

(c) Suppose we decide to draw a simple random sample of size \(n = 8\) without replacement, what will the resulting sample look like?

viewof answer_srs_sim_c = Inputs.radio(
  ["It will contain duplicates of some balls while missing others, just like sampling with replacement.",
   "It will consist of exactly the same 8 balls as the population.", 
   "Impossible to tell because the sample is random. It could be anything."],
  {label: "Without replacement (n = N): "}
)

□

In SRS with replacement, we can select the same element multiple times (because the element is not removed from the population). If we select the same element more than once, we don’t learn anything new about the population from those repeated selections. Hence, sampling with replacement is less efficient than sampling without replacement. Therefore, we always use sampling without replacement when sampling from the population.

So, why did we discuss sampling with replacement at all?

As it turns out, sampling with replacement has a key advantage: it doesn’t change the population after each draw, making the draws independent of one another.

Independent samples is another key concept assumed by most statistical methods we will learn in this course. The draws in a sample are independent if the selection of one element does not affect the probability of selecting any other element. As it turns out, SRS without replacement yields draws that are not strictly independent, because once we select an element, it is removed from the population, which slightly changes the probabilities for the remaining elements. Fortunately, if the population size, \(N\), is much larger than the sample size, \(n\), (e.g., \(N > 10n\)), the violation of independence is negligible, and we can treat the sample as “approximately independent” for practical purposes (although, technically speaking, there’s no such thing as “approximately independent”). Let’s see an example.

Example 1.2 Consider a box with \(6\) balls, \(3\) red and \(3\) blue.

We want to draw a sample of size \(n = 3\) balls without replacement. Let’s check the probability that the third ball is red:

First two balls selected	Chance the third ball is red
Blue, Blue	\(3/4 = 0.750\)
Blue, Red	\(2/4 = 0.500\)
Red, Red	\(1/4 = 0.250\)

As you can see, the probability of drawing a red ball on the third draw depends on what we drew in the first two draws. This means that the draws are not independent (i.e., the outcome of one draw affects the probabilities of the next draw).

But let’s check what happens when we have a much larger population with the same proportion of red balls. Say our box has \(10{,}000\) balls, where \(5{,}000\) are red and \(5{,}000\) are blue. We want to draw a sample of size \(n = 3\) balls without replacement. The probability that the third ball is red is:

First two balls selected	Chance the third ball is red
Blue, Blue	\(5{,}000/9{,}998 \approx 0.5001\)
Blue, Red	\(4{,}999/9{,}998 = 0.5000\)
Red, Red	\(4{,}998/9{,}998 \approx 0.4999\)

Again, the probabilities differ based on the first two draws, so the draws are not strictly independent, but the change in probabilities is very small. If we assume independence in our calculations, these tiny changes wouldn’t affect our results in any meaningful way. For this reason, in practice, if the sample size is small relative to the population size, we can treat the draws as independent even when sampling without replacement. How small is “small”? A common rule of thumb is that if the population size is at least 10 times larger than the sample size (\(N > 10n\)).

□

Rules of Thumb in Statistics

We will encounter several widely used “rules of thumb” throughout this course. Many statisticians disagree with some of these rules, as they are often generic and highly subjective. The \(N > 10n\) guideline above is one such example.

Nonetheless, we present them because they are common in practice and can be helpful for quick, initial checks. Just remember: these are guidelines, not mathematical laws, and you should always apply them with caution.

Let us return to our running example, and draw a random sample of \(n = 300\) screens from the shipment of \(50{,}000\). We will use the slice_sample() function from the dplyr package to draw our sample. (Note: remember, our population of screens is stored in the screens_pop data frame).

At this point, we have a sample of \(300\) screens. In practice, we would only have access to the data in screens_sample, and we would not know the true population distribution (which is stored in screens_pop).

Now that we have our sample, how do we use the sample to learn about the population distribution and its parameters?

1.2.2 Sample distribution and statistics

Think of your sample as all the information you have about the population. The information is not perfect (because it is just a small, observable piece of the population), but it is all you have to work with. So, if you are interested in the population distribution, the best you can do is to look at the distribution of the variable of interest within your sample. This is what we call the sample distribution.

Definition 1.6 (Sample Distribution) The distribution of the variable of interest within a given sample.

The sample distribution is something we can observe and plot, but it changes every time we take a new sample because the sample is random. The population distribution, by contrast, is fixed (it never changes, since the population is fixed) but it is unobservable.

Once again, the sample distribution is not the same as the population distribution and it can look quite different from the population distribution (depending on the random sample you take). Encouragingly, as the sample size increases, the sample distribution tends to look more and more like the population distribution. Let’s explore this convergence in action!

Exercise 1.8 (Explore: Effect of Sample Size on the Sample Distribution) Use the interactive simulator below to draw random samples of different sizes \(n\) from the screen durability population (screens_pop), and observe how the sample distribution behaves relative to the true population distribution.

Instructions:

Set the sample size (\(n\)) slider to a small value (like \(n = 10\)). Click “↺ New sample” multiple times. Notice how much the blue histogram (sample distribution) changes with each click, and how different it looks from the red line (population distribution).
Now, increase the slider to a large value (like \(n = 1000\) or \(n = 2000\)). Click “↺ New sample” a few times. Observe the shape of the blue histogram and the calculated sample statistics (\(\bar{x}\) and the \(\%\) below threshold).
Answer the questions below.

mutable cvg_state = ({ n: 100, seed: 1 })

{
  // Population parameters match screens_pop: rlnorm(meanlog=6.9, sdlog=0.15)
  const MEANLOG = 6.9, SDLOG = 0.15, THRESHOLD = 750;
  const BIN_WIDTH = 30, X_MIN = 550, X_MAX = 1700;
  const W = 660, H = 400;
  const margin = { top: 74, right: 24, bottom: 56, left: 60 };

  // Mulberry32 PRNG, deterministic, seeded
  function mulberry32(seed) {
    let a = seed | 0;
    return () => {
      a = (a + 0x6D2B79F5) | 0;
      let t = Math.imul(a ^ (a >>> 15), 1 | a);
      t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
      return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
    };
  }

  // Box-Muller log-normal sample; same seed → first n obs are identical as n grows
  function sampleLogNormal(n, seed) {
    const rng = mulberry32(seed);
    const out = [];
    for (let i = 0; i < n; i++) {
      const u1 = Math.max(rng(), 1e-10), u2 = rng();
      out.push(Math.exp(MEANLOG + SDLOG * Math.sqrt(-2 * Math.log(u1)) * Math.cos(2 * Math.PI * u2)));
    }
    return out;
  }

  const lnPDF = x => x > 0
    ? Math.exp(-Math.pow(Math.log(x) - MEANLOG, 2) / (2 * SDLOG * SDLOG)) / (x * SDLOG * Math.sqrt(2 * Math.PI))
    : 0;

  // Standard normal CDF (Abramowitz & Stegun approximation, |error| < 1.5e-7)
  function normalCDF(z) {
    const a = [0.254829592, -0.284496736, 1.421413741, -1.453152027, 1.061405429];
    const p = 0.3275911, sign = z < 0 ? -1 : 1;
    const xv = Math.abs(z) / Math.sqrt(2);
    const t = 1 / (1 + p * xv);
    const erf = 1 - ((((a[4]*t + a[3])*t + a[2])*t + a[1])*t + a[0]) * t * Math.exp(-xv * xv);
    return 0.5 * (1 + sign * erf);
  }

  const { n, seed } = cvg_state;
  const sample = sampleLogNormal(n, seed);

  // Histogram: proportion per bin (fraction of sample in each 30-psi interval)
  const edges = d3.range(X_MIN, X_MAX + BIN_WIDTH, BIN_WIDTH);
  const bins = d3.bin().domain([X_MIN, X_MAX]).thresholds(edges)(sample);
  const histData = bins.map(b => ({ x0: b.x0, x1: b.x1, y: b.length / n }));

  // PDF curve scaled to the same proportion-per-bin scale: P(X in [x, x+30]) ≈ f(x)·30
  const pdfData = d3.range(X_MIN, X_MAX, 3).map(xx => ({ x: xx, y: lnPDF(xx) * BIN_WIDTH }));

  // Live statistics
  const sampleMean = Math.round(d3.mean(sample));
  const fracBelow  = (sample.filter(v => v < THRESHOLD).length / n * 100).toFixed(1);
  const popMean    = Math.round(Math.exp(MEANLOG + SDLOG * SDLOG / 2));
  const popFrac    = (normalCDF((Math.log(THRESHOLD) - MEANLOG) / SDLOG) * 100).toFixed(1);

  // Scales
  const x = d3.scaleLinear().domain([X_MIN, X_MAX]).range([margin.left, W - margin.right]);
  const yMax = Math.max(d3.max(histData, d => d.y), d3.max(pdfData, d => d.y)) * 1.2;
  const y = d3.scaleLinear().domain([0, yMax]).range([H - margin.bottom, margin.top]);

  // Controls
  const sliderEl = html`<input type="range" min="10" max="2000" step="10" value="${n}"
    style="width:210px;accent-color:#0284c7;cursor:pointer;vertical-align:middle;">`;
  const nLabel = html`<span style="font-weight:700;color:#0284c7;min-width:44px;display:inline-block;text-align:right;">${n}</span>`;
  const drawBtn = html`<button style="padding:5px 15px;background:#0284c7;color:#fff;border:none;
    border-radius:6px;font-size:13px;font-weight:600;cursor:pointer;
    box-shadow:0 2px 8px rgba(2,132,199,0.22);">↺ New sample</button>`;

  sliderEl.addEventListener("input", () => { nLabel.textContent = +sliderEl.value; });
  sliderEl.addEventListener("change", () => { mutable cvg_state = ({ n: +sliderEl.value, seed }); });
  drawBtn.addEventListener("click", () => { mutable cvg_state = ({ n, seed: seed + 1 }); });

  const controls = html`<div style="display:flex;align-items:center;gap:14px;flex-wrap:wrap;margin-bottom:10px;">
    <label style="font-size:14px;font-weight:600;color:var(--bs-body-color);white-space:nowrap;">
      Sample size (n): ${nLabel}
    </label>
    ${sliderEl}
    ${drawBtn}
  </div>`;

  // SVG
  const svg = d3.create("svg").attr("width", W).attr("height", H)
    .style("font-family", "sans-serif").style("overflow", "visible");

  svg.append("text").attr("x", W/2).attr("y", margin.top - 46)
    .attr("text-anchor","middle").attr("font-size","15px").attr("font-weight","700")
    .attr("fill","var(--bs-body-color,#111)")
    .text(`Sample distribution (n = ${n}) vs. Population distribution`);

  svg.append("text").attr("x", W/2).attr("y", margin.top - 23)
    .attr("text-anchor","middle").attr("font-size","11.5px").attr("fill","#888")
    .text(`Sample mean x̄ = ${sampleMean} psi   |   Population mean μ ≈ ${popMean} psi`);

  // Histogram bars
  svg.append("g").selectAll("rect").data(histData).join("rect")
    .attr("x", d => x(d.x0) + 1)
    .attr("width", d => Math.max(0, x(d.x1) - x(d.x0) - 2))
    .attr("y", d => y(d.y))
    .attr("height", d => Math.max(0, y(0) - y(d.y)))
    .attr("fill","steelblue").attr("opacity",0.55);

  // Population PDF curve
  svg.append("path").datum(pdfData)
    .attr("fill","none").attr("stroke","tomato").attr("stroke-width",2.5)
    .attr("d", d3.line().x(d => x(d.x)).y(d => y(d.y)).curve(d3.curveBasis));

  // Sample mean line (x̄)
  svg.append("line")
    .attr("x1",x(sampleMean)).attr("x2",x(sampleMean))
    .attr("y1",margin.top).attr("y2",H-margin.bottom)
    .attr("stroke","steelblue").attr("stroke-width",2).attr("stroke-dasharray","4 4");
  svg.append("text").attr("x",x(sampleMean)+5).attr("y",margin.top+15)
    .attr("text-anchor","start").attr("font-size","11px").attr("fill","steelblue")
    .text(`x̄ = ${sampleMean}`);

  // Legend
  const lx = W - margin.right - 185, ly = margin.top + 12;
  svg.append("rect").attr("x",lx).attr("y",ly).attr("width",14).attr("height",14)
    .attr("fill","steelblue").attr("opacity",0.55);
  svg.append("text").attr("x",lx+18).attr("y",ly+11.5).attr("font-size","12px")
    .attr("fill","var(--bs-body-color,#333)").text("Sample distribution");
  svg.append("line").attr("x1",lx).attr("x2",lx+14).attr("y1",ly+30).attr("y2",ly+30)
    .attr("stroke","tomato").attr("stroke-width",2.5);
  svg.append("text").attr("x",lx+18).attr("y",ly+34).attr("font-size","12px")
    .attr("fill","var(--bs-body-color,#333)").text("Population distribution");

  // Axes
  svg.append("g").attr("transform",`translate(0,${H-margin.bottom})`).call(d3.axisBottom(x).ticks(8));
  svg.append("text").attr("x",W/2).attr("y",H-6).attr("text-anchor","middle").attr("font-size","13px")
    .text("Crack Pressure (psi)");

  svg.append("g").attr("transform",`translate(${margin.left},0)`)
    .call(d3.axisLeft(y).ticks(5).tickFormat(d3.format(".3f")));
  svg.append("text").attr("transform","rotate(-90)").attr("x",-(H/2)).attr("y",14)
    .attr("text-anchor","middle").attr("font-size","13px")
    .text("Proportion per 30-psi bin");

  const container = html`<div style="margin:8px 0 12px;"></div>`;
  container.append(controls);
  container.append(svg.node());
  return container;
}

(a) As you increase the sample size \(n\) using the slider, what happens to the shape of the sample distribution (blue histogram) relative to the population distribution (red curve)?

viewof answer_conv_a = Inputs.radio(
  ["The sample distribution fluctuates widely and shows no clear shape.",
   "The sample distribution looks more and more like the population distribution, and the sample statistics get closer to the population parameters.",
   "The sample distribution becomes much narrower than the population distribution.",
   "The sample distribution becomes completely flat."],
  {label: "Your observation: "}
)

(b) Click ↺ New sample several times first at \(n = 10\) and then at \(n = 2000\). Watch the steelblue dashed line (\(\bar{x}\)). How does the sample mean (\(\bar{x}\)) behave at these two sizes?

viewof answer_conv_b = Inputs.radio(
  ["The sample mean line (x̄) jumps around wildly between clicks at both sizes.",
   "The sample mean line (x̄) jumps around wildly when n = 10, but is very stable (sits near the center of the distribution) when n = 2000.",
   "The sample mean line (x̄) is stable when n = 10, but jumps around wildly when n = 2000.",
   "The sample mean line (x̄) never changes at either size."],
  {label: "Sample mean line behavior: "}
)

(c) As the sample size \(n\) increases from \(10\) to \(2000\), what happens to the overall spread (width) of the blue histogram?

viewof answer_conv_c = Inputs.radio(
  ["It becomes narrower and narrower, concentrating all data points near the center.",
   "It becomes wider and wider, because more random values are being included.",
   "Its width remains stable, reflecting the true variation of the population.",
   "There's substantial random variation of width even for large samples."],
  {label: "Histogram spread behavior: "}
)

□

Just as we compute summaries of the population distribution (parameters) to concisely describe it, we can also compute summaries of the sample distribution. Summaries of the sample distribution are called statistics, and they are used to estimate population parameters.

Definition 1.7 (Statistics, estimators, and estimates) A statistic is a numerical summary computed from sample data. When used to estimate a population parameter, a statistic is called an estimator. The specific value the estimator takes in a particular sample is called an estimate.

These three terms are easy to blend together in prose, so it helps to see them side by side for a single concrete quantity:

Concept	Role	Example (screens shipment)
Parameter	Fixed, unknown number describing the population	Population mean crack pressure, \(\mu\)
Estimator	The statistic used to estimate the parameter	Sample mean, \(\bar{X}\)
Estimate	The specific number the estimator produces for this sample	\(\bar{x} = 1{,}012\) psi

Remember, the population parameter is a fixed number that we want to learn about, but we cannot observe it directly. A statistic depends on the sample, which is random, so statistics are also random. Usually, we perform the same computation on our sample that we would on the population to calculate the parameter. For example, if we want to estimate the population mean, \(\mu\), we compute the sample mean \(\bar{X}\), which is the average of the variable of interest in our sample. If we want to estimate the population variance, \(\sigma^2\), we compute the sample variance \(S^2\).

For the screens shipment, let’s compute the sample mean \(\bar{X}\): the average crack pressure of the \(300\) sampled screens. We would use this value as our estimate of the unknown population mean \(\mu\).

In this case, the estimator is the sample mean \(\bar{X}\), and the estimate is the value we computed: sample_mean_cp.

1.2.3 Population vs. Sample: The Big Picture

Before moving to the exercises, let us consolidate everything introduced in this section. Each concept we use to describe the population has a direct counterpart in the sample.

The Group

Population

The entire collection of individuals or items that we are interested in studying.

Fixed Unobservable

➔ sampled
to obtain

Sample

The subset of the population actually selected, observed, and measured.

Random Observable

The Data Distribution

Population Distribution

The pattern and spread of values across the entire population.

Fixed Unknown

➔ approximated
by

Sample Distribution

The pattern and spread of values observed within the collected sample.

Random Observable

The Summary Measures

Parameter

A single numerical value summarizing the population distribution (e.g., population mean μ or proportion p).

Fixed Unknown

➔ estimated
by

Statistic / Estimator

A single numerical value calculated directly from the sample data (e.g., sample mean x̄ or proportion p̂).

Random Observable

Exercise 1.9 (Match the Population and Sample Concepts) Let’s put this mapping into practice. Suppose a transit agency wants to estimate the true proportion of all registered voters in Vancouver who support a new light rail proposal. They randomly select and contact \(1{,}000\) registered voters, and find that \(58\%\) of these surveyed voters support the proposal.

Match the corresponding population and sample concepts by dragging the cards from the Top Deck and dropping them into their correct roles in the Population or Sample columns below.

Top Deck (Drag cards from here)

Support/oppose status across registered voters in Vancouver

The 1,000 surveyed voters

The proportion of registered voters in Vancouver who support the proposal.

Support/oppose status across the 1,000 respondents of the survey.

All registered voters in Vancouver

58% of the respondents support the proposal.

Population

Spacer

Sample

0 of 3 counterparts matched

Exercise 1.10 (Sorting Properties) Now let’s verify if you can classify various quantities based on whether they describe the population (fixed but unknown) or the sample (random but observable).

Play the categorization game below to test your understanding.

0 of 6 cards sorted

1.2.4 Exercises

Exercise 1.11 A nutritionist is studying the daily fruit intake (in servings) of university students. She recruits \(n = 80\) students from the university cafeteria during lunch.

(a) Compute the sample mean fruit intake. This is the estimate of the population mean \(\mu\).

sample_mean_fruit <- mean(nutrition_sample$fruit_servings)
cat("Sample mean:", sample_mean_fruit)

(b) Compute the sample median and sample standard deviation.

nutrition_sample |>
  summarise(
    median  = median(fruit_servings),
    std_dev = sd(fruit_servings)
  )

viewof answer_sd_2_1_c = Inputs.radio(
  ["The sample mean fruit intake of the 80 students",
   "The mean fruit intake of all university students",
   "The fruit intake of student #42",
   "The standard deviation of the fruit intake of the 80 students"],
  {label: "Parameter: "}
)

Exercise 1.12 Below, four students each draw a random sample from the same population and compute the sample mean. Their results are: \(\bar{x}_1 = 47.3\), \(\bar{x}_2 = 51.8\), \(\bar{x}_3 = 44.9\), \(\bar{x}_4 = 49.6\).

(a) The four students all computed different estimates. Is this expected?

viewof answer_sd_2_2_a = Inputs.radio(
  ["Yes: different random samples almost always give different estimates",
   "No: all samples from the same population should give the same estimate",
   "It depends on the sample size",
   "It depends on the parameter being estimated"],
  {label: "Answer: "}
)

(b) Do any of these estimates equal the true population parameter?

We do not know, and in practice we never know, how close our estimate happens to be to the true parameter. This is the fundamental challenge of statistical inference. The sample estimates are (hopefully!) close to the truth, but essentially never exactly equal to it. Our job is to quantify how close they are likely to be.

Exercise 1.13 A political scientist surveys residents of Calgary to estimate the proportion of Calgarians who prefer cycling to driving for commuting. She recruits participants by standing outside shopping malls on weekday afternoons.

(a) What is the target population?

viewof answer_sd_2_3_a = Inputs.radio(
  ["All residents of Calgary",
   "People who shop at malls in Calgary",
   "People who commute by bicycle",
   "The participants in the survey"],
  {label: "Target population: "}
)

(b) Is there a potential problem with this study design?

Yes. The sampled population (mall visitors on weekday afternoons) is unlikely to represent all Calgary residents. People who visit malls on weekday afternoons may be retired, unemployed, or work shift jobs, groups with potentially different commuting habits than, say, 9-to-5 office workers who may never visit a mall on a weekday afternoon. The results of this survey may not generalize to all Calgarians.

Exercise 1.14 For each concept in the left column, use the dropdown to select its sample counterpart.

viewof match_sd_2_4_a = Inputs.select(
  ["(select)", "Sample", "Sample distribution", "Statistic / Estimate (x̄, p̂, s, ...)"],
  {label: html`<em>Population</em> &rarr;`}
)

viewof match_sd_2_4_b = Inputs.select(
  ["(select)", "Sample", "Sample distribution", "Statistic / Estimate (x̄, p̂, s, ...)"],
  {label: html`<em>Population distribution</em> &rarr;`}
)

viewof match_sd_2_4_c = Inputs.select(
  ["(select)", "Sample", "Sample distribution", "Statistic / Estimate (x̄, p̂, s, ...)"],
  {label: html`<em>Parameter (&mu;, p, &sigma;, &hellip;)</em> &rarr;`}
)

Exercise 1.15 (a) The population mean \(\mu\) is best described as:

viewof answer_sd_2_5_a = Inputs.radio(
  ["Fixed and unknown: we cannot directly observe it",
   "Fixed and known: it is a definite number we can look up",
   "Random (varies by sample) and unknown: it changes every time we sample",
   "Random (varies by sample) and known: we can compute it from our data"],
  {label: "The population mean μ is: "}
)

(b) The sample mean \(\bar{X}\) is best described as:

viewof answer_sd_2_5_b = Inputs.radio(
  ["Fixed (constant) and unknown: it is a single definite number",
   "Fixed (constant) and known: we can compute it from the data",
   "Random (varies by sample) and unknown: it changes but we cannot see it",
   "Random (varies by sample) and known: we compute it, but a different sample gives a different value"],
  {label: "The sample mean x̄ is: "}
)

(c) Apple’s quality-control manager uses the sample of \(300\) screens and reports: “The sample mean crack pressure is \(\bar{x} = 1{,}012\) psi, so the average crack pressure of all \(50{,}000\) screens is exactly \(\mu = 1{,}012\) psi.” What is wrong with this statement?

viewof answer_sd_2_5_c = Inputs.radio(
  ["Nothing!",
   "The sample mean is an estimate of μ, not μ itself; a different sample of 300 screens would almost certainly give a different value.",
   "The sample mean is always higher than the population mean, so μ < 1,012 psi.",
   "The sample is too small to compute a reliable mean."],
  {label: "What is wrong: "}
)

1.3 Take-home points

The target population is the complete group we want to learn about; the variable of interest is what we measure on each individual member. A parameter is a fixed (but usually unknown) numerical summary of the population distribution. Common parameters are the proportion \(p\), the mean \(\mu\), the median \(Q_{0.5}\), and the standard deviation \(\sigma\).
A sample is a subset of the population. It carries no guarantee of being representative, that is why how we sample matters. Simple random sampling (SRS) selects elements one at a time, each with equal probability, and comes in two flavours:
- without replacement: more efficient (no wasted repeat draws), but strictly speaking violates independence between draws;
- with replacement: less efficient, but keeps the draws independent. A rule of thumb is that if the population size \(N\) is at least \(10\) times the sample size \(n\) (\(N > 10n\)), the violation from sampling without replacement is negligible.
The sample distribution is the distribution of the variable of interest within a given sample. Unlike the population distribution, it is observable, but it changes every time we draw a new sample, and it is generally not the same as the population distribution (though it converges to it as \(n\) grows).
A statistic is a numerical summary computed from sample data; when used to estimate a parameter, it is called an estimator, and the specific value it takes in a given sample is an estimate. Because the sample is random, a statistic is a random variable: its value changes from sample to sample, even though it is always computable (observable) from the data at hand.
This randomness in the statistic is called sampling variability, and it is unavoidable: any single estimate could be a lucky or unlucky draw. The next chapter asks the natural follow-up question: how do we tell whether a given estimate is trustworthy?