Lesson 18: Central Limit Theorem

Calendar

Block II calendar with a red box around Tuesday 6 October, Lesson 18, Central Limit Theorem.

Block II: Statistical Inference

  • Block I: we knew the distribution and found probabilities.
  • Block II: we have a sample and say something about \(\mu\) or \(p\).
  • Today is the bridge: how a sample mean behaves.

What We’re Doing: Lesson 18

Objectives

  • Describe the sampling distribution of a statistic, including the sample mean \(\bar{X}\). (SLO 3)
  • State and apply the Central Limit Theorem to approximate the distribution of \(\bar{X}\) and compute probabilities for \(\bar{X}\). (SLO 5)
  • Apply the large-sample normal approximation to the sample proportion \(\hat{p} = X/n\). (SLO 5)

Required Reading

Devore 5.3, 5.4

Break!

Family

DMath Frisbee!!

Math vs DPE

3-0

12-7

Math vs CBSE

4-0

15-3

Math vs CBSE

5-0

15-2

Math vs DSE

6-0

15-3

Math vs EECS/PANE

7-0

15-3

Math vs BSL/SCPME

8-0

15-5

Course Lecture Follow Up

COL Tim Sikora.

The Takeaway for Today

NoteKey Concepts from Lesson 18
  • Central Limit Theorem: \(\bar{X}\) is approximately \(N\left(\mu,\ \left(\dfrac{\sigma}{\sqrt{n}}\right)^2\right)\), whatever the population looks like
  • Large enough: \(n > 40\). A normal population works for any \(n\)
  • pnorm(x, mu, sigma/sqrt(n)). The third slot is \(\sigma/\sqrt{n}\), not \(\sigma\)
  • \(\hat{p} = X/n\) is approximately \(N\left(p,\ \dfrac{p(1-p)}{n}\right)\) when \(np \ge 10\) and \(n(1-p) \ge 10\)

Last Block: Distributions

story \(X\) distribution
\(10\) free throws, \(0.75\) each number made \(X \sim \text{Binom}(n = 10,\ p = 0.75)\)
\(6\) cadets per hour arrive number in an hour \(X \sim \text{Pois}(\mu = 6)\)
\(6\) cadets per hour arrive hours until the next one \(X \sim \text{Exp}(\lambda = 6)\)
Heights: mean \(70\) in, sd \(3\) in one cadet’s height \(X \sim N(\mu = 70,\ \sigma^2 = 3^2)\)

Four panels. The Binom(10, 0.75) pmf, bars from 0 to 10 peaking at 8. The Pois(6) pmf, bars peaking at 5 and 6 with a right tail. The Exp(6) pdf, starting high at 0 and decaying. The N(70, 3 squared) pdf, a bell centered at 70.

And Some Without Names

Discrete: tardies in a week

\[p(x) = \begin{cases} 0.4 & x = 0 \\ 0.3 & x = 1 \\ 0.2 & x = 2 \\ 0.1 & x = 3 \\ 0 & \text{otherwise} \end{cases}\]

Continuous: hours of study on a weeknight

\[f(x) = \begin{cases} \dfrac{x}{8} & 0 \le x \le 4 \\ 0 & \text{otherwise} \end{cases}\]

Two panels. Left, a pmf with bars of height 0.4, 0.3, 0.2, and 0.1 at 0, 1, 2, and 3. Right, a pdf that rises in a straight line from 0 at x = 0 to 0.5 at x = 4, with a labeled y axis. The pmf bars carry their probabilities as labels.

Without any further ado…

(The Big Reveal)

Tom Brady is the Central Limit Theorem of football.

The Central Limit Theorem

Just like arrival rates have a distribution, and so does shooting free throws, so does \(\bar{X}\).

What does that mean? If we resample from a population over and over again, the sample mean will follow a distribution.

ImportantThe Distribution of the Sample Mean

\[\bar{X} \sim N\left(\mu,\ \left(\frac{\sigma}{\sqrt{n}}\right)^2\right)\]

  • The mean is the mean of the underlying distribution, \(\mu\).
  • The standard deviation is the standard deviation of the underlying distribution divided by the square root of the sample size, \(\dfrac{\sigma}{\sqrt{n}}\).

So what does this mean?

Imagine you don’t know anything about the underlying distribution. Take cadet 2 mile run times. We don’t actually know how they are distributed.

Say I sample 50 cadets.

A histogram of 50 cadet 2 mile run times. Most are bunched between 13 and 15 minutes with a long tail stretching right past 19 minutes. It is clearly not bell shaped.

But I care about the average cadet run time. How is that distributed?

Let’s plot the average of those 50.

One bar: the average run time of the first sample of 50 cadets.

Now resample. 10 times.

Ten sample averages scattered around 15 minutes.

100 times.

One hundred sample averages forming a rough mound centered near 14.9 minutes.

1,000 times.

One thousand sample averages forming a bell shape centered near 14.9 minutes.

10,000 times.

Ten thousand sample averages forming a smooth bell shape centered near 14.9 minutes.

Does It Work for Any Distribution?

Heights

Here’s the distribution.

\[X \sim N(\mu = 70,\ \sigma^2 = 3^2)\]

\[\mu = 70 \qquad\qquad \sigma = 3\]

The normal pdf of one cadet's height, a bell centered at 70 inches.

We sample \(n = 49\) cadets and average them. We do that 1,000 times.

A histogram of sample mean heights, a narrow bell centered at 70 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 70,\ \sigma^2 = \left(\frac{3}{\sqrt{49}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{3}{7} = 0.429\]

Arrivals in an Hour

Here’s the distribution.

\[X \sim \text{Pois}(\mu = 6)\]

\[\mu = 6 \qquad\qquad \sigma = \sqrt{\mu} = \sqrt{6} = 2.449\]

The Pois(6) pmf, bars peaking at 5 and 6 with a tail to the right.

We sample \(n = 100\) hours and average them. We do that 1,000 times.

A histogram of sample mean arrivals, a bell centered at 6 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 6,\ \sigma^2 = \left(\frac{2.449}{\sqrt{100}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{2.449}{10} = 0.245\]

Wait for the Next Cadet

Here’s the distribution.

\[X \sim \text{Exp}(\lambda = 6)\]

\[\mu = \frac{1}{\lambda} = \frac{1}{6} = 0.167 \qquad\qquad \sigma = \frac{1}{\lambda} = \frac{1}{6} = 0.167\]

The Exp(6) pdf, starting high at 0 and decaying to the right, strongly right skewed.

We sample \(n = 81\) waits and average them. We do that 1,000 times.

A histogram of sample mean waits, a bell centered at 0.167 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 0.167,\ \sigma^2 = \left(\frac{0.167}{\sqrt{81}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{0.167}{9} = 0.0185\]

Free Throws

Here’s the distribution.

\[X \sim \text{Binom}(n = 10,\ p = 0.75)\]

\[\mu = np = 10(0.75) = 7.5 \qquad\qquad \sigma = \sqrt{np(1-p)} = \sqrt{10(0.75)(0.25)} = 1.369\]

The Binom(10, 0.75) pmf, bars peaking at 8 with a tail to the left.

We sample \(n = 64\) cadets, 10 shots each and average them. We do that 1,000 times.

A histogram of sample mean free throws made, a bell centered at 7.5 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 7.5,\ \sigma^2 = \left(\frac{1.369}{\sqrt{64}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{1.369}{8} = 0.171\]

Discrete, Unnamed

Here’s the distribution.

\[p(x) = \begin{cases} 0.4 & x = 0 \\ 0.3 & x = 1 \\ 0.2 & x = 2 \\ 0.1 & x = 3 \\ 0 & \text{otherwise} \end{cases}\]

\[\mu = \sum x\,p(x) = 1 \qquad\qquad \sigma = \sqrt{E(X^2) - \mu^2} = \sqrt{2 - 1^2} = 1\]

A pmf with bars of height 0.4, 0.3, 0.2, 0.1 at 0, 1, 2, 3.

We sample \(n = 50\) values and average them. We do that 1,000 times.

A histogram of sample means, a bell centered at 1 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 1,\ \sigma^2 = \left(\frac{1}{\sqrt{50}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{1}{\sqrt{50}} = 0.141\]

Continuous, Unnamed

Here’s the distribution.

\[f(x) = \begin{cases} \dfrac{x}{8} & 0 \le x \le 4 \\ 0 & \text{otherwise} \end{cases}\]

\[\mu = \int_0^4 x \cdot \frac{x}{8}\,dx = 2.667 \qquad\qquad \sigma = \sqrt{E(X^2) - \mu^2} = \sqrt{8 - 2.667^2} = 0.943\]

A pdf rising in a straight line from 0 at x = 0 to 0.5 at x = 4.

We sample \(n = 45\) values and average them. We do that 1,000 times.

A histogram of sample means, a bell centered at 2.667 that matches the orange normal curve.

According to the CLT:

\[\bar{X} \sim N\left(\mu = 2.667,\ \sigma^2 = \left(\frac{0.943}{\sqrt{45}}\right)^2\right), \qquad \sigma_{\bar{X}} = \frac{0.943}{\sqrt{45}} = 0.141\]

How Large is “Large Enough”?

NoteRules of Thumb for CLT

The CLT approximation is good when:

For \(\bar{X}\) (sample mean):

  • If the population is normal: CLT works for any \(n\)
  • Otherwise: \(n > 40\)

Computing Probabilities for \(\bar{X}\)

ImportantSame pnorm, new standard deviation

\[P(\bar{X} \le x) = \texttt{pnorm(x, mu, sigma/sqrt(n))}\]

WarningWhere cadets go wrong
  • \(\sigma\) in the third slot. That answers a question about one observation, not about \(\bar{X}\).
  • Forgetting the square root. It is \(\sigma/\sqrt{n}\), not \(\sigma/n\).

Barbershop: one wait is \(X \sim \text{Exp}(\lambda = 1/14)\), so \(\mu = \sigma = 14\) minutes. A random sample of \(49\) cadets records their waits.

a) Fully specify the distribution of \(\bar{X}\).

\[\bar{X} \sim N\left(\mu = 14,\ \sigma^2 = \left(\frac{14}{\sqrt{49}}\right)^2 = 2^2\right)\]

  • Designate the random variable. \(\bar{X}\) is the mean wait of 49 cadets, in minutes.
  • Name the distribution. Normal, approximately, by the CLT (\(49 > 40\)).
  • Specify the parameters. \(\mu = 14\), \(\sigma_{\bar{X}} = 14/\sqrt{49} = 2\).

b) \(P(\bar{X} > 16)\)

\[z = \frac{16 - 14}{14/\sqrt{49}} = 1, \qquad P(\bar{X} > 16) = P(Z > 1) = \mathbf{0.1587}\]

The normal curve for the mean wait, centered at 14 with standard deviation 2, with the area right of 16 shaded.

1 - pnorm(16, 14, 14/sqrt(49)) = 0.1587

c) \(P(12 < \bar{X} < 16)\)

\[P(12 < \bar{X} < 16) = P(-1 < Z < 1) = \mathbf{0.6827}\]

The normal curve for the mean wait with the area between 12 and 16 shaded.

pnorm(16, 14, 14/sqrt(49)) - pnorm(12, 14, 14/sqrt(49)) = 0.6827

d) What is the probability one cadet waits more than 16 minutes?

One observation, so no CLT. \(X \sim \text{Exp}(\lambda = 1/14)\).

\[P(X > 16) = e^{-16/14} = \mathbf{0.3189}\]

1 - pexp(16, 1/14) = 0.3189

e) Find the 95th percentile of \(\bar{X}\).

\[14 + 1.645(2) = \mathbf{17.29} \text{ minutes}\]

The normal curve for the mean wait with the leftmost 95 percent of the area shaded, cut off at 17.29 minutes.

qnorm(0.95, 14, 14/sqrt(49)) = 17.2897

The Sample Proportion

ImportantThe sampling distribution of the sample proportion

\(X \sim \text{Binom}(n, p)\) counts successes in a sample of \(n\). The sample proportion is \(\hat{p} = X/n\).

\[E(\hat{p}) = p, \qquad \sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}\]

When \(np \ge 10\) and \(n(1-p) \ge 10\),

\[\hat{p} \sim N\left(p,\ \frac{p(1-p)}{n}\right) \text{ approximately}\]

NoteWhy this is the CLT

Code each success as \(1\) and each failure as \(0\). Then \(\hat{p}\) is the sample mean of the \(1\)s and \(0\)s.

Coffee: \(40\%\) of cadets drink coffee before first hour. Take a random sample of \(150\) cadets.

Find \(P(\hat{p} > 0.46)\).

\(np = 150(0.40) = 60 \ge 10\) and \(n(1-p) = 150(0.60) = 90 \ge 10\).

\[\hat{p} \sim N\left(\mu = 0.40,\ \sigma^2 = \frac{0.40(0.60)}{150} = 0.04^2\right)\]

\[z =\frac{0.46 - 0.40}{0.04} = 1.5, \qquad P(\hat{p} > 0.46) = P(Z > 1.5) = \mathbf{0.0668}\]

The normal curve for the sample proportion, centered at 0.40 with standard deviation 0.04, with the area right of 0.46 shaded.

1 - pnorm(0.46, 0.40, sqrt(0.40 * 0.60 / 150)) = 0.0668

A Question for Next Lesson

Back to the 50 cadets and their 2 mile run times. Their average was \(\bar{x} = 14.81\) minutes with \(s = 1.86\) minutes.

We don’t know \(\mu\), the mean run time of all cadets. What can we say about it?

Start with what the CLT gives us.

\[P\left(-1.96 < \frac{\bar{X} - \mu}{\sigma/\sqrt{n}} < 1.96\right) = 0.95\]

Solve the inside for \(\mu\).

\[P\left(\bar{X} - 1.96\,\frac{\sigma}{\sqrt{n}} < \mu < \bar{X} + 1.96\,\frac{\sigma}{\sqrt{n}}\right) = 0.95\]

Plug in the sample. With \(n > 40\), \(s\) stands in for \(\sigma\).

\[14.81 \pm 1.96\,\frac{1.86}{\sqrt{50}} = (14.29,\ 15.33)\]

That is a 95% confidence interval for \(\mu\).

Board Problems

Problem 1: Ruck March

Times on a 12 mile ruck march are right skewed with \(\mu = 190\) minutes and \(\sigma = 28\) minutes. A random sample of \(49\) cadets rucks.

  1. Can you find the probability one cadet finishes in under 180 minutes?

  2. Fully specify the distribution of \(\bar{X}\).

  3. Find \(P(\bar{X} < 180)\).

  4. Find \(P(185 < \bar{X} < 195)\).

  1. No. One time is not normal, and we do not know its distribution.

  2. \(\bar{X}\) is the mean time of the 49 cadets, in minutes. By the CLT (\(49 > 40\)),

\[\bar{X} \sim N\left(\mu = 190,\ \sigma^2 = \left(\frac{28}{\sqrt{49}}\right)^2 = 4^2\right)\]

  1. \[z = \frac{180 - 190}{4} = -2.5, \qquad P(\bar{X} < 180) = \mathbf{0.0062}\]

The normal curve for the mean ruck time, centered at 190 with standard deviation 4, with the small area left of 180 shaded.

pnorm(180, 190, 28/sqrt(49)) = 0.0062

  1. \[P(185 < \bar{X} < 195) = P(-1.25 < Z < 1.25) = \mathbf{0.7887}\]

The normal curve for the mean ruck time with the area between 185 and 195 shaded.

pnorm(195, 190, 4) - pnorm(185, 190, 4) = 0.7887

Problem 2: Two Mile Run

Two mile run times in a battalion are normal with \(\mu = 15\) minutes and \(\sigma = 1.5\) minutes. A squad of \(9\) soldiers is a random sample.

  1. \(n = 9\) is not more than 40. Is \(\bar{X}\) normal?

  2. Fully specify the distribution of \(\bar{X}\).

  3. Find the probability the squad average is under 14 minutes.

  4. Find the probability one soldier runs under 14 minutes.

  5. How large a sample makes the standard error \(0.25\) minutes?

  1. Yes. The population is normal, so \(\bar{X}\) is exactly normal for any \(n\).

  2. \(\bar{X}\) is the mean run time of the 9 soldiers, in minutes.

\[\bar{X} \sim N\left(\mu = 15,\ \sigma^2 = \left(\frac{1.5}{\sqrt{9}}\right)^2 = 0.5^2\right)\]

  1. \[z = \frac{14 - 15}{0.5} = -2, \qquad P(\bar{X} < 14) = \mathbf{0.0228}\]

pnorm(14, 15, 1.5/sqrt(9)) = 0.0228

  1. \[z = \frac{14 - 15}{1.5} = -0.67, \qquad P(X < 14) = \mathbf{0.2525}\]

pnorm(14, 15, 1.5) = 0.2525

Two normal curves on the same axis scale from 10.5 to 19.5 minutes. Left, one soldier, wide, with the area left of 14 shaded, 0.2525. Right, the mean of 9 soldiers, narrow, with the small area left of 14 shaded, 0.0228.

  1. \[\frac{1.5}{\sqrt{n}} = 0.25 \quad\Rightarrow\quad \sqrt{n} = 6 \quad\Rightarrow\quad n = \mathbf{36}\]

Problem 3: PMCS Faults

\(30\%\) of the vehicles in a brigade have a fault on their PMCS. An inspector takes a random sample of \(84\) vehicles.

  1. Check the conditions for the normal approximation.

  2. Fully specify the distribution of \(\hat{p}\).

  3. Find the probability at most \(25\%\) of the sample has a fault.

  4. Find \(P(0.25 < \hat{p} < 0.40)\).

  1. \(np = 84(0.30) = 25.2 \ge 10\) and \(n(1-p) = 84(0.70) = 58.8 \ge 10\).

  2. \(\hat{p}\) is the proportion of the 84 vehicles with a fault.

\[\hat{p} \sim N\left(\mu = 0.30,\ \sigma^2 = \frac{0.30(0.70)}{84} = 0.05^2\right)\]

  1. \[z = \frac{0.25 - 0.30}{0.05} = -1, \qquad P(\hat{p} \le 0.25) = \mathbf{0.1587}\]

pnorm(0.25, 0.30, sqrt(0.30 * 0.70 / 84)) = 0.1587

  1. \[P(0.25 < \hat{p} < 0.40) = P(-1 < Z < 2) = \mathbf{0.8186}\]

The normal curve for the sample proportion of vehicles with a fault, centered at 0.30 with standard deviation 0.05, with the area between 0.25 and 0.40 shaded.

pnorm(0.40, 0.30, 0.05) - pnorm(0.25, 0.30, 0.05) = 0.8186

Problem 4: Radio Calls

A battalion TOC receives radio calls at an average rate of \(25\) per shift. Let \(X\) be the number of calls in one shift.

  1. Fully specify the distribution of \(X\). Give \(\mu\) and \(\sigma\).

  2. Find the probability one shift has more than 26 calls.

  3. The counts on \(100\) shifts are a random sample. Fully specify the distribution of \(\bar{X}\).

  4. Find the probability the mean over the 100 shifts is more than 26 calls.

  1. \(X \sim \text{Pois}(\mu = 25)\), with \(\mu = 25\) and \(\sigma = \sqrt{25} = 5\) calls.

  2. One observation, so no CLT.

\[P(X > 26) = 1 - P(X \le 26) = \mathbf{0.3706}\]

1 - ppois(26, 25) = 0.3706

  1. \(\bar{X}\) is the mean number of calls per shift over the 100 shifts. By the CLT (\(100 > 40\)),

\[\bar{X} \sim N\left(\mu = 25,\ \sigma^2 = \left(\frac{5}{\sqrt{100}}\right)^2 = 0.5^2\right)\]

  1. \[z = \frac{26 - 25}{0.5} = 2, \qquad P(\bar{X} > 26) = \mathbf{0.0228}\]

The normal curve for the mean calls per shift, centered at 25 with standard deviation 0.5, with the small area right of 26 shaded.

1 - pnorm(26, 25, 5/sqrt(100)) = 0.0228

Before You Leave

Today

  • \(E(\bar{X}) = \mu\) and \(\sigma_{\bar{X}} = \sigma/\sqrt{n}\)
  • Normal population: \(\bar{X}\) is normal for any \(n\). Any population: \(\bar{X}\) is approximately normal for \(n > 40\)
  • \(\hat{p}\) is approximately \(N\left(p,\ p(1-p)/n\right)\) when \(np \ge 10\) and \(n(1-p) \ge 10\)

Any questions?

Next Lesson

Lesson 19: Confidence Intervals I

  • Interpret a confidence level and the basic properties of a confidence interval
  • Construct a large-sample confidence interval for a population mean
  • Construct a large-sample confidence interval for a population proportion

Reading: Devore 7.1, 7.2

Upcoming Graded Events

  • WebAssign 5.4: due at the start of Lesson 19
  • EDA: due 11 October
  • WPR II: Lesson 28 (covers Lessons 18-26)
  • TEE: 15-18 Dec 2026