From the null hypothesis to the conclusion — step by step
Progress:
Module 8
Interactive Hypothesis Test
Use this tool to carry out a complete binomial hypothesis test. Set up your hypotheses, enter the data, and the tool will walk through each step.
Set up the test
Module 9
p-values
The p-value is the probability, computed under \(H_0\), of observing a result at least as extreme as the one obtained. It measures how surprised we should be by the data if \(H_0\) were true.
Small p-value → the observed data would be very unlikely if \(H_0\) were true → strong evidence against \(H_0\).
Large p-value → the observed data is consistent with \(H_0\) → insufficient evidence to reject.
Test type
p-value formula
Lower-tail \((H_1: p < p_0)\)
\(P(X \leq x_{obs})\)
Upper-tail \((H_1: p > p_0)\)
\(P(X \geq x_{obs}) = 1 - P(X \leq x_{obs}-1)\)
Two-tailed \((H_1: p \neq p_0)\)
Smaller of \(P(X \leq x_{obs})\) and \(P(X \geq x_{obs})\), then compare with \(\alpha/2\)
p-value Calculator
Enter the parameters to compute the exact p-value using \(X \sim B(n, p_0)\).
Misinterpretations of the p-value to avoid:
✗ "The p-value is the probability that \(H_0\) is true." — Incorrect. It is the probability of the data given that \(H_0\) is true.
✗ "A p-value of 0.04 means there is a 4% chance we are wrong." — Incorrect.
✓ Correct: "If \(H_0\) were true, the probability of observing data this extreme or more extreme is 0.04."
Think about it
Can we interpret a high p-value as evidence for \(H_0\)?
No. A high p-value means the data is consistent with \(H_0\), but it is also consistent with many other hypotheses. Absence of evidence is not evidence of absence. The only way to gather positive evidence for \(H_0\) is through a different methodology (equivalence testing or Bayesian approaches) that is beyond A-level scope.
Why is 0.05 such a common threshold?
Largely historical convention, popularised by the statistician R.A. Fisher in the 1920s who described results with \(p < 0.05\) as "significant". There is nothing magical about 5% — it is simply a widely adopted convention. Different fields use different thresholds (particle physics uses \(p < 0.0000003\), the "5-sigma" standard).
Module 10
Type I and Type II Errors
In any hypothesis test there are two ways to make an error. Understanding these is essential for interpreting results and designing studies.
\(H_0\) is actually true
\(H_0\) is actually false
Reject \(H_0\)
Type I Error ✗
Correct ✓
Do not reject \(H_0\)
Correct ✓
Type II Error ✗
Type I Error (false positive)
Rejecting \(H_0\) when it is true.
Probability = \(\alpha\) (the significance level).
Example: Concluding the coin is biased when it is actually fair.
Type II Error (false negative)
Not rejecting \(H_0\) when it is false.
Probability = \(\beta\) (depends on true \(p\) and \(n\)).
Example: Failing to detect that the coin really is biased.
For the Binomial, the Type I error probability equals the actual significance level — the probability that the test statistic falls in the critical region under \(H_0\). This is at most \(\alpha\) (and often slightly less due to discreteness).
Type I Error Calculator
The Type I error probability is the actual probability of falling in the critical region when \(H_0\) is true.
20
0.30
Think about it
How can we reduce both types of error simultaneously?
The only way to reduce both is to increase the sample size \(n\). For a fixed \(n\), reducing \(\alpha\) (Type I) automatically increases \(\beta\) (Type II), and vice versa. A larger sample makes the test statistic concentrate more tightly around its expected value under each hypothesis, making it easier to distinguish between them.
Which type of error is more serious?
It depends on the context. In medical testing (e.g. a cancer screening test), a Type II error (missing a real case) may be more serious — hence screening tests are made highly sensitive even at the cost of more false positives. In legal contexts (criminal conviction) a Type I error (convicting an innocent person) is typically considered more serious, motivating "beyond reasonable doubt" standards (very small \(\alpha\)).
Module 11
Worked Examples
Lower-tail · 5% level · Small sample
A biologist claims that 40% of seeds of a certain plant germinate. A student plants 15 seeds and finds only 3 germinate. Test at the 5% significance level whether there is evidence that the germination rate is lower than claimed.
Solution
1
Hypotheses: \(H_0: p = 0.4\), \(H_1: p < 0.4\)
2
Distribution under \(H_0\): \(X \sim B(15, 0.4)\) where \(X\) = number of seeds germinating.
Decision: \(0.0905 > 0.05\), so we do not reject \(H_0\).
6
Conclusion: There is insufficient evidence at the 5% significance level to conclude that the germination rate is lower than 40%.
Upper-tail · 5% level · Critical region method
A manufacturer claims that 20% of items from a production line are faulty. An inspector suspects the true rate is higher. She tests 25 items and finds 9 are faulty. Find the critical region for a one-tailed test at the 5% level and state the conclusion.
Conclusion: There is insufficient evidence at the 5% level to conclude that the coin is biased.
Two-tailed · Critical region · 1% level
A drug is claimed to be effective 35% of the time. A researcher tests it on 40 patients. Find the critical region for a two-tailed test at the 1% significance level. The drug is effective for 22 patients — state the conclusion.
Solution
1
Hypotheses: \(H_0: p = 0.35\), \(H_1: p \neq 0.35\)
2
Distribution: \(X \sim B(40, 0.35)\). Use \(\alpha/2 = 0.005\).
Critical region: \(X \leq 7\) or \(X \geq 24\). Actual significance level: \(0.0027 + 0.0026 = 0.53\%\).
6
Observed \(x = 22\): \(22\) is not in the critical region \((22 < 24)\). Do not reject \(H_0\). There is insufficient evidence at the 1% level that the drug's effectiveness differs from 35%.
Module 12
Testing for Zero Correlation (PMCC)
The Product Moment Correlation Coefficient (PMCC), denoted \(\rho\) (rho) for the population and \(r\) for a sample, measures the strength and direction of a linear relationship between two variables. We can test whether any apparent correlation is genuine or just the result of chance variation.
The parameter and the hypotheses
The population PMCC is \(\rho\) (always between \(-1\) and \(+1\)). The null hypothesis always states that there is no linear correlation in the population:
\[H_0: \rho = 0\]
One-tailed (positive): \(H_1: \rho > 0\)
We suspect there is a positive correlation.
One-tailed (negative): \(H_1: \rho < 0\)
We suspect there is a negative correlation.
Two-tailed: \(H_1: \rho \neq 0\) — some correlation exists, direction unknown.
How to carry out the test
Unlike the binomial test, we do not compute a p-value from scratch. Instead we use a table of critical values for \(r\):
The PMCC Test Procedure
1
State hypotheses: \(H_0: \rho = 0\) and \(H_1\) (choose direction or two-tailed).
2
Calculate the sample PMCC \(r\) from the data (usually done on a calculator).
3
Look up the critical value from the PMCC table using the sample size \(n\) and the significance level. For a two-tailed test, halve the significance level before looking up.
4
Compare: if \(|r|\) exceeds the critical value (i.e. \(r\) is closer to \(\pm 1\) than the critical value), reject \(H_0\). Otherwise do not reject.
5
Conclude in context.
Key rule: Reject \(H_0\) if \(r\) is closer to \(\pm 1\) than the critical value. Do not reject if \(r\) is closer to 0 than the critical value.
For a one-tailed test (\(H_1: \rho > 0\)): reject if \(r > r_{\text{crit}}\).
For a one-tailed test (\(H_1: \rho < 0\)): reject if \(r < -r_{\text{crit}}\).
For a two-tailed test: reject if \(|r| > r_{\text{crit}}\) (use \(\alpha/2\) in the table).
Critical Values Table (extract)
One-tailed significance levels. For a two-tailed test at \(\alpha\), use the column for \(\alpha/2\).
\(n\)
10% (one-tail)
5% (one-tail)
2.5% (one-tail)
1% (one-tail)
0.5% (one-tail)
5
0.6870
0.8054
0.8783
0.9343
0.9587
6
0.6084
0.7293
0.8114
0.8822
0.9172
7
0.5509
0.6694
0.7545
0.8329
0.8745
8
0.5067
0.6215
0.7067
0.7887
0.8343
9
0.4716
0.5822
0.6664
0.7498
0.7977
10
0.4428
0.5494
0.6319
0.7155
0.7646
12
0.3981
0.4973
0.5760
0.6581
0.7079
15
0.3507
0.4409
0.5140
0.5923
0.6411
20
0.2960
0.3783
0.4438
0.5155
0.5614
25
0.2573
0.3365
0.3961
0.4622
0.5052
30
0.2327
0.3061
0.3610
0.4226
0.4629
Worked Examples
A researcher measures hours of revision and exam score for 10 students. The sample PMCC is \(r = 0.582\). Test at the 5% significance level whether there is evidence of a positive correlation between revision time and exam score.
Critical value: from the table, one-tail 5%, \(n = 10\): critical value \(= 0.5494\).
4
Compare: \(r = 0.582 > 0.5494\) (i.e. \(r\) is closer to \(+1\) than the critical value). Reject \(H_0\).
5
Conclusion: There is sufficient evidence at the 5% significance level that there is a positive correlation between revision hours and exam score.
Data on temperature and ice-cream sales for 12 days gives \(r = 0.611\). Test at the 5% significance level whether there is evidence of a correlation (no prior expectation of direction).
Conclusion: There is sufficient evidence at the 5% level of a correlation between temperature and ice-cream sales.
Enter your sample PMCC and size to check against the critical value.
Common mistakes
Using the full significance level for a two-tailed test
For a two-tailed test you must halve the significance level before looking up the critical value. If you're testing at 5% two-tailed, you look in the 2.5% column of the PMCC table, not the 5% column. Using the wrong column makes the test too easy to reject \(H_0\).
Choosing the direction of \(H_1\) after computing \(r\)
If \(r = +0.7\) and you then write \(H_1: \rho > 0\), you have used the data to choose your test. The direction of \(H_1\) must be chosen based on the research question before the data are analysed. Otherwise the true Type I error rate is doubled.
Module 13
Hypothesis Test for a Population Mean (Normal Distribution)
When the underlying variable is normally distributed and we know (or can assume) the population variance \(\sigma^2\), we can test whether the population mean \(\mu\) has changed — using the distribution of the sample mean.
The Sampling Distribution of \(\bar{X}\)
Suppose we have a large population where some variable \(X\) is distributed:
\[X \sim N\!\left(\mu,\, \sigma^2\right)\]
If we take many samples of size \(n\) from this population and compute the mean \(\bar{X}\) of each sample, then those sample means follow their own normal distribution — with the same mean but a smaller variance:
The standard deviation of \(\bar{X}\) is \(\dfrac{\sigma}{\sqrt{n}}\), often called the standard error. The larger the sample, the tighter the distribution of sample means clusters around \(\mu\).
Intuition: A single observation can be quite far from \(\mu\) by chance. But the mean of 25 observations is much less likely to be far from \(\mu\) — extreme values tend to cancel out. Dividing \(\sigma^2\) by \(n\) captures this: larger samples give more precise estimates of the mean.
The Test Procedure
We observe a sample of size \(n\) with mean \(\bar{x}\). We test whether this is consistent with the population having mean \(\mu_0\).
Steps for the Normal Mean Test
1
State hypotheses: \(H_0: \mu = \mu_0\) and \(H_1\) (choose \(\mu < \mu_0\), \(\mu > \mu_0\), or \(\mu \neq \mu_0\)).
2
State the distribution under \(H_0\): \(\bar{X} \sim N\!\left(\mu_0,\, \dfrac{\sigma^2}{n}\right)\).
3
Calculate the p-value — the probability of observing \(\bar{X}\) at least as extreme as \(\bar{x}\), using the Normal distribution. Or standardise: \(Z = \dfrac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}\) and use the standard Normal \(N(0,1)\).
4
Compare p-value with \(\alpha\) (or \(\alpha/2\) for two-tailed). Reject \(H_0\) if p-value \(\leq\) threshold.
5
State conclusion in context.
On a calculator (Casio fx): Menu → 7 (Distribution) → Normal CD for cumulative probabilities, or Inverse Normal. Input lower bound, upper bound, \(\sigma/\sqrt{n}\), \(\mu_0\). For \(P(\bar{X} \geq \bar{x})\), use lower = \(\bar{x}\), upper = \(10^{99}\).
Worked Examples
Chocolate bars are supposed to have mean weight \(\mu = 50\,\text{g}\) with \(\sigma^2 = 4\,\text{g}^2\) (i.e. \(\sigma = 2\,\text{g}\)). A quality-control inspector weighs a random sample of 25 bars and finds \(\bar{x} = 49.2\,\text{g}\). Test at the 5% level whether the mean weight has decreased.
Solution
1
Hypotheses: \(H_0: \mu = 50\) \(H_1: \mu < 50\)
2
Distribution under \(H_0\): \(\bar{X} \sim N\!\left(50,\, \dfrac{4}{25}\right) = N(50,\, 0.16)\)
Conclusion: There is significant evidence at the 5% level that the mean weight of chocolate bars has decreased below 50 g.
The reaction times of drivers are known to follow \(N(0.8,\, 0.04)\) (seconds). After a new training programme, 16 drivers have mean reaction time \(\bar{x} = 0.72\,\text{s}\). Test at the 1% significance level whether the training has changed mean reaction time.
Compare with \(\alpha/2 = 0.005\): \(0.0548 > 0.005\). Do not reject \(H_0\).
5
Conclusion: There is insufficient evidence at the 1% level that the training programme has changed mean reaction time.
Enter the parameters to carry out a Normal mean test. The calculator uses the exact Normal CDF.
Common mistakes
Using \(\sigma^2\) where \(\sigma\) is required (or vice versa)
The distribution of the sample mean is \(\bar{X} \sim N(\mu_0, \sigma^2/n)\). The standard deviation of \(\bar{X}\) (the standard error) is \(\sigma/\sqrt{n}\), not \(\sigma^2/n\). When standardising, divide by \(\sigma/\sqrt{n}\). Confusing the variance and standard deviation is one of the most frequent errors in this topic.
Using the distribution of \(X\) rather than \(\bar{X}\)
If \(X \sim N(\mu, \sigma^2)\), a single observation has standard deviation \(\sigma\). But we observed a sample mean \(\bar{x}\), and the distribution of the sample mean has standard deviation \(\sigma/\sqrt{n}\). Using \(\sigma\) instead of \(\sigma/\sqrt{n}\) makes the standard error far too large and typically leads to failing to reject \(H_0\) when you should.
Summary
Hypothesis Testing — Key Points
The Hypotheses
\(H_0\): always specifies exact value of parameter, e.g. \(p = 0.3\)