1. Introduction: Setting Up the Test
A hypothesis test uses sample data to decide whether there is sufficient evidence to reject a claim about a population parameter. When outcomes are binary — success or failure — and trials are independent, the binomial distribution provides the probability model.
The Random Variable and Hypotheses
Let X be the number of successes in S independent trials, where each trial has probability p of success. Under certain conditions (discussed below), X\sim B(S,\,p).
The two competing hypotheses are:
Null Hypothesis H_0
The default assumption — that the probability of success equals some specified value k:
We assume H_0 is true when we perform calculations.
Alternative Hypothesis H_1
The claim we are testing — that p differs from k in a particular direction:
These are one-tailed tests (lower or upper tail).
The Significance Level \alpha
The significance level \alpha (commonly 0.05 or 0.01) is the threshold probability below which we consider our observed result too unlikely under H_0. It controls the risk of a Type I error — incorrectly rejecting a true null hypothesis.
k — the hypothesised probability of success under H_0
S — the sample size (number of trials)
C — the observed number of successes in the sample
\alpha — the significance level
The Procedure in Brief
- State H_0: p = k and H_1 (lower or upper tail).
- Assume H_0 is true, so X \sim B(S,\, k).
- Observe C successes in the sample.
- Calculate the probability of a result at least as extreme as C, in the direction of H_1.
- If this probability \leq \alpha, reject H_0; otherwise, do not reject H_0.
2. The Core Logic of the Test
The fundamental idea is a proof by contradiction applied to probability. We temporarily assume H_0 is true, then ask: "If H_0 were true, how likely would it be to observe a result as extreme as — or more extreme than — what we actually saw?"
Tail Direction and Extremeness
The meaning of "extreme" depends on H_1. The direction of H_1 tells us which way we expect the data to deviate from H_0 if H_0 is false.
Lower tail H_1: p < k
We suspect p is smaller than k. A surprisingly small number of successes supports H_1. Extremeness means "unusually few successes".
Upper tail H_1: p > k
We suspect p is larger than k. A surprisingly large number of successes supports H_1. Extremeness means "unusually many successes".
Accumulating the Probability
Because we want the probability of a result "as extreme or more extreme", we must accumulate probabilities across all values of X that are at least as extreme as C. This is why a cumulative probability — not just P(X=C) — is computed.
For a continuous distribution the probability of any single exact value is zero, but even for discrete distributions, using only P(X=C) would under-count the evidence against H_0. We should count all outcomes at least as contradictory to H_0 as the one we observed.
The Decision Rule
If H_1: p < k
If H_1: p > k
If the cumulative probability exceeds \alpha, the result is not sufficiently unusual under H_0, so we do not reject H_0. (We never "accept H_0" — we simply lack evidence to reject it.)
3. The Lower Tail Test H_1: p < k
When is this test used?
Use a lower tail test when you have reason to believe the true probability of success may be less than the claimed value k. The observed number of successes C provides the evidence.
The Probability to Calculate
Assuming H_0 is true (X \sim B(S, k)), we compute:
This is the probability of observing C successes or fewer if H_0 were true.
Visual Interpretation
The diagram below shows a binomial distribution. The shaded bars represent P(X \leq C) — the probability mass in the lower tail, accumulated from 0 up to C.
The Decision
Reject H_0 if:
The observed result falls in the critical region.
There is sufficient evidence at the \alpha significance level to conclude p < k.
Do not reject H_0 if:
The result is not sufficiently unusual under H_0.
There is insufficient evidence to conclude p < k.
Worked Example
A die is rolled S = 20 times. Under H_0, each roll has p = \tfrac{1}{6} \approx 0.167 chance of showing a six. A player suspects the die gives fewer sixes than expected. In the experiment, C = 1 six is observed. Test at \alpha = 0.05.
H_0: p = \tfrac{1}{6}, H_1: p < \tfrac{1}{6}, X \sim B(20, \tfrac{1}{6})
P(X \leq 1) = P(X=0) + P(X=1)
= \left(\tfrac{5}{6}\right)^{20} + \binom{20}{1}\left(\tfrac{1}{6}\right)\left(\tfrac{5}{6}\right)^{19}
Since 0.1304 > 0.05, we do not reject H_0. There is insufficient evidence at the 5% level that the die produces fewer sixes than expected.
4. The Upper Tail Test H_1: p > k
When is this test used?
Use an upper tail test when you have reason to believe the true probability of success may be greater than the claimed value k. The observed number of successes C provides the evidence.
The Probability to Calculate
Assuming H_0 is true (X \sim B(S, k)), we compute:
This is the probability of observing C successes or more if H_0 were true.
In practice, it is often easier to compute the complement:
Visual Interpretation
The diagram below shows a binomial distribution. The shaded bars represent P(X \geq C) — the probability mass in the upper tail, accumulated from C upwards.
The Decision
Reject H_0 if:
The observed result falls in the critical region.
There is sufficient evidence at the \alpha significance level to conclude p > k.
Do not reject H_0 if:
The result is not sufficiently unusual under H_0.
There is insufficient evidence to conclude p > k.
Worked Example
A coin is tossed S = 15 times. Under H_0, the probability of heads is p = 0.5. A gambler suspects the coin is biased towards heads. They observe C = 11 heads. Test at \alpha = 0.05.
H_0: p = 0.5, H_1: p > 0.5, X \sim B(15, 0.5)
5. Why the Inequalities Work That Way
This is perhaps the most conceptually important question in the test: why does H_1: p < k lead us to compute P(X \leq C), and H_1: p > k lead to P(X \geq C)? The answer comes from thinking carefully about what "extreme" means in each case.
The Guiding Principle
Case 1: H_1: p < k → Compute P(X \leq C)
If p is truly less than k, we expect the number of successes to be smaller than the mean Sk under H_0. So the evidence against H_0 and for H_1 comes in the form of a small value of X.
Now, if C successes is strong evidence against H_0, then observing even fewer successes — say C-1, C-2, \ldots, 0 — would be even stronger evidence against H_0. All values from 0 to C are "at least as extreme" as C in the lower direction.
Therefore, we accumulate probability from 0 up to and including C:
If observing C successes gives evidence against H_0, observing C-1 or fewer would give even more evidence. We must count all such outcomes in our probability — otherwise we would underestimate how unlikely the tail is.
Case 2: H_1: p > k → Compute P(X \geq C)
If p is truly greater than k, we expect the number of successes to be larger than the mean Sk under H_0. So the evidence against H_0 and for H_1 comes in the form of a large value of X.
If C successes is strong evidence against H_0, then observing even more successes — say C+1, C+2, \ldots, S — would be even stronger evidence against H_0. All values from C to S are "at least as extreme" as C in the upper direction.
Therefore, we accumulate probability from C up to S:
If observing C successes gives evidence against H_0, observing C+1 or more would give even more evidence. We must count all such outcomes in our probability — otherwise we would underestimate how unlikely the tail is.
Summary: The Matching Rule
| Alternative H_1 | Direction of evidence | "More extreme" means | Cumulative probability |
|---|---|---|---|
| H_1: p < k | Small X supports H_1 | Even fewer successes: X < C | P(X \leq C) lower tail |
| H_1: p > k | Large X supports H_1 | Even more successes: X > C | P(X \geq C) upper tail |
H_1: p \mathbf{<} k \;\Rightarrow\; P(X \,\mathbf{\leq}\, C) H_1: p \mathbf{>} k \;\Rightarrow\; P(X \,\mathbf{\geq}\, C)
This works because H_1 tells you which direction is "extreme" — and you accumulate probability in that same direction from C outward.
A Useful Way to Think About It
Imagine you are a sceptic trying to challenge H_0. You observe C successes. You ask: "What is the probability that a world where H_0 is true would produce a result this extreme, or more so, in the direction I'm investigating?"
For H_1: p < k, "more extreme in the direction I'm investigating" means fewer successes, so you look left from C on the distribution. For H_1: p > k, it means more successes, so you look right. In both cases you include C itself, because you want the probability of C or worse (from H_0's point of view).
6. Interactive Calculator
Enter the parameters of your hypothesis test below. The calculator will compute the relevant cumulative probability and give a decision.