1. Binomial distribution · Probability distributions

Preliminary definitions

Probability distribution

A probability distribution is the function that assigns a probability to each event of the sample space. It can be discrete (the possible values are finite or countably infinite) or continuous (the values fill an interval).

Random variable

A random variable $X$ is the function that assigns a real number to each element of the sample space. If the values it can take are finite or countably infinite, $X$ is discrete; if there are uncountably many (filling an interval), it is continuous.

Discrete examples: number of heads when flipping 5 coins, number of aces when drawing 3 cards. Continuous examples: height of a student, lifetime of a light bulb.

Probability mass function

When $X$ is discrete, the probability mass function $f$ assigns to each value $X$ can take its probability: $f(x) = P(X = x)$.

For $f$ to be a valid probability mass function, two conditions must hold:

0 \le f(x_i) \le 1 \quad \text{for all } i, \qquad \sum_{i} f(x_i) = 1.

Example — Rolling a die

We roll a fair die. The sample space is $E = \{1, 2, 3, 4, 5, 6\}$ and $X$ is the variable "face shown".

f(1) = P(X{=}1) = \tfrac{1}{6}, \quad f(2) = \tfrac{1}{6}, \;\dots\; , f(6) = \tfrac{1}{6}.

The graph of the probability mass function is uniform: 6 bars of the same height $\tfrac{1}{6}$.

This is the so-called discrete uniform distribution: every value of $X$ has the same probability.

Continuous random variables also have associated distributions — the most famous is the normal distribution (Gauss's bell curve) — but in this course we focus on discrete distributions, and in particular the binomial.

Bernoulli trial

Bernoulli experiment

A Bernoulli experiment is a random experiment with only two possible outcomes, which we label success and failure.

We denote:

p = P(\text{success}), \qquad q = P(\text{failure}) = 1 - p.

By construction, $p + q = 1$.

Example — Penalty kick

A footballer has probability $0{,}7$ of scoring a penalty. Taking the penalty is a Bernoulli experiment with:

p = 0{,}7, \qquad q = 1 - p = 0{,}3.

"Scoring" is the success and "missing" the failure (which outcome is success and which failure is a labelling we choose to match the question being asked).

Binomial distribution

Definition

The binomial distribution is the distribution followed by a random variable $X$ that counts the number of successes in $n$ independent repetitions of a Bernoulli experiment with success probability $p$.

We write:

X \sim B(n, p),

where $n$ is the number of trials and $p$ the individual success probability.

When to use the binomial

A phenomenon is well modelled by $B(n,p)$ when it satisfies the 4 conditions:

1. The same experiment is repeated $n$ times.

2. Each experiment has only two possible outcomes (success / failure).

3. The success probability $p$ is constant on each repetition.

4. The repetitions are independent of each other.

Example — Five penalties

The same footballer as in the previous example takes 5 penalties. Let $X = $ "number of penalties scored". Then:

X \sim B(5,\;0{,}7).

a) What is the probability of scoring exactly 3 penalties?

P(X = 3) \approx 0{,}3087.

This value comes out either from the formula we'll derive next, or from a binomial table.

b) What is the probability of scoring at least 3 penalties?

"At least 3" means $X \ge 3$. It can be obtained via the complement:

P(X \ge 3) \;=\; 1 - P(X \le 2) \;=\; 1 - 0{,}16308 \;=\; \boxed{\;0{,}83692.\;}

$P(X \le 2) = P(X{=}0) + P(X{=}1) + P(X{=}2)$ — the sum of the first three values.

Binomial formula

Probability mass function of $B(n,p)$

If $X \sim B(n,p)$, the probability of getting exactly $k$ successes in $n$ trials is:

\boxed{\;P(X = k) \;=\; \binom{n}{k}\, p^{k}\, q^{\,n-k}\;}, \qquad k = 0, 1, 2, \dots, n.

Where $\binom{n}{k}$ is the binomial coefficient (defined below) and $q = 1 - p$.

The formula has three ingredients:

· $\binom{n}{k}$ — how many ways the $k$ successes can be placed among the $n$ positions.

· $p^{k}$ — probability of a specific sequence with $k$ successes.

· $q^{n-k}$ — probability of the $n - k$ failures.

Penalty kicks — detailed computation

With $X \sim B(5,\;0{,}7)$, let's compute $P(X = 3)$ using the formula:

P(X = 3) \;=\; \binom{5}{3}\, 0{,}7^{3}\cdot 0{,}3^{2} \;=\; 10 \cdot 0{,}343 \cdot 0{,}09 \;=\; \boxed{\;0{,}3087.\;}

Binomial coefficients

Definition

The binomial coefficient "$m$ choose $n$" is the number of $n$-element subsets that can be formed from a set with $m$ elements. We write and compute it as:

C_{m,n} \;=\; \binom{m}{n} \;=\; \frac{m!}{n!\,(m-n)!}.

Where $m!$ is the factorial of $m$:

m! \;=\; m\cdot(m-1)\cdot(m-2)\cdots 2\cdot 1, \qquad 0! = 1.

Examples

· $5! = 5\cdot 4\cdot 3\cdot 2\cdot 1 = 120$.

· Compute $C_{8,3} = \binom{8}{3}$:

\binom{8}{3} \;=\; \frac{8!}{3!\,5!} \;=\; \frac{8\cdot 7\cdot 6\cdot \cancel{5!}}{3!\,\cancel{5!}} \;=\; \frac{8\cdot 7\cdot 6}{6} \;=\; 56.

· Compute $\binom{5}{3}$ (the one in the penalty example):

\binom{5}{3} \;=\; \frac{5!}{3!\,2!} \;=\; \frac{5\cdot 4}{2} \;=\; 10.

Calculation trick

To compute $\binom{m}{n}$ quickly, write the $n$ decreasing factors from $m$ in the numerator and $n!$ in the denominator:

\binom{m}{n} = \frac{m\,(m{-}1)\,(m{-}2)\cdots(m{-}n{+}1)}{n!}.

That way you don't have to compute the whole $m!$.

On a scientific calculator they are entered as nCr (combination) and ! (factorial). Always double-check on a calculator the binomial coefficients you get in an exam.

Cumulative probabilities

With $X \sim B(5,\;0{,}7)$ we can compute any cumulative probability. Recall the formulas above:

P(X{=}0) = \tbinom{5}{0}\,0{,}7^{0}\,0{,}3^{5} = 0{,}00243,$$ $$P(X{=}1) = \tbinom{5}{1}\,0{,}7^{1}\,0{,}3^{4} = 0{,}02835,$$ $$P(X{=}2) = \tbinom{5}{2}\,0{,}7^{2}\,0{,}3^{3} = 0{,}13230,$$ $$P(X{=}3) = \tbinom{5}{3}\,0{,}7^{3}\,0{,}3^{2} = 0{,}30870,$$ $$P(X{=}4) = \tbinom{5}{4}\,0{,}7^{4}\,0{,}3^{1} = 0{,}36015,$$ $$P(X{=}5) = \tbinom{5}{5}\,0{,}7^{5}\,0{,}3^{0} = 0{,}16807.

Sanity check: the sum of all of them must equal exactly $1$. $0{,}00243 + 0{,}02835 + 0{,}13230 + 0{,}30870 + 0{,}36015 + 0{,}16807 = 1$ ✓.

Calculations with $X \sim B(5,\;0{,}7)$

a) $P(X < 3) = P(X{=}0) + P(X{=}1) + P(X{=}2)$:

P(X < 3) = 0{,}00243 + 0{,}02835 + 0{,}13230 = \boxed{\;0{,}16308.\;}

b) $P(X \ge 4) = P(X{=}4) + P(X{=}5)$:

P(X \ge 4) = 0{,}36015 + 0{,}16807 = \boxed{\;0{,}52822.\;}

Or, via the complement: $P(X\ge 4) = 1 - P(X\le 3) = 1 - 0{,}47178 = 0{,}52822$.

c) $P(X \le 2) = P(X{=}0) + P(X{=}1) + P(X{=}2) = 0{,}16308$ (same as a)).

Watch out for the inequalities!

$P(X < 3)$ does not include the 3: it is $P(X{=}0) + P(X{=}1) + P(X{=}2)$.

$P(X \le 3)$ does include the 3: it is $P(X{=}0) + \cdots + P(X{=}3)$.

Always read the statement carefully to tell whether the inequality is strict or not.

Expectation and standard deviation

Parameters of a binomial

For a variable $X \sim B(n,p)$, the expectation (mean) and the variance have very simple formulas:

E(X) = n\cdot p, \qquad \mathrm{Var}(X) = n\cdot p\cdot q.

And therefore the standard deviation:

\sigma \;=\; \sqrt{\mathrm{Var}(X)} \;=\; \sqrt{n\cdot p\cdot q}.

Example — The 5 penalties

With $X\sim B(5,\,0{,}7)$:

E(X) = 5 \cdot 0{,}7 = 3{,}5 \text{ penalties scored on average}.$$ $$\sigma = \sqrt{5\cdot 0{,}7\cdot 0{,}3} = \sqrt{1{,}05} \approx 1{,}02.

Interpretation: if the footballer took many series of 5 penalties, on average he would score 3,5 penalties, with a typical spread of about 1 penalty around that value.

Intuition

· $E(X) = np$ — makes sense: if you run $n$ trials and each has probability $p$ of success, you expect $np$ successes in total.

· $\mathrm{Var}(X) = npq$ — depends on the product $pq$, which is maximal when $p = 0{,}5$ (and $0$ when $p = 0$ or $p = 1$). In other words: the binomial is less unpredictable when $p$ is close to 0 or 1.

Exercises

1 Candidate function — is it a p.m.f.?

Consider the function $f(x) = \dfrac{2x + 1}{6}$ with $x \in \{1, 2, 3\}$.

a) Compute $f(1)$, $f(2)$ and $f(3)$.

f(1) = \tfrac{2\cdot 1 + 1}{6} = \tfrac{3}{6} = \tfrac{1}{2} = 0{,}5.$$ $$f(2) = \tfrac{2\cdot 2 + 1}{6} = \tfrac{5}{6}.$$ $$f(3) = \tfrac{2\cdot 3 + 1}{6} = \tfrac{7}{6}.

b) Can $f$ be a probability mass function? Why?

A probability mass function must satisfy two conditions: $0 \le f(x_i) \le 1$ for all $i$, and $\sum f(x_i) = 1$.

· $f(3) = \tfrac{7}{6} > 1$ → violates the first condition.

· Also, the sum:

f(1) + f(2) + f(3) = \tfrac{1}{2} + \tfrac{5}{6} + \tfrac{7}{6} = \tfrac{3 + 5 + 7}{6} = \tfrac{15}{6} = \tfrac{5}{2} \neq 1.

It also violates the second condition.

\boxed{\;f(x) = \tfrac{2x+1}{6} \text{ is NOT a probability mass function.}\;}

2 Table of a discrete variable — missing entry

The following table gives the probability mass function of a discrete random variable $X$. The value $P(X{=}3)$ is unknown.

$x_i$	0	1	2	3	4	5
$P_i$	0,2	0,2	0,1	?	0,1	0,1

a) $P(X = 3)$.

Since all probabilities must sum to 1:

0{,}2 + 0{,}2 + 0{,}1 + P(X{=}3) + 0{,}1 + 0{,}1 = 1.$$ $$P(X{=}3) = 1 - 0{,}7 = \boxed{\;0{,}3.\;}

b) $P(X > 3)$.

P(X > 3) = P(X{=}4) + P(X{=}5) = 0{,}1 + 0{,}1 = \boxed{\;0{,}2.\;}

c) $P(X < 2)$.

P(X < 2) = P(X{=}0) + P(X{=}1) = 0{,}2 + 0{,}2 = \boxed{\;0{,}4.\;}

Cross-check via the complement: $P(X < 2) = 1 - P(X\ge 2) = 1 - (0{,}1 + 0{,}3 + 0{,}1 + 0{,}1) = 1 - 0{,}6 = 0{,}4$. ✓

d) $P(1 < X \le 5)$.

Careful: the lower bound is strict ($X > 1$), but the upper one includes the 5.

P(1 < X \le 5) = P(X{=}2) + P(X{=}3) + P(X{=}4) + P(X{=}5)$$ $$= 0{,}1 + 0{,}3 + 0{,}1 + 0{,}1 = \boxed{\;0{,}6.\;}

e) The mean, $E(X) = \sum x_i \cdot P_i$.

E(X) = 0\cdot 0{,}2 + 1\cdot 0{,}2 + 2\cdot 0{,}1 + 3\cdot 0{,}3 + 4\cdot 0{,}1 + 5\cdot 0{,}1$$ $$= 0 + 0{,}2 + 0{,}2 + 0{,}9 + 0{,}4 + 0{,}5 = \boxed{\;2{,}2.\;}

f) The standard deviation.

First the variance $\mathrm{Var}(X) = \sum (x_i - E(X))^2 \cdot P_i$:

\mathrm{Var}(X) = (0{-}2{,}2)^2\cdot 0{,}2 + (1{-}2{,}2)^2\cdot 0{,}2 + (2{-}2{,}2)^2\cdot 0{,}1 + (3{-}2{,}2)^2\cdot 0{,}3 + (4{-}2{,}2)^2\cdot 0{,}1 + (5{-}2{,}2)^2\cdot 0{,}1$$ $$= 4{,}84\cdot 0{,}2 + 1{,}44\cdot 0{,}2 + 0{,}04\cdot 0{,}1 + 0{,}64\cdot 0{,}3 + 3{,}24\cdot 0{,}1 + 7{,}84\cdot 0{,}1$$ $$= 0{,}968 + 0{,}288 + 0{,}004 + 0{,}192 + 0{,}324 + 0{,}784 = 2{,}56.

\sigma = \sqrt{\mathrm{Var}(X)} = \sqrt{2{,}56} = \boxed{\;1{,}6.\;}

Shortcut: $\mathrm{Var}(X) = E(X^2) - E(X)^2$. Here $E(X^2) = 0 + 0{,}2 + 0{,}4 + 2{,}7 + 1{,}6 + 2{,}5 = 7{,}4$, so $\mathrm{Var}(X) = 7{,}4 - 2{,}2^2 = 7{,}4 - 4{,}84 = 2{,}56$. ✓

3 Direct application of the formula

Let $X \sim B(6,\;0{,}4)$. Compute:

a) $P(X = 2)$.

P(X{=}2) = \binom{6}{2}\, 0{,}4^{2}\, 0{,}6^{4} = 15\cdot 0{,}16\cdot 0{,}1296 \approx \boxed{\;0{,}3110.\;}

b) $P(X = 0)$.

P(X{=}0) = \binom{6}{0}\, 0{,}4^{0}\, 0{,}6^{6} = 1\cdot 1\cdot 0{,}046656 \approx \boxed{\;0{,}0467.\;}

c) $P(X \ge 1)$ — at least 1 success.

Via the complement, faster:

P(X \ge 1) = 1 - P(X{=}0) = 1 - 0{,}0467 = \boxed{\;0{,}9533.\;}

d) $E(X)$ and $\sigma$.

E(X) = 6\cdot 0{,}4 = 2{,}4, \qquad \sigma = \sqrt{6\cdot 0{,}4\cdot 0{,}6} = \sqrt{1{,}44} = \boxed{\;1{,}2.\;}

4 Philosophy — eight students drawn at random

80 % of the students of a secondary school passed Philosophy last year. From a group of 8 students chosen at random, what is the probability that only two have failed?

a) Justify why this is a binomial distribution.

Each student: passed or failed → it's a Bernoulli trial. We repeat the same trial 8 times (one per student), with constant success probability $p = 0{,}8$ (assuming independence) → it's a binomial $B(8,\,0{,}8)$.

b) Identify the success and failure probabilities.

If "passing" is success: $p = 0{,}8$, $q = 0{,}2$.

But the question is about the number of failures; it is convenient to swap:

If "failing" is success: $p = 0{,}2$, $q = 0{,}8$.

c) Write down the probability mass function.

We have two equivalent formulations:

· $X_1 = $ "number of passes" $\sim B(8,\,0{,}8)$.

· $X_2 = $ "number of failures" $\sim B(8,\,0{,}2)$.

If two students fail, six pass. Hence $X_2 = 2$ is equivalent to $X_1 = 6$. We'll work with $X_2$, which matches the question directly.

P(X_2 = k) = \binom{8}{k}\, 0{,}2^{k}\, 0{,}8^{\,8-k}, \quad k = 0, 1, \dots, 8.

d) Compute $P(\text{only two fail})$.

P(X_2 = 2) = \binom{8}{2}\, 0{,}2^{2}\, 0{,}8^{6} = 28\cdot 0{,}04\cdot 0{,}262144 \approx \boxed{\;0{,}2936.\;}

Cross-check: $P(X_1 = 6) = \binom{8}{6}\, 0{,}8^{6}\, 0{,}2^{2} = 28\cdot 0{,}262144\cdot 0{,}04 \approx 0{,}2936$. ✓ They match, as expected.

5 Production line — defective parts

A production line makes parts that have a 5 % probability of being defective. We pick 10 parts at random and inspect them.

a) Identify the distribution and its parameters.

Let $X = $ "number of defective parts". Then $X \sim B(10,\,0{,}05)$.

b) $P(\text{no defective parts})$.

P(X{=}0) = \binom{10}{0}\, 0{,}05^{0}\, 0{,}95^{10} = 0{,}95^{10} \approx \boxed{\;0{,}5987.\;}

Nearly 60 % of the lots of 10 parts have no defective part at all.

c) $P(\text{exactly 2 defective parts})$.

P(X{=}2) = \binom{10}{2}\, 0{,}05^{2}\, 0{,}95^{8} = 45\cdot 0{,}0025\cdot 0{,}6634 \approx \boxed{\;0{,}0746.\;}

d) $P(\text{at least 1 defective})$.

Via the complement:

P(X \ge 1) = 1 - P(X{=}0) = 1 - 0{,}5987 = \boxed{\;0{,}4013.\;}

e) $E(X)$ and $\sigma$.

E(X) = 10\cdot 0{,}05 = 0{,}5,\qquad \sigma = \sqrt{10\cdot 0{,}05\cdot 0{,}95} = \sqrt{0{,}475} \approx \boxed{\;0{,}689.\;}

On average there is half a defective part per lot of 10. The standard deviation is larger than the mean — quite normal when $p$ is small.

6 Random guessing on a multiple-choice test

A test has 20 questions with 4 options each, only one correct. A student answers at random, without having studied.

a) Identify the distribution and its parameters.

Let $X = $ "number of correct answers". On each question, $p = \tfrac{1}{4} = 0{,}25$ (guessing) and $q = 0{,}75$. Hence $X \sim B(20,\,0{,}25)$.

b) How many correct answers will the student get on average?

E(X) = 20\cdot 0{,}25 = \boxed{\;5 \text{ correct answers.}\;}

c) What is the standard deviation?

\sigma = \sqrt{20\cdot 0{,}25\cdot 0{,}75} = \sqrt{3{,}75} \approx \boxed{\;1{,}936.\;}

d) $P(X = 10)$ — exactly 10 correct (half).

P(X{=}10) = \binom{20}{10}\, 0{,}25^{10}\, 0{,}75^{10}.$$ $$\binom{20}{10} = 184\,756, \qquad 0{,}25^{10}\approx 9{,}54\cdot 10^{-7}, \qquad 0{,}75^{10}\approx 0{,}0563.$$ $$P(X{=}10) \approx 184\,756\cdot 9{,}54\cdot 10^{-7}\cdot 0{,}0563 \approx \boxed{\;0{,}0099.\;}

Under 1 %: the probability of getting half the questions right by pure guessing is almost negligible — that's where the intuition "if I answer everything at random I'll definitely fail" comes from.