Basic concepts
Events and sample spaces
Sample space
The sample space is what defines the possible set of all outcomes of an experiment or its measurements. It is usually denoted by \mathbf {S} or \Omega.
The simple experiment of tossing a coin could produce an outcome of heads or tails.
The sample space is:
\Omega =\{ {Head,Tail}\}
Considering the experiment or rolling a dice, the sample space would be:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
Event
An event is any amount of elements of the sample space. As such, it is a subset of the sample space. An event can contain multiple outcomes and it is considered as occurred if either of its elements is produced.
Consider the experiment of tossing a coin in which the sample space is \Omega =\{ {H,T}\} . Any subset of that sample space would be an event. For example:
- \{H\} is the event in which the outcome is heads.
- \{T\} is the event in which the outcome is tails.
Consider the experiment or rolling a dice with the sample space:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
Some possible events are:
- \{1\} if the outcome is number 1.
- \{3\} if the dice shows 3.
- A composite event like A=\{2{,}4{,}6\} that represents the event of the outcome being an even number.
Null event
The null event of a sample space is the empty subset of the sample space.
In a sample space, both the null event and the entire sample space are events.
Operations with events
Compatible Events
Compatible events are two events that can happen at the same time. They share one or more elements.
Consider the example of rolling a dice with the sample space:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
Lets have:
- An event A with the outcome being a prime number: A=\{2{,}3{,}5\}
- An event B with the outcome being an even number: B=\{2{,}4{,}6\}
- An event C with the outcome being an odd number: C=\{1{,}3{,}5\} .
Events A and B are compatible since they can both happen at the same time (the outcome can be both a prime and even number, precisely if it is 2).
A and C are also compatible (the outcome can be both a prime and odd number, precisely if it is 3 or 5).
Incompatible Events
Incompatible events are two events that cannot happen at the same time. They don't share elements and are also known as mutually exclusive events.
In the previous example, B and C are not compatible since the number cannot be even and odd at the same time.
Complementary Events
The complement of an event A, denoted by \overline{A}, is the event that happens when A doesn't occur.
Event \overline{A} is said to be complementary to event A when the event contains exactly the elements in the sample space that were not included in A.
Consider the example of rolling a die with the sample space:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
Event B is the event with the outcome being an even number:
B=\{2{,}4{,}6\}
The complement of this event, denoted by \overline{B}, is the event that happens when B does not happen:
\overline{B}=\{1{,}3{,}5\}.
Together, an event and its complement make up the entire sample space.
In the previous example:
- B is the event that the outcome is an even number.
- \overline{B} is the event that the outcome is not an even number.
Together, events B and \overline{B} contain all the elements of the sample espace.
Union of two events
The union of events A and B is the event that happens when either A or B occurs. It is a set that contains all the elements in A or B . It is denoted:
A\cup B
If we have:
- A=\{2{,}3{,}5\}
- B=\{3{,}4\}
Then:
A\cup B = \{2{,}3{,}4{,}5\}
Intersection of two events
The intersection of events A and B is the event that happens when both A and B occur. It is a set that contains the elements that exist in both A and B. It is denoted:
A\cap B
If we have:
- A=\{2{,}3{,}5\}
- B=\{3{,}4\}
Then:
A\cap B = \{3\}
Two events are incompatible when their intersection is empty or null.
Consider the example of rolling a dice with the sample space:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
We have:
- Event B with the outcome being an even number: B=\{2{,}4{,}6\}.
- Event C with the outcome being an odd number: C=\{1{,}3{,}5\} .
Events B and C are incompatible since they cannot happen at the same time and do not share any elements.
Here, the intersection of the two sets is:
B\cap C = \phi
Properties of probabilities
Probability
In the equiprobability scenario, meaning that all elementary events have the same probability of happening, the probability of an event A to happen, denoted by P\left(A\right) , is calculated with the following formula:
P\left(A\right)=\dfrac{\text{Number of elements of A}}{\text{Number of elements of } \Omega}
Consider the probability of rolling an even number when rolling a fair six-sided dice:
The sample space is:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
It has 6 elements.
Event A with the outcome being an even number is:
A=\{2{,}4{,}6\}
It has 3 elements.
Therefore, the probability of this occuring is:
P\left(A\right)=\dfrac{\text{Number of elements of A}}{\text{Number of elements of } \Omega}=\dfrac{3}{6}=\dfrac{1}{2}
For any event A :
0 \leq P\left(A\right) \leq 1
The probability of the intersection of two events A and B is:
P\left(A\cup B\right)=P\left(A\right)+P\left(B\right)-P\left(A\cap B\right)
Consider the following:
- Event A is “a six-sided dice rolls an odd number”
- Event B is “a six-sided dice rolls a number greater than 3"
We want to calculate P\left(A \cup B\right).
The sample space is:
\Omega=\{1{,}2{,}3{,}4{,}5{,}6\}
It has 6 elements.
The events A and B are:
- A=\{1{,}3{,}5\}
- B=\{4{,}5{,}6\}
So we have:
- P\left(A\right) = \dfrac{3}{6} = \dfrac{1}{2}
- P\left(B\right) =\dfrac{3}{6}= \dfrac{1}{2}
The intersection of these two events contains only the outcome 5:
A \cap B = \{5\}
A \cap B contains only 1 element. Therefore:
P\left(A \cap B\right) = \dfrac{1}{6}
We get:
P\left(A \cup B\right) =P\left(A\right) + P\left(B\right) - P\left(A \cap B\right)
P\left(A \cup B\right) = \dfrac{1}{2}+ \dfrac{1}{2} - \dfrac{1}{6}
P\left(A \cup B\right) = \dfrac{5}{6}
If events A and B are incompatible, then the probability of the intersection becomes:
P\left(A\cup B\right)=P\left(A\right)+P\left(B\right)
Consider the following events:
- Event A is “a six-sided dice rolls an odd number”
- Event B is “a six-sided dice rolls 4”
We want to calculate P\left(A \cup B\right).
Here:
- \Omega=\{1{,}2{,}3{,}4{,}5{,}6\} contains 6 elements.
- A=\{1{,}3{,}5\} contains 3 elements.
- B=\{4\} contains 1 element.
So:
- P\left(A\right) = \dfrac{3}{6}
- P\left(B\right) =\dfrac{1}{6}
These two events are incompatible since they do not have any outcomes in common and can't happen at the same time. It follows that:
P\left(A\cap B\right)=0
Therefore:
P\left(A \cup B\right) =P\left(A\right) + P\left(B\right)= \dfrac{3}{2}+ \dfrac{1}{6} = \dfrac{4}{6}
Independence of events and conditional probabilities
Independent and dependent events
Independent Events
Two events are independent if the occurrence of either event has no influence on the occurrence of the other event.
Suppose we are a flipping a coin twice. The event of landing on heads on the first flip does not effect the occurrence of flipping heads on the second flip.
Dependent Events
If two events are not independent, then they are said to be dependent.
Suppose there are three flowers for sale at a floral shop, a red flower, a white flower, and a blue flower. Suppose two customers are in line to buy a flower. The event of the first customer buying the red flower and the event of the second customer buying the red flower are dependent events. If the first customer buys the red flower, then it will be impossible for the second customer to buy a red flower.
The multiplication rule
If two events A and B are independent, then:
p\left(A\cap B\right)=p\left(A\right).p\left(B\right)
Consider the two events:
- A is "getting a heads with a coin flip"
- B is "rolling a six on a six-sided dice"
The probability of one is not affected by the outcome of the other, so they are independent events. We know that:
- P\left(A\right) = \dfrac{1}{2}
- P\left(B\right) = \dfrac{1}{6}
Therefore:
P\left(A \cap B\right) = P\left(A\right) P\left(B\right) = \left( \dfrac{1}{2}\right) \left( \dfrac{1}{6} \right) = \dfrac{1}{12}
The converse to the previous property holds. If A and B are two events and P\left(A\cap B\right)=P\left(A\right)P\left(B\right), then A and B are independent events.
Conditional probability
Conditional probability
When p\left(B\right)\neq0, the conditional probability of B, given that A has happened, is denoted by P\left(B \mid A\right) :
P\left(B \mid A \right) = \dfrac{P\left(A \cap B\right)}{P\left(A\right)}
At a local library, 65% of the fiction books involve a mystery and 25% of them involve both mystery and romance. If a book involving a mystery is selected at random, what is the probability that it will also involve a romance?
By specifying that the selection is to be made from among the books that contain a mystery, this situation becomes equivalent to finding P\left(B \mid A\right) :
- A is “select a book with mystery”
- B is “select a book with romance"
The probability of choosing a book with both mystery and romance, if the selection is made from the entire sample set (all the fiction books in the library), is 0.25. Therefore:
P\left(A \cap B\right) = 0.25
The probability of randomly choosing a book with mystery out of all of the fiction books in the library, is:
P\left(A\right) = 0.65
Using these values, we find:
P\left(B \mid A \right) = \dfrac{P\left(A \cap B\right)}{P\left(A\right)} = \dfrac{0.25}{0.65}
Observe that if A and B are independent events, i.e. if P\left(A\cap B\right)=P\left(A\right)P\left(B\right), then the conditional probability formula simplifies as:
P\left(A\mid B\right)=\dfrac{P\left(A\cap B\right)}{P\left(B\right)}=\dfrac{P\left(A\right)P\left(B\right)}{P\left(B\right)}=P\left(A\right)
Counting techniques
The counting principle
The counting principle
The counting principle states that the number of ways that a set of independent events could transpire is the product of the number of ways that each event could transpire individually.
Applied to only two independant events, the counting principle would say that if there are m ways for event A to transpire and n ways for event B to transpire, then there are m \times n different ways that events A and B could both transpire.
Determine the total number of outcomes for both flipping a coin and rolling a six-sided dice.
- We assign to variable m the value of the total number of outcomes of flipping a coin: m = 2 .
- We assign to variable n the total number of outcomes of rolling a dice: n = 6.
Applying the counting principle, the total number of outcomes is:
2 \times 6 = 12
Kevin has 2 pairs of pants, 3 shirts, and 4 scarves. Find out how many different unique outfits he can wear.
There are three events here:
- "Kevin wears a pair of pants"
- "Kevin wears a shirt"
- "Kevin wears a scarf"
The first event has 2 possible outcomes, the second has 3, and the third has 4.
The total number of possible different outcomes for all of these events happening together is:
2 \times 3 \times 4 = 24
Permutations
Permutation
A permutation is an ordering of distinct objects.
For the set \{a, b, c\}, the possible permutations are the different ways in which the elements a, b, and c can be ordered:
- \{a, b, c\}
- \{a, c, b\}
- \{b, a, c\}
- \{b, c, a\}
- \{c, a, b\}
- \{c, b, a\}
Recall that if n is a nonnegative integer, then:
n!=\begin{cases} 1&\mbox{if } n=0 \cr \cr 1\cdot 2\cdots n & \mbox{if }n\gt 0 \end{cases}
5!=1\left(2\right)\left(3\right)\left(4\right)\left(5\right)=120
P\left(n,r\right) denotes the total number of possible permutations of a r number of objects selected out of n objects:
P\left(n,r\right) = \dfrac{n!}{\left(n-r\right)!}
If there are 10 people running in a race, but only the first four get ribbons, and the ribbons are distinct from one another, then the total possible number of ways that the ribbons might end up being distributed can be found with P\left(n,r\right) where n = 10 and r = 4 :
P\left(10{,}4\right) = \dfrac{10!}{\left(10-4\right)!} = \dfrac{10!}{6!} = \left(10\right)\left(9\right)\left(8\right)\left(7\right) = 5\ 040
If r = n , then:
P\left(n,r\right) =\dfrac{n!}{\left(n-n\right)!}=\dfrac{n!}{1} = n!
This means that there will always be n! permutations for a set of n objects.
Assume that ten people are in a room with ten chairs.
- For the first chair, there are 10 possible people.
- For the second chair, there are nine possible people/
- Etc.
Therefore, the total number of ways for ten people to sit on ten chairs is:
10\left(9\right)\left(8\right)\cdots \left(2\right)\left(1\right)=10!
Combinations
Combinations
A combination is a set of objects in which the order of the objects in the set does not matter.
A lottery ticket is a winning ticket if the numbers on the ticket match the numbers that are drawn. The order of the numbers do not matter. Therefore, combinations can be used to study odds of winning a lottery.
The formula for calculating combinations is:
{n \choose k} = \dfrac{n!}{\left(n-k\right)! k!}
- n is the total number of objects from which to choose.
- k is the number of objects chosen.
If a philanthropist has 8 books and is going to donate 3 of them to a shelter, then the number of different combinations of books that they could donate is:
{n \choose k} = {8 \choose 3} = \dfrac{8!}{\left(8-3\right)! 3!} = \dfrac{8!}{\left(5!\right) \left(3!\right)} = \dfrac{\left(8\right)\left(7\right)\left(6\right)}{\left(3\right)\left(2\right)\left(1\right)} = \dfrac{336}{6} = 56
The expression {n \choose k} is read as " n choose k."
The expression 10\choose 3 is read as "10 choose 3."
If k=n, then the formula yields 1. This corresponds to saying that there is only one way of selecting n objects from a group of n objects if the order in which they are selected does not matter.
A permutation is for an ordered list in which order matters, while a combination is for groups in which order does not matter. For example, understanding how people finish in a race is a permutation whereas numbers drawn in a lottery are a combination.