Chapter 2 Conditional Probability

2.1 Defining Conditional Probability (D&S 2.1)

  1. Out of \(n=17\) students in a class, I will chose one at random student to give gummi bears to. In this class there are \(12\) men and \(5\) women. Student \(A\) is very interested in her probability of being selected. Denote the event \(A\) as being that the student gets the gummi bears. Denote event \(W\) as a woman is selected.
    1. What is the probability that student \(A\) is selected?
    2. What is the probability that a woman is selected?
    3. Suppose I restricted my selection to only the women? What is the probability that student \(A\) is selected. Can you write this probability as a function of your answers in parts (a) and (b)?
  2. Consider the case where we take the sum of 2 six-sided dice. Define \(A\) as the event that the sum is greater than or equal to 9, and \(B\) being the event that the sum is greater than or equal to 6.
    1. Create a table of the possible outcomes. Notice that each outcome is equally likely. What is the probability that \(A\) occurs? Use correct notation and leave it as a fraction \(\frac{???}{36}\)
    2. Suppose that you are told that event \(B\) has occured. How many equally likely outcomes are there and what is the probability that \(A\) occurs? This will be denoted as \(Pr(A|B)\). (Leave this as a fraction).
    3. Notice that you the numerator in answer in part (a) is the same as the numerator as you had in part (b), but the denominators are different. What number do you need to multiple your part (a) answer by to get your part (b) answer. How does this relate to \(P(B)\)?
  3. It seems that my children (Casey and Elise) get sick with annoying frequency. Suppose the probability that my son Casey gets sick is \(Pr(C) = 0.05\) and furthermore the probability that both children get sick is \(Pr(E \cap C) = 0.03\) If Casey is sick right now, what is the probability that Elise is also sick?

When we talk about events \(A\) and \(B\), we defined them as possible outcomes, but it isn’t until we define probability of the events \(Pr(A)\) \(Pr(B)\) that we care about the sample space \(\mathcal{S}\) of all possible events. What we are now trying to do is to claim that some addition knowledge allows us to refine the sample space to some smaller subset of \(\mathcal{S}\), perhaps \(B\subset \mathcal{S}\). So for events in \(B\), we now need to re-scale all the probabilities to reflect that we now know that event \(B\) did happen.

If we previously know that \(Pr(A\cap B)=0.3\) and \(Pr(B)=0.6\), then half of the probability associated with \(B\) is overlapping with \(A\). So if we just restrict ourselves to cases where \(B\) occurs, then the probability that \(A\) will occur is \(1/2\).

Notice that our notation \(Pr( A | B)\) is addressing the refinement of the sample space, but we’ve just defined our notation this way. We have not, and will not ever define \(A|B\) because event \(A\) is event \(A\), regardless of the sample space we use to figure out its probability.

We formally define the conditional probability of \(A\) given \(B\) as \[Pr(A|B) = \frac{ Pr(A \cap B)}{Pr(B)} \;\;\; \textrm{Assuming } Pr(B) \ne 0\] If \(Pr(B)=0\) then the conditional probablity is undefined. Notice that this can be happily re-arranged to \[ Pr( A \cap B) = Pr(A | B) Pr(B) \] I can also notice that I could condition on either event \(A\) or event \(B\), so we could also have \[Pr( A \cap B) = Pr(B|A)Pr(A)\]

  1. If \(B \subset A\) and \(Pr(B)>0\), what is \(Pr(A|B)\)

  2. If \(A\) and \(B\) are disjoint and \(Pr(B)>0\), what is \(Pr(A|B)\)?

  3. Suppose that events \(A_1,A_2,\dots,A_n\) are events such that \(Pr\left( \bigcap_{i=1}^n A_i \right) > 0\). Show that \[Pr\left( \bigcap_{i=1}^n A_i \right) = Pr\left( A_n \Big\rvert \bigcap_{i=1}^{n-1}A_i \right) Pr\left( A_{n-1} \Big\rvert \bigcap_{i=1}^{n-2}A_i \right)\dots Pr\left( A_3 \Big\rvert A_2 \cap A_1 \right) Pr\left( A_2 \Big\rvert A_1 \right) Pr\left( A_1 \right)\]

  4. For events \(A\), \(B\), and \(D\) such that \(Pr(D)>0\) show that:
    1. \(Pr(A^C | D) = 1 - Pr(A | D)\).
    2. \(Pr( A \cup B \Big\rvert D) = Pr(A | D) + Pr(B|D) - Pr( A \cap B | D)\)

Suppose that we have \(K\) events \(B_k\) such that the \(B_1, B_2, \dots, B_K\) are disjoint and \(\bigcup B_k=\mathcal{S}\). Then we call the events \(B_1,\dots,B_K\) a partition of the sample space \(\mathcal{S}\). A partition of \(\mathcal{S}\) is often useful for caculating probabilities due to the disjoint nature of the \(B_k\) elements.

  1. Law of Total Probability Prove that for for a partition \(B_1, \dots, B_K\) of \(\mathcal{S}\), that \[Pr(A) = \sum_{k=1}^K Pr(A \cap B_k) = \sum_{k=1}^K Pr(A \Big\rvert B_k) Pr( B_k )\]

Note: There is a conditional version of the Law of Total probability, which is proved in an analogous fashion: \[Pr(A|C) = \sum_{k=1}^K Pr(A \cap B_k \big\rvert C) = \sum_{k=1}^K Pr(A \big\rvert B_k \cap C) Pr( B_k \big\rvert C)\]

  1. A child’s bookshelf contains three shelves. On the shelves are \(n_1=10\),\(n_2=20\) and \(n_3=30\) books. Within each set of books, there are some number of Dr Suess books \(m_1=5\), \(m_2=4\), \(m_3=2\). The child will select a shelf at random (equal probability) and then from the shelf will select a book at random. What is the probability the child selects a Dr Suess book?

  2. A camera with a motion detector was mounted facing a forest trail. \(50\%\) of the pictures were taken during the daytime, \(15\%\) were taken during twilight hours (dawn and dusk), and \(35\%\) were taken during the night. Of the pictures taken during the daytime, \(80\%\) were of hikers and \(20\%\) were of wild animals. Of the pictures taken at twilight \(30\%\) were of hikers and \(70\%\) were of wild animals. Finally, of the pictures taken during the night, \(100\%\) were wild animals.
    1. What is the probability that a randomly selected photo is of a hiker and was taken at twilight?
    2. What is the probability a photo was taken at night given that is of a wild animal?

  3. I have kept track of the probabilities of how many cats will sit with me and/or my wife on the couch. Below is a table of probabilities.

    0 Cats 1 Cat 2 Cats 3 Cats
    0 People 0.08 0.08 0.03 0.01
    1 Person 0.1 0.25 0.125 0.025
    2 People 0.03 0.09 0.12 0.06
    1. What is the probability that one human is sitting on the couch?
    2. What is the probability that at least two cats are sitting on the couch?
    3. Given that there are two cats sitting on the couch, what is the probability that there are two humans also on the couch?

  4. My cat Kaylee occasionally likes to sit on people’s laps while they are seated at the table. My wife is strongly opposed to this and will scold the cat when she catches her in the act. Suppose that that Kaylee will select my lap \(60\%\) of the time and the remaining \(40\%\) of the time she jumps into my wife’s lap. If Kaylee jumps into my wife’s lap, there is a \(100\%\) chance of being scolded, while if she jumps into mine, there is only a \(20\%\) chance of being scolded. Given that Kaylee was just scolded for being in a lap, what is the probability she was in my wife’s lap?

  5. There are two brands of Mac & Cheese that my daughter will eat. When I go shopping I will pick from the two brands with a \(70\%\) probability of choosing the brand that I bought the previous time. The first time I went shopping, I chose from the two brands with equal probability. What is the probability that I chose brand \(A\) on the first and second trips, and brand \(B\) on the third and fourth trips?

2.2 Independence

Definition: Two events, \(A\) and \(B\) are independent if \(Pr( A \cap B) = Pr(A)Pr(B)\).

Definition: Events \(A_1, A_2,\dots,A_K\) are pairwise independent if \(A_i\) and \(A_j\) are independent for any \(i,j\).

Definition: Events \(A_1, A_2,\dots,A_K\) are mutually independent if for all subsets \(I\) of \(1,2,\dots,K\), \(Pr( \bigcap_{i\in I} A_i) = \prod_{i \in I} Pr(A_i)\)

  1. Show that if \(Pr(A)>0\) and \(Pr(B)>0\), then \(A\) and \(B\) are independent if and only if \(Pr(A|B)=Pr(A)\) and \(Pr(B|A)=Pr(B)\)

  2. Give an example of three events that are pairwise independent but not mutually independent.

Convention If I say that a set of events are “independent”“, then we intend to say”mutually independent“” but are being lazy.

  1. Show that if \(A\) and \(B\) are indendent, then \(A\) and \(B^c\) are also independent.

  2. Suppose that we flip a fair coin three times. Denote \(H_i\) as the event that I flip a head on the \(i\)th flip.
    1. Find \(Pr( H_1 \cap H_2 \cap H_3 )\)
    2. Find \(Pr( H_1 \cap H_2^c \cap H_3)\)
    3. Find \(Pr( H_1^c \cap H_2 \cap H_3)\)
    4. How many ways can we have 2 heads?
    5. What is the probability of 2 heads?
  3. I will roll a 20-sided die three times. Define the event \(H_i\) as the event that I roll a 17 or greater on the \(i\)th roll.
    1. Find \(Pr( H_1 \cap H_2 \cap H_3 )\)
    2. Find \(Pr( H_1 \cap H_2^c \cap H_3)\)
    3. Find \(Pr( H_1^c \cap H_2 \cap H_3)\)
    4. How many ways can we have exactly 2 \(H\) events happen?
    5. What is the probability of exactly 2 \(H\) events happening?
  4. I will roll my 20-sided die until I roll a 20.
    1. What is the probability that I roll a 20 on my first roll?
    2. What is the probability that the first 20 I roll is on the \(5\)th roll?

  5. A family has two children. It is known that at least one is a boy. What is the probability that the family has two boys, given that at least is one a boy? Assume that genders are equally likely and that genders of siblings are independent.

2.3 Bayes’ Theorem

The goal of Bayes’ Theorem is to reverse the order of conditioning. Suppose we are interested in two events \(A\) and \(B\). We might be given some information about \(P(A|B)\) but we want to know about \(P(B|A)\).

  1. Bayes’ Theorem Prove that for two events \(A\) and \(B\) such that \(Pr(A)>0\) then \[\begin{aligned} Pr(B|A) &= \frac{Pr(A|B)Pr(B)}{Pr(A|B)Pr(B)+Pr(A| B^c)Pr(B^c)} \end{aligned}\]

  2. Bayes’ Theorem (general case) Again consider an event \(A\) such that \(Pr(A)>0\). For an arbitrary partition of \(\mathcal{S}\), say \(B_k\) \(k=1,\dots,K\), prove that
    \[\begin{aligned} Pr(B_k|A) &= \frac{Pr(A|B_k)Pr(B_k)}{\sum_{i=1}^K Pr(A| B_k)Pr(B_k)} \end{aligned}\]

  3. A softball team has two pitchers, Jeff and Bob. Of the two Jeff is the better pitcher and wins 80% of the games he pitches for but can only play in 30% of the games. Bob pitches in the rest of the games, but only wins 40% of his games.
    1. What is the probability that the softball team wins a game?
    2. Given that the team won, what is the probability that Jeff pitched?
  4. One card is selected at random from a standard deck of 52 playing cards. It is inserted into a second standard deck and the second deck is then well shuffled.
    1. A card is drawn at random from the second deck. What is the probability it is an ace?
    2. Given that an ace was drawn from the second deck, what is the probability that an ace was transfered from the first deck?
  5. My three cats love licking up the milk out of my cereal bowl if I leave it unattended. If unattended, there is a 30% chance that Beau will clean the bowl, a 50% chance that Tess will, and 20% chance that Kaylee will. Unfortunately the milk makes the cats nauseous and if a cat gets milk there is a good chance the cat will puke. In particular the probability that Beau will puke given he has had milk is 30%, for Tess it is 60%, and for Kaylee it is 40%. My daughter recently left a cereal bowl out and a cat finished the milk.
    1. What is the probability that a cat has puked as a result.
    2. Given that a cat has puked in response, what is the probability it was Kaylee?

  6. I have two decks of cards. The first deck has 40 red cards and 10 black. The second deck has 25 red and 25 black. I select a deck at random, and then draw two cards. Given that I’ve selected two red cards, what is the probability that I initially chose the first deck?

  7. An inexpensive and convenient enzyme immunoassay screening tests for HIV in a human. If the person is actually HIV negative then the test returns negative with a probability of \(0.985\). If the person is HIV positive, the test returns a positive result with probability \(0.9997\). HIV is a major epidemic in Sub-Saharan Africa with approximately 5% of the adult population having HIV. Major aid organizations want to help identify people with HIV for treatment and will use this cheap and convenient test in their efforts. Suppose that an adult in Sub-Sahran Africa is selected and tested and the test result is that the person has HIV. What is the probability that the person actual has HIV?