The Sum Rule, Conditional Probability, and the Product Rule

The Sum Rule, Conditional Probability, and the Product Rule The Sum Rule is a method of finding the probability of two or more independent eventsoccurring together. It states that the probability of two independent events happening atthe same time is equal to the sum of the probabilities of them happening separately.Conditional Probability is the possibility that a certain event will occur given that anotherevent has occurred. In other words, conditional probability is the likelihood that an eventwill happen if another event has happened.The Product Rule is the sum of products rule for differentiation. It is used to calculate thederivative of a composite function, which is a function formed by multiplying two or morefunctions together. Next what is coming up is marginal probabilities and the sum rule. In probability problems where the joint probabilities of two events are known, one maywish to find out the individual probability of one event occurring regardless of whether ornot the other event occurs. Suppose we have probabilities for events x1, x2, x3… and y1, y2, y3... Then we can referto the probability of x1 as the marginal probability of x1. Something like  being out on themargins of this matrix. The sum rule tells us that the marginal probability of x 1, given thaty is a proper probability distribution, is equal to the sum of the joint probabilities. Thus, theprobability of x1 = 1 +, 1% + 10% + 4% = 15%. Similarly, the marginal probability of y2 equals the sum of the marginal probabilities ofx1y2, x2y2, and x3y2. This would be 79 percent or .79.

That we can add together joint probabilities to get a marginal probability is due tosomething called the sum rule. Those are two versions of the sum rule written out. The first case is for a probability distribution involving two possible outcomes. Theprobability of occurrence of outcome B (event A) is given by the formula P(B) = P(A∩B).The probability of non-occurrence of outcome B (event A) is given by the formula P(not B)= P(A∩not B). The joint probability of occurrence and non-occurrence of both events A andB is given by the formula P(A∩B∩not B) = P(A∩B) +P(not B). Similarly, if we consider the joint probability of a series of events, n events, we can addtogether their individual probabilities to find the marginal probability of A. Therefore, it'sexactly the same principle. Then we will discuss conditional probability. Conditional probability is defined as theprobability that a statement is true given that some other statement is true with certainty. Itis the likelihood that a fact is true given another fact. All items to the right of the dividing line are considered true with certainty. The notation P(A|B) is read: "the probability of A given B." In other words, if B is true, whatis the probability of A? So if I throw a six-sided die and it comes up odd, what is theconditional probability that the number on top is a 3? Three odd rolls, one, three and five,yield a conditional probability of one-third. If I throw a three with certainty, what is the probability that my throw is odd? In this case,the probability is one. It appears odd with certainty if it is three with certainty. When we are working with conditional probabilities, we are dealing with dependent ratherthan independent variables. The general formula for calculating conditional probabilities isthat we take the relevant outcomes, the ones that meet our definition of A, and divide themby the total outcomes in our universe. On the other hand, our results have been diminished because B must be true. So, in theexample of the die I just gave you, we are reducing our options from six possibilities—one,two, three, four, five and six—to the odd possibilities: one, three and five. So there is oneoutcome of three and three outcomes of five. My probability is one-third. The conditional probability of throwing a three if I know that the die is odd is equal to thejoint probability of throwing a three and the die being odd divided by the conditionalprobability of the die being odd. We now want to relate our ideas of joint probability,marginal probability, and conditional probability.

For that one, we need the product rule. The product rule tells us that the conditionalprobability of A given that B is true with certainty is equal to the joint probability that both Aand B are true divided by the marginal probability that B is true. So it's clear that we don't assume the truth of B when it's to the right of the magic line. Thisstatement is what makes it true, as there is no need to equal zero anywhere else in theequation. The product rule allows us to develop a new definition of independence. Our previousdefinition is that the joint distribution is equal to the product distribution, okay? In order to arrive at our new definition, we divide both sides by the probability of B. Wethen have the joint probability of A and B divided by probability of B; and we have theprobability of A given B divided by the probability of B assuming that B is not 0. We can derive the product rule by saying that this term is equal to the probability of Agiven B. And that is equal to the probability of A. Our intuition about what this means is that knowing that B is true tells us nothing about theprobability of A. Therefore, A and B must be independent. The converse is true as well.When the conditional probability of A given B does not equal to probability of A, it meansthey are dependent. Two distributions are either independent or dependent; there is no middle ground.