Introduction to probability

Slide 1: FW536 – Statistical modeling for ecology and conservation AKA Get your data on!

(No text on this slide.)

Slide 2: What is this course?

Goal: Provide a strong statistical foundation and transferable skills to help you (1) understand and analyze data, and (2) conduct more complex statistical modeling

We are going for deep understanding not just implementing R code.

Slide 3: Course oath

• “I agree to ask questions when something is not clear or I am confused.
• I will strive to work together to help one another.
• I will not be afraid to look stupid because we’re all just apes in a room”

Slide 4: Grading

• Precourse exam (50 points)
• Labs (180 points)

Slide 5: This course is just a jumping off point

• Train yourself by reading books.
• Consider pursuing side project using methods you want to learn
• Consider attending weekly math/stats seminars

Slide 6

• one sample t-test
• two sample t-test
• paired t-test
• Analysis of Variance (ANOVA)
• Analysis of Covariance (ANCOVA)
• Chi.sq test
• linear regression
• multiple regression
• logistic regression

GLM: the superset of all stuff you learned

• Useful to get started.
• But a really stupid way to learn.

• General linear model vs.
• Generalized linear model?

Slide 7

• one sample t-test
• two sample t-test
• paired t-test
• Analysis of Variance (ANOVA)
• Analysis of Covariance (ANCOVA)
• linear regression
• multiple regression
• logistic regression

GLM: the superset of almost all stuff you learned

• Is mean of one measured continuous thing different from what you expect?
• Are means of two measured continuous things different from each other?
• Are means of two measured continuous things collected in a paired treatment (e.g. same subject at two times) different?
• X=factor, Y=continuous
• X=both factor and continuous, Y=continuous
• X=continuous, Y=continuous
• X1, X2…=continuous, Y=continuous
• X=continuous, Y=binary (mostly taught this way) – Note that is a “generalized” linear model because the errors are not Normal

Slide 8

Slide 9

This can be generalized further

Slide 10

• Probability vs. Statistics
• Different subjects: both about random processes
• Probability
• Logically self-contained
• A few rules for computing probabilities
• One correct answer
• Statistics
• Messier and more of an art
• Get experimental data and try to draw probabilistic conclusions
• No single correct answer
• But built on probability

Slide 11: Probability Experiments

A probability experiment is a process or action that can be repeated under the same conditions and results in well-defined outcomes, but the outcome is uncertain or random. Each possible outcome of the experiment has an associated probability, which is a measure of how likely that outcome is to occur.

Slide 12: Probability Experiments

Slide 13: Probability Experiments

Slide 14: Classical Probability

In probability theory, we are concerned with outcomes of “experiments,” broadly defined. We let S be all the possible outcomes (often called the sample space) and A, B, etc., particular outcomes that might interest us (Figure 3.1a). We then define the probability that A occurs, denoted by Pr{A}, by

Pr{A} = (Area of A) / (Area of S) (3.1)

Slide 15: Classical Probability-II(Example)

• What is the (classical) probability of selecting the King of Clubs from a pack of 52 cards?
• What are the sample space and event?
• If you drew a card from a pack and it was the King of Clubs, what is the probability of then randomly drawing another club?
• What are the sample space and event?

Slide 16: DNA Example

• DNA is made of sequences of nucleotides: A, C, G, T.
• How many DNA sequences of length 3 are there? (This is a sample space of all possible outcome)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• How many DNA sequences of length 3 are there with no repeats? (This could be an event)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• What is the probability that a DNA sequence of length 3 has no repeat nucleotides?

Slide 17: DNA Example

• DNA is made of sequences of nucleotides: A, C, G, T.
• How many DNA sequences of length 3 are there? (This is a sample space of all possible outcome)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• How many DNA sequences of length 3 are there with no repeats? (This could be an event)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• What is the probability that a DNA sequence of length 3 has no repeat nucleotides?

Slide 18: DNA Example

• DNA is made of sequences of nucleotides: A, C, G, T.
• How many DNA sequences of length 3 are there? (This is a sample space of all possible outcome)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• How many DNA sequences of length 3 are there with no repeats? (This could be an event)
• (i) 12 (ii) 24 (iii) 64 (iv) 81
• What is the probability that a DNA sequence of length 3 has no repeat nucleotides?

Slide 19: Probabilities Formalized

• A probability cannot be negative and cannot exceed 1, i.e.:
• The complement of the event E is the set of outcomes not part of E. The probability of the complement of E (denoted E ’) is:

Why “1” here?

0 ≤ P(E) ≤ 1

P(E') = 1 − P(E)

Slide 20: Example

• The sex ratio at birth for a particular pinniped is 55:45 male:female. You select two pups at random from the (large) population. What is the probability that:
• a. they are both male?
• b. at least one of them is male?
• Solution:
• We can assume that the population is “large” (not 100!)
• P(male) * P(male) = 0.55*0.55
• P(at least one male) = 1-P(both female) = 1 - 0.45*0.45

How would this be different if the population wasn’t large?

Slide 21: Conditional Probability-I

• A conditional probability is the probability of an event occurring given that another event has already occurred.
• The conditional probability of event B occurring given that event A has already occurred is denoted P(B|A) and is read as “probability of B given A”

Slide 22: Conditional Probability-II

• What is the probability of being a spring run fish?
• What is the probability of being a spring run fish if you have the gene?

Slide 23: Conditional Probability-III

• What is the probability of being a spring run fish?
• Solution: This question says nothing about the other factor (gene)
• so we look at the last column (total); =52/102

Slide 24: Conditional Probability-IV

• What is the probability of being a spring run fish if you have the gene?
• Solution: The question relates only to having the gene so we are dealing with a conditional probability. We focus on the “Gene present” column and find the probability of being a spring run fish GIVEN having the gene = 33/72.

Slide 25: Conditional Probability-V

Mangel’s Toolbox

Now suppose we are told that B has occurred. We may then ask, what is the probability that A has also occurred? The answer to this question is called the conditional probability of A given B and is denoted by Pr{A|B}. If we know that B has occurred, the collection of all possible outcomes is no longer S, but is B. Applying the definition in Eq. (3.1) to this situation (Figure 3.1a) we must have

Pr{A|B} = (Area common to A and B) / (Area of B) (3.4)

Slide 26: Independent Events-I

• Two events are said to be independent if the occurrence of one does not affect the probability of the other
• Two events are independent therefore if:

P(A) = P(A | B); P(B) = P(B | A)

Slide 27: “AND” - The Multiplication RuleThe “joint distribution” of A and B

• The probability that two events A and B will occur in sequence is:
• If A and B are independent, then the multiplication rule becomes:

P(A and B) = P(A ∩ B) = P(A) · P(B | A)

P(A and B) = P(A) · P(B)

Slide 28: “AND” - The Multiplication RuleThe “joint distribution” of A and B

Note the multiplication. What’s another way to write this?

Pr{A, B} = (Area common to A and B) / (Area of S)

P(A and B) = P(A ∩ B) = P(A) · P(B | A)

Slide 29

Factoring joint distributions into conditionals “the chain rule”

P(A, B, C, D) = P(A|B,C,D)P(B|C,D)P(C|D)P(D)

• The probability of multiple events (or random variables) is called the joint distribution of A and B.
• The joint distribution can be factored into conditional distributions

The conditional distributions can further be simplified if variables are independent

Slide 30

The conditional distributions can be represented by a Directed Acyclic Graph (DAG) based on the dependencies of the process

• Variables with arrowheads: LHS of conditionality
• They depend on tails of arrows, which go on RHS
• Variables without arrow pointing to them are independent

Slide 31

Bayesian models are often represented by DAGs

Slide 32: “OR” - The Additive Rule

• The probability that events A or B will occur is:
• If events A and B are mutually exclusive, then:

Note the addition

Why?

P(A or B) = P(A ∪ B) = P(A) + P(B) − P(A and B)

P(A or B) = P(A) + P(B)

Slide 33: “OR” - The Additive Rule

The probability that events A or B will occur

To avoid double count

P(A or B) = P(A ∪ B) = P(A) + P(B) − P(A and B)

Slide 34: Using the Rules of Probability(Example)

• The above table is based on data for 409 randomly selected blood donors
• What is the probability that a donor has type O or type A blood?
• What is the probability that a donor has type B blood and is Rh-negative?
• What is the probability that a donor has type A blood and is Rh-negative?
• What is the probability that a donor is Rh-positive given he / she has
• blood type O?

Slide 35: Review of Concepts and Formulae

• P(A) : probability of event A
• P(A') = 1 − P(A) : probability of the complement of the event A
• P(A) = P(A and B) + P(A and B') : decomposition of the probability of event A
• P(B | A) = P(A and B) / P(A) : conditional events
• P(B | A) = P(B) : independent events
• P(A and B) = P(B | A) · P(A) : multiplication rule
• P(A and B) = P(A) · P(B) : independent events
• P(A or B) = P(A) + P(B) − P(A ∩ B) : addition rule
• P(A or B) = P(A) + P(B) : mutually exclusive events

Slide 36: QUESTION: Prove the equality in Bayes rule using simple probability rules

P(A | B) = ( P(B | A) · P(A) ) / P(B)

Slide 37: BREAK: Up Next - Operationalizing Probability

(No text on this slide.)

Slide 38: Random Variables-I

• A random variable X represents a numerical value associated with each outcome of a probability experiment.
• “Random” implies that there is an element of chance involved. Typical random variables:
• Time that a tagged animal is at large.
• Number of offspring produced by a female Panda bear in one year / over her lifespan.
• Length / weight of a sampled fish

Slide 39: Random Variables - II

• Notation:
• Convention is to use upper case letters to denote random variables and lower case letters to denote particular values of random variables.
• The probability that the random variable X has value x is written as P (X=x).

Slide 40: Random Variables - III

• A random variable is:
• Discrete if it has a countable number of possible outcomes that can be listed.
• Continuous if it has an uncountable number of possible outcomes (or the number of outcomes cannot be listed). Continuous random variables can (at least conceptually) be measured to any degree of accuracy.

Slide 41

• The number of scales forming the lateral line of a fish?
• Discrete random variable
• Max swimming speed of a blue whale?
• Continuous random variable
• Number of reproductive events of a Fathead minnow each year?
• Discrete random variable

Random Variables - III

Slide 42: Probability Distributions (Discrete-I)

• A discrete probability distribution lists each possible value the random variable can take, together with its probability.
• typically called a probability mass function (PMF)
• From our earlier discussion of probability, it must be true that:
• The probability of each value of the discrete random variable must lie between 0 and 1.
• The sum of all the probabilities must equal 1.

Slide 43: Probability Distributions (Discrete-II)

• Mathematically:
• Let the possible values for the random variable be:
• By the first condition:
• By the second condition:

The probability of each value of the discrete random variable is between 0 and 1, inclusive.

The sum of all the probabilities is 1.

{xi : i = 1, …, N}

0 ≤ P(xi) ≤ 1

Σi=1N P(xi) = 1

Slide 44: Discrete Cumulative Distribution Function

• The probability that random variable X takes values less than or equal to a specific value x
• It is common for f(x), or here p(x), to refer to the probability mass function (what people usually mean when they say “distribution”) and F(x) to refer to the cumulative distribution function

F(x) = P(X ≤ x) = ∑x_i ≤ x P(X = xi) = ∑x_i ≤ x p(xi)

Slide 45: Discrete Cumulative Distribution Function

• The probability that random variable X takes values less than or equal to a specific value x
• The probability that X is between a and b is
• Notice the inequalities are not the same. Why?

F(x) = P(X ≤ x) = ∑x_i ≤ x P(X = xi) = ∑x_i ≤ x p(xi)

P(a < X ≤ b) = FX(b) − FX(a)

Slide 46: Means, Variances and Standard Deviations-I

The mean of a discrete random variable is given by:

Each value of x is multiplied by its corresponding probability and the products are added.

This is just a weighted average!

μ = Σi=1N xi P(xi)

Slide 47: Expected Value

The expected value of a discrete random variable (jargon for the mean) is:

This is just a weighted average!

E(X) = μX = Σi=1N xi P(xi)

Slide 48: Means, Variances and Standard Deviations-II

• The variance of a discrete random variable is given by:
• Which is the same as E{(X - µ )2}
• Question: show that an equivalent definition is

Var{X} =

Recall that

Var{X} = E{X2} - (E{X})2

σ2 = Σi=1N (xi − μ)2 P(xi)

E(X) = μX = Σi=1N xi P(xi)

Slide 49

Slide 50

Computation from tables

Example. Compute the variance and standard deviation of X.

Slide 51

Computation from tables

From the table we compute the mean:

μ = 1/10 + 4/10 + 12/10 + 8/10 + 5/10 = 3

Then we add a line to the table for (X − μ)2.

Using the table we compute variance E((X − μ)2):

1/10 · 4 + 2/10 · 1 + 4/10 · 0 + 2/10 · 1 + 1/10 · 4 = 1.2

The standard deviation is then σ = √1.2.

Slide 52: Algebra with Expectation and Variance

Slide 53: Algebra with Expectation and Variance

• Expectation is a linear operator that can be distributed across a function of a random variable, X, but variance is not
• What is E{a + bX}
• What is Var{a + bX}

Slide 54: Algebra with Expectation and Variance

If X and Y are independent, what is Var{X + Y}? Let’s derive rather than look at rules

Recall: Var{X} = E{X2} - (E{X})2

Slide 55: Answer: Algebra with Expectation and Variance

• If X and Y are independent, what is Var{X + Y}?
• Var{X +Y} = E{X2 + 2XY + Y2} – (E{X} + E{Y})2
• = E{X2} + 2E{XY} + E{Y2} – (E{X}2 + 2E{X}E{Y} + E{Y} 2)
• = E{X2} – E{X}2 + E{Y2} – E{Y}2
• Var{X +Y} = Var{X} + Var{Y}

Recall: Var{X} = E{X2} - (E{X})2

So, variances of independent random variables add!

Slide 56: Key terms moving forward

• Probability mass function, PMF
• Cumulative distribution function, CDF
• Expectation/Expected value
• Variance
• Random variable