Continuous distributions

Slide 1: Continuous Probability Distributions

(No text on this slide.)

Slide 2: Continuous Distributions

(No text on this slide.)

Slide 3: Relationship between PDF and CDF

• Recall that the CDF is P[ X ≤ x]
• CDF is integral of PDF. PDF is derivative of CDF

FX(x) = ∫−∞x fX(t) dt

Slide 4

Slide 5

Note the inequality. It really matters for discrete distributions

Slide 6: Expected Value

The expected value of a continuous random variable is:

Just like the discrete case but with an integral instead of summation!

Slide 7: Variance

• The variance of a continuous random variable is given by:
• Which is the same as E{(X - µ )2}

Recall our derivation showed that

Slide 8: The Normal Distributionaka Gaussian

(No text on this slide.)

Slide 9: Introduction

• The Normal (or Gaussian) distribution is the most used distribution in statistics.
• Normal random variables are continuous (they can take any value on the real line) so the Normal distribution is an example of a continuous probability distribution.

Slide 10: The Normal Distribution-I

μ

μ-σ

μ-2σ

μ-3σ

μ+3σ

μ+2σ

μ+σ

• The graph of the normal distribution
• is called the normal (or bell) curve.

Total area=1

• Inflection
• points

Slide 11: The Normal Distribution-II

• The mean, median, and mode are the same.
• The normal curve is symmetric about its mean.
• The total area under the normal curve is one.
• The normal curve approaches, but never touches, the x-axis.

Slide 12: The Normal Distribution-III

• Notation
• We say that the random variable X is normally distributed with mean μ and standard deviation σ
• The equation defining the normal curve is:
• f (x) is referred to as a probability density function (pdf).

X ~ N(μ, σ2)

f(x) = 1 / (√(2π) σ) · exp( −(x − μ)2 / (2σ2) )

Slide 13: Normal Cumulative Distribution Function-Notice that all CDFs are non-decreasing

Recall that the CDF is P[ X ≤ x]

Slide 14: Relationship between PDF and CDF

• Recall that the CDF is P[ X ≤ x]
• CDF is integral of PDF. PDF is derivative of CDF

FX(x) = ∫−∞x fX(t) dt

Slide 15: Normal Distributions and Probability-II

99.7%

95%

68%

34%

13.5%

• Of the total area under the curve about:
• 68% lies between μ-σ and μ+σ
• 95% lies between μ-2σ and μ+2σ
• 99.7% lies between μ-3σ and μ+3σ

μ

2.5%

2.5%

Really 1.96 SD, but ~2

Slide 16: Example

• The mean length of a fish of age 5 is 40cm and the standard deviation is 4. What is the approximate probability that the length of a randomly selected fish of age 5 is less than 48cm?
• 48cm is two standard
• deviations above the
• mean so the area
• to the left of 48cm is
• 0.5+0.475 = 0.975.

Group Work

Slide 17: Example

• An anchovy net can hold 10 t of anchovy. The mean mass of anchovy school is 8 t with a standard deviation of 2 t. What proportion of schools cannot be fully caught?
• Hint: we are interested in the proportion of schools greater than 10t.
• ie. 1- P(X ≤ 10) =

Slide 18: The Standard Normal Distribution

The normal distribution with a mean of 0 and a standard deviation of 1 is called the standard normal distribution

• We can make a normal RV Y with any mean and variance
• Y = μ + σz

z = (value − mean) / (standard deviation) = (x − μ) / σ

Slide 19: Further Examples

• For standard normally distributed RV X, find the probability that:
• X ≤ -0.95
• X ≥ 1
• 0.1 ≤ X ≤ 0.5 OR 1.5 ≤ X ≤ 2

Group Work

Slide 20: Sampling and Sampling Distributions

A sampling distribution is the probability distribution of a sample statistic that is formed when samples of size n are repeatedly taken from a population. If the sample statistic is the sample mean, then the distribution is the sampling distribution of sample means.

Why be Normal?

Slide 21: Sampling and Sampling Distributions-II(Example)

• Consider a population of fish in a lake. The mean and standard deviation of the lengths of these fish are 300 mm and 50 mm respectively.
• Take 100 random samples where each sample is of size 10, 20, or 100.
• What can we learn about the population mean and how does this depend on the size of our sample?

Slide 22: Sampling and Sampling Distributions-IV(Example)

N=10

N=20

N=100

We know the population we sampled from has mean of 300 and standard deviation of 50

Slide 23: Properties of the Sampling Distribution for the Sample Mean

• The mean of the sample means is equal to the population mean:
• The standard deviation of the sample means is equal to the population standard deviation divided by the square root of n.
• σx̄ is often called the standard error!

μx̄ = μ

σx̄ = σ / √n

Slide 24: The Central Limit Theorem(Why we use the Normal distribution so much!)

• The sum (and therefore mean) of random variables of any distribution tends toward a normal distribution as sample size increases
• If samples of size n (where n ≥~30) are drawn from any population with a mean μ and a standard deviation σ, the distribution of the means approximates a normal distribution. The greater the sample size, the better the approximation.
• If the population is itself normally distributed, the sampling distribution of the sample means is normally distributed for any sample size.

• Compare hist(rgamma(1000,1,1)) with
• hist(rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1)+rgamma(1000,1,1))

Slide 25: The Central Limit Theorem(Example)

Slide 26: Approximating a Binomial Distribution

• It is possible to compute approximate probabilities for the binomial distribution using the normal distribution.
• If np ≥ 5 and nq ≥ 5, the binomial random variable X ~B(n,p) is approximately normally distributed:
• This approximation can be quite accurate even though the binomial distribution is discrete and the normal distribution is continuous.

These are mean and stdev of a binomial random variable

X ≈ N(np, √(npq))

Slide 27: Approximating a Binomial Distribution-II

• BUT we need an interval when defining the probability associated with a discrete value for x using the normal distribution, i.e. PB(X=x) is approximated byPN(x-Δ<X<x+Δ).
• We call Δ the correction for continuity and set it to 0.5.

Slide 28: Approximating a Binomial Distribution-III

• n=30
• p=0.25

Slide 29: Approximating a Poisson Distribution

• It is possible to compute approximate probabilities for the Poisson distribution using the normal distribution.
• If μ>10, the Poisson random variable X ~P(μ) is approximately normally distributed:

• In R, compare
• hist(rpois(10000,50))
• hist(rnorm(10000,50,sqrt(50)))

These are mean and stdev of a Poisson random variable

X ≈ N(μ, √μ)

Slide 30: Small Samples and the t-Distribution

• The sampling distribution of a test statistic (like a mean or a regression coefficient) is Normal.
• Dividing a Normal by its standard deviation (called the standard error) makes a standard Normal, z . This is used in hypothesis testing.
• If |z value| > 1.96, then p<0.05.
• However, the standard error depends on the sample standard deviation s, which is not precisely known.
• When the population standard deviation is unknown and estimated (and n<~30), the sample mean is t-distributed rather than Normally distributed.

σx̄ ≈ s/√n

Slide 31: Side Note – Stout and Statistics.

• The t-distribution is often referred to as “Student’s” t-distribution.
• “Student” (W.S. Gossett) was an employee of the Guinness Brewing Company and was prohibited from publishing papers (after someone published trade secrets) so he published under the pseudonym Student for his publications to avoid detection of his publications by his employer

Slide 32: Properties of the t-distribution-I

• Bell-shaped and symmetric like Normal
• But can have “fattails”, i.e. falls to zero less steeply
• The mean, median and mode are equal to 0.
• The t-distribution is a family of curves. Each element of the family is determined by a parameter called the degrees of freedom (n-1 for the case of the sample mean)
• Smaller samples have more uncertainty about standard deviation and so tails are fatter, but as sample size increases this approaches the standard normal distribution because standard deviation is well estimated.

Slide 33: Properties of the t-distribution-II

df=2

df=5

Taal, open R and show estimate and standard error produce t

Estimate

True Mean. Typically zero for hypothesis testing

Standard error

t = (x̄ − μ) / (s / √n)

Slide 34: The Lognormal distribution

• The log of a lognormal random variable is Normally distributed!
• If X~Normal(μ, σ2), then Y=eX ~ lognormal(μ, σ2)

• Lognormal(μ, σ2)
• Parameters: μ ∈ (−∞, +∞), σ > 0
• Support: x ∈ (0, +∞)
• PDF: f(x) = 1/(x σ √(2π)) · exp( −(ln x − μ)2 / (2σ2) )

• Mean: exp(μ + σ2/2)
• Median: exp(μ)
• Mode: exp(μ − σ2)
• Variance: [exp(σ2) − 1] · exp(2μ + σ2)

Slide 35: Remember the Gamma distribution

Flexible distribution for positive data. VERY useful in statistical modelling

Standard in R

• E[X] = kθ (shape-scale)
• E[X] = α/β (shape-rate)

• Params — shape-scale: k>0, θ>0; shape-rate: α>0, β>0 (std in R)
• Support: x ∈ (0, ∞)
• PDF (shape-scale): f(x) = 1/(Γ(k) θk) · xk−1 · e−x/θ
• PDF (shape-rate): f(x) = βα/Γ(α) · xα−1 · e−βx

• Var(X) = kθ2 (shape-scale)
• Var(X) = α/β2 (shape-rate)

Slide 36: The Beta distribution

Distribution for proportions

How might this relate to the binomial distribution?

What does a Beta(1,1) become?

E[X] = α / (α + β)

• Notation: Beta(α, β)
• Parameters: α > 0 shape (real), β > 0 shape (real)
• Support: x ∈ [0, 1] or x ∈ (0, 1)
• PDF: f(x) = xα−1 (1−x)β−1 / B(α, β)
• where B(α, β) = Γ(α) Γ(β) / Γ(α + β)

Slide 37: The Exponential DistributionDistribution for time between random events (e.g. survival)

• Parameters: λ > 0, rate (or inverse scale)
• Support: x ∈ [0, ∞)
• PDF: f(x) = λ e−λx
• CDF: F(x) = 1 − e−λx
• Quantile: −ln(1−p) / λ
• Mean: 1/λ
• Median: ln 2 / λ
• Mode: 0
• Variance: 1/λ2

Slide 38: The Exponential Distribution-I

• The Poisson distribution arose when counting the number of random events in an interval. The exponential distribution considers the interval of time between random events.
• If events are occurring at random with average rate of λ per interval, then the probability density function for the length of time between events is:

SD and Mean =

f(x) = λ e−λx

Slide 39: This Poisson – Exponential relationship is basis for animal density estimation from camera trap data

# animals seen by camera

Time between Poisson observations

Density is lambda divided by mean area in camera viewshed

T ~ Exp(λ)

D = λ / ā

Nijk ~ Pois(λ)

Slide 40: The Exponential Distribution-II(Example-I)

• If a herring spawns 1.5 times per week on average, what is the probability that it doesn’t spawn for 2 weeks?
• Note that this is a continuous probability distribution so we are looking for the area under the curve
• Can be done analytically or in R

(wks)

Group Work

Slide 41: The Exponential Distribution-II(Example-II)

• Solution:
• P[X>2] = 1-P[X≤2]=0.0498

Slide 42: The Uniform Distribution

• This is the simplest possible continuous distribution. It is sometimes used to deal with the situation where all values in some interval (a, b) are equally likely.
• Notation:
• The probability density function for the uniform distribution is:

Can you verify that this is a proper PDF? Use calculus or geometry.

Group Work

X ~ U(a, b)

f(x) = 1 / (b − a)

Slide 43: The Uniform Distribution(Example)

Suppose the mass of a nominally 500 kg bag of fish is equally likely to lie between 480 and 505 kg. What is the probability that the bag contains at least the advertised mass of fish?

Group Work

Can you verify that this is a proper PDF? Use calculus or geometry.

1

2

f(x) = 1 / (b − a)

Slide 44: Which distribution and why?

• The mass of carbon in above ground biomass in square m plot.
• The number of seals on a haul-out beach in the gulf of AK.
• Presence or absence of an invasive species in forest patches.
• The probability that a white male will vote republican in a presidential election.
• The number of individuals in four mutually exclusive income categories.
• The number of diseased individuals in a sample of 100.
• The political party affiliation (democrat, republican, independent) of a voter.

Group Work

Slide 45

(No text on this slide.)

Slide 46