Data, Statistics and Probability

Lecture 5: Continuous Random Variables

Jul-Nov 2022

Real-valued outcomes

  • Many physical observations and measurements are quantified as real numbers
    • is assumed to be intuitively defined; precise mathematical definition is arduous
  • If the outcomes are in a discrete set of real numbers, we can use discrete random variables
    • Actually, we do not really need real numbers in this case
    • Map the outcomes to
  • If the outcomes can fall inside an interval, such as , what do we do?
    • Example: weight of an asteroid entering earth's atmosphere
      • spread over 0.01 grams to 60 tonnes
    • Quantizing to a discrete set (such as weight up to 3 decimal places) is too arduous
  • An interval has uncountably many real numbers
    • How to efficiently define "atomic" events and probability function?

Events when

Sample space:
Events: all intervals in , their unions, intersections, complements

  • What kind of events are there?
    • : interval . So, any discrete set is covered.
    • Any type of interval: , , , , , etc
    • "either 100" or "> 200":
  • Does this cover interesting events in practical cases?
    • Yes. All sets of practical interests and much more.
  • Does it cover all subsets of ? Take some graduate courses in Real Analysis!

"Atomic" events

Sample space:
Events: all intervals in , their unions, intersections, complements
Atoms: for all

  • Can all events be generated from atoms by unions, intersections, complements?
    • Key result:
    • So, any interval , , , can be generated by the atoms
      • For example,
    • The above implies that all events can be generated by the atoms

Probability function for

Sample space:
Events: all intervals in , their unions, intersections, complements
Atoms: for all
Probability function: Assign probabilities to consistently

  • Suppose . Specify to specify .
  • For consistency, we need the following properties for
    • , , , is non-decreasing

Any valid distribution function will result in a valid probability function!

Probability space for

Sample space:
Events: all intervals in , their unions, intersections, complements
Atoms: for all
Prob. func.: , where : valid distribution function

  • A valid CDF is an important ingredient in the probability space
  • Two important special cases of CDFs
    • Discrete: CDF jumps only at a set of discrete points
      • CDF can be expressed using a Probability Mass Function (PMF)
    • Continuous: CDF has "no jumps"
      • CDF can be expressed using a Probability Density Function (PDF)

Examples of distribution functions

  • Discrete
    • Jumps at discrete set
  • Continuous
    • No jumps
    • Check rigorous definition
  • Mixture
    • Some jumps
    • Some parts continuous

Computations with distribution functions

Consider a random variable with distribution function

  • Proof:

Example

  • , etc

What about ?

If is continuous at , then .

Technical argument

  • for all

Is this intuitive? Yes, agrees with our sense of measure.

  • Length of a point
  • Length of an interval made of points

Computations with continuous distribution functions

Random variable with distribution function continuous at and

Example

  • , etc
  • for all
    • CDF does not directly show this "uniform" property

Distributions with a density

A distribution function is said to have a density function if
, .

  • : probability, : probability per unit measure
    • Numerical value of is not a probability
  • If is differentiable, we set , which denotes the derivative of
  • Example

Probability density functions (PDFs)

A function is a valid PDF if (1) for all , and (2)
( needs to be integrable)

Suppose has a density .

  • is continuous, which implies for all
    • Why? is continuous
    • Why?

Continuous distributions: Uniform, Exponential

  • Uniform, supp
    • Uniform is discrete!

  • Exponential or Exp,
    • supp

Some uniform and exponential PDFs

Continuous distributions: Gamma, Beta, Cauchy, Pareto

  • Gamma, , , supp
    • ,
  • Beta, , , supp
    • ,
  • Cauchy, supp
  • Pareto, , supp

Exercises: Find constants, make plots to see behaviour

Normal or Gaussian distribution

N, , , supp
PDF: ,

Plots of PDFs

Histograms and PDFs

How to map histograms to
PDF shapes?