Essentials of Data Science – Probability and Statistical Inference – Expectation of Variables

By / March 22, 2022

In the previous note on discrete random variables, we have seen what a discrete random variable is and its distribution function. In the case of a discrete random variable, the distribution function is expressed in terms of probability mass function (PDF) and cumulative distribution function (CDF). 

In this note, we will further extend the concept of the random variables and explore what an expectation is and how to compute for discrete and continuous random variables.

Table of Contents hide
1 Introduction
2 Expectation of a Continuous Random Variable
3 Expectation of a Discrete Random Variable
4 Special Cases of Expectation of a Random Variable
5 Summary
6 References
6.1 Share this:
6.2 Like this:

Introduction

Let us understand what an expectation is. Suppose two participants are playing a coin-tossing game, assume that coin is fair and chances of getting head and tail are equally likely, as well as same rewards for winner or loser.

  • Case 1: The outcome will be equally likely to gain or lose in the game if we do the mental calculation.
  • Case 2: If we change the reward proportion, let’s say, X amount needs to be paid by the first participant when they lose and receive 2X amount when they win. It is a special offer for the first participant. Although the first participant knows that winning or losing chances are equally likely, the expected chances of success are higher due to changes in reward proportion. Moreover, indirectly the first participant does mental calculations and figures out that the average or expected success rate is higher than failure in the game.

This is how we always compute the average or expected value for grabbing or passing the opportunities in every perspective of our life, which means we know how to calculate the expected value from the observations.

Now, we will try to understand what does expectation mean using the same example, but in more mathematical way.

Example 1:

Suppose we toss a coin, assume that coin is fair and chances of getting head and tail are equally likely. The possible outcomes are head and tail, {H,T}. The reward formulation are as follows:

  • If we get head, we get a reward of Rs. 2.
  • If we get tail, we get a reward of Rs. 4.

What do we expect to get an average reward ?

  • P(H) = P(T) = 1/2
  • Expected average reward = Rs. 2 * P(H) + Rs. 4 * P(T) = 2 * 1/2 + 4 * 1/2 = Rs. 3

Example 2:

Suppose we roll a dice and rewards are considered in the following ways:

\begin{array}{|c|c|c|c|c|c|c|} \hline \text{Point (x)} & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline \text{Reward (INR)} & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline \text{P(X=x)} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} \\ \hline \end{array}

In this example, Point(x) represents a number that appears on the upper face of the die, and Reward (INR) represents rewards for different values of x. However, each outcome’s probabilities are equally likely, which means anyone occurs with the same probability.

The expected reward means when we play this game a significantly higher number of times, the accumulated rewards would be equal to the expected reward value.

Expected reward: Rs. 1 * 1/6 + Rs. 2 * 1/6 + Rs. 3 * 1/6 + Rs. 4 * 1/6 + Rs. 5 * 1/6 + Rs. 6 * 1/6 = Rs. 3.50

Expectation of a Continuous Random Variable

Let X be a continuous random variable having the probability density function f(x). Suppose g(X) is a real valued function of X. The expectation of g(X) is defined as:

E[g(X)] = \int_{-\infty}^{\infty} g(x)f(x) dx provided with \int |g(x)| f(x) dx < \infty

Example 1: Consider the continuous random variable “waiting time for the train”. Suppose that a train arrives every 20 min. Therefore, the waiting time of a particular person is random and can be any time contained in the interval [0,20].

The required probability density function is

f(x) = \begin{cases} 1/20 & \text{for } 0 \leq x \leq 20 \\ 0 & \text{otherwise} \end{cases}

Solution:

\begin{aligned}E(X) &= \int_{-\infty}^{\infty} xf(x) dx \\ &= \int_{-\infty}^{0} xf(x)dx + \int_{0}^{20} xf(x)dx + \int_{20}^{\infty} xf(x)dx \\ &= 0 + \int_{0}^{20} + x \text{ 1/20 } dx + 0 \\ &= 10 \end{aligned}

Thus the average waiting time for the train is 10 minutes, which means that if a person has to wait for the train every day, then the waiting time will vary randomly between 0 and 20 minutes and, avarage it will be 10 minutes.

Expectation of a Discrete Random Variable

Let X be a discrete random variable having the probability mass function P(X=x_i) = p_i. Suppose g(X) is a real valued function of X. Thus, X takes the values x_1, x_2, x_3, \cdots x_k \cdots with respective probabilities p_1, p_2, p_3, \cdots p_k \cdots and if the expectation of g(X) exists then,

E[g(X)] = \sum_{i=1}^{\infty} g(x_i)P(X=x_i) = \sum_{i=1}^{\infty} g(x_i)p_i provided g(X) = \sum_{i=1}^{\infty} |g(x_i)|p_i < \infty

Example 1: Suppose we roll a dice and following is the scheme for award based on outcomes.

\begin{array}{|c|c|c|c|c|c|c|} \hline \text{Point (x)} & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline \text{Reward (INR)} & 1 & 2 & 3 & 4 & 5 & 6 \\ \hline \text{P(X=x)} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} & \text{1/6} \\ \hline \end{array}

The required probability mass function is:

p(X) = \begin{cases} 1/6 & \text{for } 1 \leq x \leq 6 \\ 0 & \text{otherwise} \end{cases}

Solution: 

\begin{aligned} E[X] &= \sum_{i=1}^{\infty} g(x_i) \times P(X=x_i) \\ &= \sum_{i=1}^{6} g(x_i)p_i \\ &= 1 \times 1/6 + 2 \times 1/6 + 3 \times 1/6 + 4 \times 1/6 + 5 \times 1/6 + 6 \times 1/6 \\ & = Rs. 3.5 \end{aligned}

Special Cases of Expectation of a Random Variable

Case1: g(X) = X then E[g(X)] = E(X). The expectation of X, i.e. E(X), is ususally denoted by \mu = E(X) and relates to the arithmetic mean of the distribution of the population. It reflects the central tendency of the population.

Case2: If a and b are any real constants, then E(a) = a and E[aX + b] = aE(X) + b

Case3: Let g_1, g_2, g_3, \cdots g_r be r real valued functions such that E[g_i(X)] exists for all i = 1,2,3, \cdots ,r then E[\sum_{1=1}^r g_i(X)] = \sum_{i=1}^r E[g(X_i)]

Summary

Probability Space

We have seen what an experiment is and the possible outcomes of the experiments. Based on the experiment’s outcomes, we can decide whether random variables would be discrete and continuous. The definition of the random variable is entirely dependent on the experiment goal. However, it should satisfy the random variable criteria.

Once we have a random variable and associated distribution function, we can calculate probabilities at a point and probability area under the curve, in the case of a discrete random variable and continuous random variable, respectively. These values can be stored in a tabular fashion or we can say that values of the probability function of a random variable.

Statistical Space

From the distribution data, we can learn various parameters that may help to understand the characteristics of an experiment based on our interest which is defined using a random variable. The characteristics could be the central tendency, data dispersion, skewness, kurtosis, correlation coefficient, etc. We always represent data based on these characteristics because it is impossible to talk about individual data points, which may not give relevant information to make statistical decisions.

It shows how we have shifted our random experiment model from probability to statistical space to derive meaning information from the results.

References

  1. Essentials of Data Science With R Software – 1: Probability and Statistical Inference, By Prof. Shalabh, Dept. of Mathematics and Statistics, IIT Kanpur.

CITE THIS AS:

“Expectation of Variables”  From NotePub.io – Publish & Share Note! https://notepub.io/notes/mathematics/statistics/statistical-inference-for-data-science/expectation-of-variables/

 65,443 total views,  1 views today

Scroll to Top
Scroll to Top
%d bloggers like this: