Thanks to convolution, we can obtain the probability distribution of a sum of independent random variables.

### The Convolution Series

- Definition of convolution and intuition behind it
- Mathematical properties of convolution
- Convolution property of Fourier, Laplace, and z-transforms
- Identity element of the convolution
- Star notation of the convolution
- Circular vs. linear convolution
- Fast convolution
- Convolution vs. correlation
- Convolution in MATLAB, NumPy, and SciPy
- Deconvolution: Inverse convolution
**Convolution in probability: Sum of independent random variables**

So far, we have looked into various aspects of convolution. One of its important applications is in probability: thanks to the convolution, we can obtain the *probability density function* (pdf) of a sum of two independent random variables (RVs). It turns out that the pdf of that sum is a convolution of pdfs of the two random variables.

In this article, we will show the proof of this theorem. This proof takes advantage of the convolution property of the Fourier transform.

## Convolution Theorem in Probability

The probability density function of a sum of statistically independent random variables is the convolution of the contributing probability density functions.

## Proof

Before we conduct the actual proof, we need to introduce the concept of the *characteristic function*.

### The Characteristic Function

The characteristic function

Let us observe that

Another building block of the proof is the independence assumption which we examine next.

### Independence of Random Variables

Two random variables are called **statistically independent** if their joint probability density function factorizes into the respective pdfs of the RVs.

If we have two RVs,

where

### Sum of Two Independent Random Variables

Now to the main part of the proof!

We have two independent random variables,

Note that we could separate the integrals only thanks to the independence of the two random variables: splitting

### Convolution Property of the Fourier Transform

We found out that the characteristic function of a sum of two independent random variables is equal to the product of the individual characteristic functions of these random variables (Equation 4). Additionally, the characteristic function of a random variable with a negated argument is the Fourier transform of this RV’s probability density function (Equation 3). We thus have

and

The convolution property of the Fourier transform tells us that the multiplication in the Fourier domain is equivalent to convolution in the other domain (here: the domain of the random variable). Therefore,

what concludes the proof

*Note: $x$ is used instead of $y$ as the argument of $f_Y$ in Equation 7 because it doesn’t matter what letter we use; $f_X$, $f_Y$, and $f_{X+Y}$ are all pdfs of one-dimensional random variables.*

### Final Remark

This proof can be extended to arbitrarily many random variables with the requirement that all of them are mutually independent.

## Summary

In this article, we have proven that the probability distribution of a sum of independent random variables is a convolution of probability distributions of these random variables.

## Bibliography

[1] Walter Kellermann, *Statistical Signal Processing Lecture Notes*, Winter Semester 2019/2020, University of Erlangen-Nürnberg.

Comments powered by Talkyard.