SQUARE45

Theorem

Central limit theorem

The central limit theorem (CLT) explains the ubiquitous occurrence of the normal distribution in nature, and this theorem, according to David Williams, "is one of the great results of mathematics." The theorem states that the average of many independent and identically distributed random variables with finite variance tends towards a normal distribution irrespective of the distribution followed by the original random variables. Formally, let

X_{1},X_{2},\dots \,

be independent random variables with mean

\mu

and variance

\sigma ^{2}>0.\,

Then the sequence of random variables

Z_{n}={\frac {\sum _{i=1}^{n}(X_{i}-\mu )}{\sigma {\sqrt {n}}}}\,

converges in distribution to a standard normal random variable. For some classes of random variables, the classic central limit theorem works rather fast, as illustrated in the Berry–Esseen theorem. For example, the distributions with finite first, second, and third moment from the exponential family; on the other hand, for some random variables of the heavy tail and fat tail variety, it works very slowly or may not work at all: in such cases one may use the Generalized Central Limit Theorem (GCLT). - ^David Williams, "Probability with martingales", Cambridge 1991/2008

Definition

Probability Theory (Definition)

A probability theory is formally defined by a probability space

(\Omega, \mathcal{F}, P)

, where: \n\n1.

\Omega

is the sample space, representing the set of all possible outcomes. \n2.

\mathcal{F}

is a

\sigma

-algebra on

\Omega

, meaning

\mathcal{F}

is a subset of

\mathcal{P}(\Omega)

(the power set of

\Omega

) such that: \n a)

\Omega \in \mathcal{F}

(The sample space is an event). \n b) If

A \in \mathcal{F}

, then its complement

A^c = \Omega \setminus A

is also in

\mathcal{F}

(Closure under complementation). \n c) If

(A_i)_{i=1}^{\infty}

is a countable sequence of sets in

\mathcal{F}

, then their union

\cup_{i=1}^{\infty} A_i

is also in

\mathcal{F}

(Closure under countable unions). \n3.

P

is the probability measure, a function

P: \mathcal{F} \to [0, 1]

, satisfying the Kolmogorov axioms:\n a) Non-negativity:

P(A) \ge 0

for all

A \in \mathcal{F}

. \n b) Normalization:

P(\Omega) = 1

. \n c) Countable Additivity: For any countable sequence of pairwise disjoint events

(A_i)_{i=1}^{\infty}

in

\mathcal{F}

, the probability of their union is the sum of their individual probabilities:\n

P(\cup_{i=1}^{\infty} A_i) = \sum_{i=1}^{\infty} P(A_i)

Theorem

Central Limit Theorem

Intermediate

The distribution of the sum (or average) of a large number of independent, identically distributed variables approaches a normal distribution.\n

\frac{S_n - n\mu}{\sigma\sqrt{n}} \xrightarrow{d} N(0,1)

Theorem

Law of Large Numbers

Beginner

The average of the results obtained from a large number of trials should be close to the expected value.\n

\bar{X}_n \to \mu

Sequence of Expressions