Interactive Probability & Statistics

Distribution (Thin-Tail)

The Exponential distribution models the time between events in a Poisson point process. It's "thin-tailed," meaning extreme events are rare. Its probability density function (PDF) is:

$f(x; \lambda) = \lambda e^{-\lambda x}$ for $x \ge 0$

Adjust the rate parameter ($\lambda$) and generate samples to see how the empirical data aligns with the theoretical PDF and CDF.

Distribution (Fat-Tail)

The Pareto distribution models phenomena where a small number of items have a large effect (the "80/20 rule"). It's "fat-tailed," meaning extreme events are more likely. For a minimum value ($x_m$) of 1, its PDF is:

$f(x; \alpha) = \frac{\alpha}{x^{\alpha+1}}$ for $x \ge 1$

Adjust the shape ($\alpha$). As $\alpha$ decreases, the tail gets "fatter," and the probability of very large values increases. Note that the variance is infinite if $\alpha \le 2$, and the mean is infinite if $\alpha \le 1$.

Conditional Probability (Thin-Tail)

Imagine a world where income is distributed exponentially (thin-tailed). This implies a strong tendency towards an average income. The plot shows the probability of doubling your income, given you've already earned an amount $a$. Formally, this is $P(X > 2a \mid X > a)$.

For the Exponential distribution, this probability, $e^{-\lambda a}$, decreases as your income $a$ grows. This means the richer you are, the less likely you are to double your wealth. This "wear-out" effect leads to less extreme income inequality, as high incomes are statistically likely to regress towards the mean rather than grow indefinitely.

Conditional Probability (Fat-Tail)

Now, imagine a world where income follows a Pareto distribution (fat-tailed), which is a much better model for real-world wealth. The plot shows the probability of doubling your income, $P(X > 2a \mid X > a)$, given you've already earned an amount $a$.

For the Pareto distribution, this probability is $(1/2)^\alpha$, a constant that does not depend on your current income $a$. This remarkable result is called scale-invariance. It implies that the probability of a millionaire doubling their wealth is the same as the probability of a billionaire doubling theirs. This "the rich get richer" dynamic is why fat-tailed distributions are essential for understanding phenomena with extreme inequality.

Sample Mean of Thin-Tailed Distribution

The Central Limit Theorem (CLT) states that the distribution of sample means will be approximately normal, even if the underlying distribution is not. For a thin-tailed distribution like the Exponential, this convergence happens relatively quickly. Adjust the parameters below to see how the distribution of sample means approaches a bell curve.

Sample Mean of Fat-Tailed Distribution

While the CLT still applies to fat-tailed distributions (with finite variance), the distribution of sample means converges to normal much more slowly. Because of the higher probability of extreme events in the Pareto distribution, the sample means will have a much wider spread (higher variance) compared to the thin-tailed case. Adjust $\alpha$ and see how it affects the result.

Correlation with Thin-Tailed Data

Here we generate data from an Exponential distribution. With thin-tailed data, correlation is stable. We construct Z = 2X + Y, where X and Y are independent. Notice Corr(X, Y) is near 0, and Corr(X, Z) is high and positive. The histogram of sample correlations is narrow, showing stability.

Correlation with Fat-Tailed Data

Here we generate data from a Pareto distribution. Outliers in fat-tailed data can heavily influence the correlation coefficient. The histogram of sample correlations is much wider than in the thin-tailed case, showing this volatility.

Regression with Thin-Tailed Data

With thin-tailed data, the regression model is stable. The R-squared histogram is narrow, showing that different samples produce very similar model fits.

Regression with Fat-Tailed Data

Outliers in fat-tailed data can dramatically affect the regression line. The R-squared histogram is much wider, showing that the model's explanatory power is highly volatile and sensitive to the specific sample drawn.