Statistical Inference
Statistical inference draws conclusions about populations from data. Estimation, confidence intervals, and hypothesis testing provide the framework for data-driven decisions.
Key Points
- Maximum likelihood estimation finds parameters that maximize the likelihood of observed data.
- The bias-variance decomposition explains model error.
- Bayesian inference updates beliefs via the posterior distribution.
Formulas
Maximum likelihood
$$\hat{\theta}_{MLE} = \arg\max_\theta \prod_{i=1}^n p(x_i | \theta)$$
Bayes' theorem for parameters
$$p(\theta | \mathcal{D}) = \frac{p(\mathcal{D}|\theta)p(\theta)}{p(\mathcal{D})}$$
Bias-variance decomposition
$$\mathbb{E}[(y - \hat{f}(x))^2] = \text{Bias}^2 + \text{Variance} + \sigma^2$$
Code Example
from scipy import stats
data = stats.norm.rvs(loc=5, scale=2, size=100)
mu_mle, sigma_mle = stats.norm.fit(data)
print(mu_mle, sigma_mle)