Stochastic Processes
Stochastic processes model systems evolving randomly over time or space. They are essential for finance, physics, operations research, and reinforcement learning.
Key Points
- A stochastic process is a collection of random variables indexed by time or space.
- Markov processes have memoryless future given the present.
- Martingales model fair games and are central to stochastic calculus.
Formulas
Markov property
$$P(X_{t+1} | X_t, X_{t-1}, \dots) = P(X_{t+1} | X_t)$$
Martingale
$$\mathbb{E}[X_{t+1} | \mathcal{F}_t] = X_t$$
AR(1) process
$$X_t = \phi X_{t-1} + \varepsilon_t$$
Code Example
import numpy as np
T = 1000
X = np.zeros(T)
for t in range(1, T):
X[t] = 0.9 * X[t-1] + np.random.randn()
print(X.mean(), X.var()) # var ~ 1/(1-0.9^2) = 5.26