Adam Optimizer — 3D View
m
t
= β₁·m
t-1
+ (1−β₁)·g
t
v
t
= β₂·v
t-1
+ (1−β₂)·g
t
²
θ
t
= θ
t-1
− α · m̂
t
/ (√v̂
t
+ ε)
α (learning rate)
0.100
β₁ (momentum decay)
0.900
β₂ (variance decay)
0.999
Reset
Pause
Step
t
0
θ₁
3.000
θ₂
3.000
m₁
0.000
m₂
0.000
v₁
0.000
v₂
0.000
loss
22.500
Raw gradient gₜ
1st moment mₜ (momentum)
Adam effective step Δθ
Adaptive metric (1/√v̂ₜ)
Drag to rotate · Scroll to zoom · Right-drag to pan · Based on Kingma & Ba (2015) Adam