Run a real dynamic-programming solver on the traffic MDP. Watch the Bellman residual and policy updates as training progresses.
Ready to train.
Models will be saved under a configuration-derived name and reused when the same settings are selected.
LIVE TRAINING
Convergence
● Bellman residual / value change (log scale)0 iterations
BACKEND EVENTS
Training console
[READY] Configure an algorithm and click Start training.
[INFO] All plotted points come from actual backend iterations.
HUMAN VS AI · MATCH SETUP
Configure your match
Configure the traffic environment. The intersections will appear after the countdown.
AI training environment
One traffic step = 3 seconds. Both intersections receive the same Poisson arrivals; the switching penalty affects the reward score, not car movement.
Trained models control the AI intersection using their learned policies. Q-values are computed from the saved value function and training MDP.
Automatic controller · switches every 15 s of green
📊 AI decision history & Q-value evolution
Q-values refer to the observed state at each decision step.
Time
State (NS,EW,green)
Q KEEP
Q SWITCH
Action
Explain Bellman action values
Q(s,a) = Σ P(s′|s,a) [R(s,a,s′) + γV(s′)]. These are expected discounted rewards from the trained planning MDP, not direct measurements from the animated simulator.
Select a trained model to inspect the reward and future-value components.