From Vehicle Detection to Smart Traffic Control: Perception, RL and Emergency Priority

A traffic camera that detects vehicles is useful, but it is not yet an intelligent traffic-control system. To change signal timing intelligently, the system needs a complete loop: perception → state estimation → decision → control → feedback.

This is the thread that connects my M.S. thesis with the vehicle-detection datasets I later worked on.

Layer 1 — Perception

The first layer answers visual questions: What vehicles are present? How many? Which lane? Are any of them emergency vehicles?

Object detection is a natural tool because it returns both class and location. But the class design must reflect the control objective. If an ambulance is grouped into a generic van class, an emergency-priority policy cannot use that information reliably.

That is one reason our seven-class dataset explicitly separates ambulance and fire-truck classes.

Layer 2 — State estimation

Raw detections are not the state of the traffic-control problem. A controller needs a compact representation such as:

• queue length by approach;
• vehicle density;
• waiting-time estimates;
• lane occupancy;
• emergency-vehicle presence;
• current signal phase and elapsed phase time.

This step is easy to underestimate. Detection uncertainty becomes control uncertainty. Missed vehicles, occlusion and class confusion can distort the state sent to the decision agent.

Layer 3 — Sequential decision-making

In my M.S. thesis, the intersection was modeled in SUMO and the controller was studied with Q-learning / deep-network ideas. Reinforcement learning fits the problem because one signal action changes future queues. The controller cannot optimize only the current frame; it must learn a policy over sequences of decisions.

A simplified loop is:

text Copy
Camera / detector
       ↓
Traffic-state estimator
       ↓
RL policy → signal action
       ↓
Intersection / SUMO
       ↓
Delay, queue, throughput → reward
       ↺

Emergency priority changes the objective

Adding emergency-vehicle detection is not useful only because the dataset becomes more detailed. It changes what the controller can optimize.

A normal objective might minimize average delay. With an ambulance present, the objective can become multi-criteria: reduce general congestion while imposing a much larger penalty on emergency-vehicle delay.

That immediately creates interesting research questions:

1. How should emergency priority be balanced against gridlock on conflicting approaches?
2. How robust should the policy be to a false emergency detection?
3. Should priority be rule-based, reward-based or handled by a supervisory layer?
4. How early must the perception system detect the emergency vehicle to create a useful green corridor?

A better research metric than detector mAP alone

If the detector feeds a controller, the final evaluation should not stop at image metrics. We should also measure system-level outcomes:

This is a direction I find especially interesting: evaluate perception by the decisions it enables, not only by the boxes it draws.

Where I would take the project next

A strong next experiment would compare three controllers under the same SUMO traffic scenarios:

• fixed-time control;
• RL control using ideal simulator state;
• RL control using noisy states generated from a vision model.

The gap between the second and third systems would quantify how perception quality affects control performance. We could then inject controlled detection errors and test whether uncertainty-aware state estimation or a safety supervisor makes the policy more robust.

That moves the project from “YOLO + RL” as two technologies placed next to each other toward a single perception-and-control research system.