OVERVIEW
UK Government estimates predict national cumulative vehicular travel to increase by up to 54% by 2060, potentially increasing journey delays by up to 85%. With many cities unable to widen intra city roads, traffic control optimisations must be investigated to reduce journey times. This project looked at the use of advanced machine learning algorithms to better control traffic light junctions for improved traffic flow.
The group created a traffic simulation that was used by the ML algorithm to train and, later on, to evaluate and compare different traffic control methods under different traffic scenarios. The simulation authentically replicated autonomous driver behaviour including acceleration, braking, and steering as well as replicating traffic patterns for a range of traffic loads across various types of junctions of ranging complexity. The traffic lights in the sumation could be controlled under different methodology altering the traffic response. A Deep Q-Learning algorithm was implemented that uses a neural network to learn from the simulation and improve traffic control. At discrete time intervals, the machine learning algorithm extracted statistical features from the simulation that was used as input to the machine learning algorithm. It would then manipulate the traffic lights in the simulation and the affect on traffic was then observed. A traffic efficiency metric was calculated which was used as feedback to improve the machine learning algorithm. |
COMPARISONS
Left: Fixed time control (simplest control method)
Center: Vehical actuation control (commonly used in the UK)
Right: Deep Q-Learning algorithm (developed for this project)
NOTE: The simulation is not running in real-time
Center: Vehical actuation control (commonly used in the UK)
Right: Deep Q-Learning algorithm (developed for this project)
NOTE: The simulation is not running in real-time
RESULTS
We compared the performance of the machine learning model with existing methods, such as vehicle actuation and fixed timings, several metrics were measured during test simulations. The machine learning model outperformed traditional methods of junction control, especially at lower traffic demands.The greatest performance increase was seen at low demand, with a 50% reduction in the mean delay at 20 vehicles per minute compared to the vehicle actuation technique. Maximum wait times were also consistently lower. At higher demands, the ML model reduced delay by up to an average of 27 seconds however it showed that, at high-demand, the optimal policy is fixed time control.
Tests on unequal spawning, during which some entrances had different spawning rates, showed poor performance, with worse delays over the entire demand range tested. These tests also recorded a significant number of crashes compared to far fewer in even spawning tests. During training the model was only exposed to even spawning behaviour, something that could be adjusted with additional development.
These mixed results show the potential of machine learning in traffic control with further optimisation.
Tests on unequal spawning, during which some entrances had different spawning rates, showed poor performance, with worse delays over the entire demand range tested. These tests also recorded a significant number of crashes compared to far fewer in even spawning tests. During training the model was only exposed to even spawning behaviour, something that could be adjusted with additional development.
These mixed results show the potential of machine learning in traffic control with further optimisation.
REPORTS
Individual Report
Group Report