Researchers develop a new reinforcement learning framework for precise, real-time, and sustainable automatic railway control
Urban railways play a key role in sustainable transportation systems. However, achieving precise and energy-efficient train control under diverse operating conditions of urban railways remains a challenge. Researchers from Japan have now developed a recurrent reinforcement learning framework for urban railway speed control that combines expert-guided learning and safety filtering. The proposed method demonstrated stable, real-time operation and energy-saving performance, highlighting its potential for next-generation automatic train operation systems.
As cities continue to expand,railways are expected to become an important component of urban mobility systems. Compared to automobiles and air transport, railways are highly energy efficient and produce relatively low environmental impacts and therefore form an important component of sustainable transportation systems. Thus, researchers have been exploring advanced train control technologies for further improvement of operational efficiency, passenger comfort, and punctuality while reducing energy consumption.
In this context, Professor Masafumi Miyatake from the Department of Engineering and Applied Sciences in collaboration with doctoral studentMingyu Lyu from Graduate School of Science and Technology, Sophia University,investigated a new AI-based framework for urban railway speed control. The findings were published in Volume 14 of IEEE Access on March 25, 2026.
Reinforcement learning is a branch of artificial intelligence (AI) which has attracted attention as a promising approach for automatic train operations as it enables an AI agent to learn optimal control strategies by interacting repeatedly with an environment and improving its behavior through trial and error. However, implementing this in railway operation presents several practical challenges including incomplete information about operating conditions, train inertia, braking delays, and strict safety requirements.
To overcome this, researchers developed a recurrent reinforcement learning framework based on Recurrent Soft Actor–Critic (RSAC) algorithm. RSAC is an AI method that enables train control systems to learn from past driving patterns and adapt to changing railway conditions over time.
“Our reinforcement learning-based algorithm was capable of adapting train driving control to a wide range of track and rolling stock conditions,” explains Prof. Miyatake.
Unlike conventional reinforcement learning methods,the proposed approach used arecurrent neural network that can retain information from previous train states. This allowed the system to capture time-based relationships associated with train operation, such as traction response delays, braking history, and inertial effects, thereby improving decision-making under partially observable conditions.The team also used a training approach that allowed the AI system to first learn from examples of expert driving behavior before learning on its own. By studying these optimized driving patterns early in training, the AI was able to learn faster, make better decisions, and develop more stable and efficient train control behavior.
Apart from this, the team also integrated a safety filter into the framework. The safety filter would override potentially unsafe control commands generated by the AI policy and ensure compliance with the operational constraints such as speed limits and braking feasibility. This mechanism also helped ensure safe operation when the learned policy encountered unfamiliar situations.
The proposed framework was evaluated through various simulations of urban railway operations over approximately two-kilometer sections between stations. The simulation environment was designed to closely resemble real-world railway conditions, including uphill and downhill tracks, changing speed limits, and required train arrival times.
Additionally, the proposed AI framework was also compared with several other widely used reinforcement learning algorithms. These included Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), Twin Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor–Critic (SAC), which are commonly used to train autonomous decision-making systems.
“Among all the methods tested, our developed frameworklearned faster and performed better than all tested methods,” explains Prof. Miyatake.
The research also demonstrated strong energy efficiency. While dynamic programming achieved the theoretical minimum energy consumption under ideal conditions, the developed system attained comparable performance while remaining suitable for real-time operation. Meanwhile, conventional reinforcement learning approaches exhibited less stable behavior and higher energy consumption.
The findings suggest that this framework could help introduce a cleaner and more convenient public transportation by facilitating accurate and energy-efficient train operation at relatively low cost. The study refines the advantages of railways as an energy-efficient and environmentally friendly transportation system contributing towards a sustainable future. Looking ahead, the study may contribute to enhancing the role of railways as a sustainable transportation system—supporting climate change mitigation and sustainable urban development.
The findings also highlight the potential of AI-driven railway systems to support safer, greener, and more reliable urban transportation in future smart cities.
