Reinforcement Learning-Based Adaptive Control for Complex Nonlinear Systems: Application to a CSTR
DOI:
https://doi.org/10.17979/ja-cea.2026.47.13801Keywords:
Process control, Model reference adaptive control, Machine Learning, Gain schedulingAbstract
This work proposes an adaptive control strategy for concentration regulation in a Continuous Stirred Tank Reactor (CSTR), in which a reinforcement learning agent adjusts the gains of a PID controller online in order to impose a desired closed-loop dynamic behavior. The proposed methodology combines the interpretability and robustness of classical PID-type controllers with the adaptive capabilities of reinforcement learning through a reward function defined from the error between the system response and a reference dynamic behavior. To this end, an agent based on the Deep Deterministic Policy Gradient (DDPG) algorithm is employed. Simulation results show that the proposed strategy is able to maintain a more consistent dynamic response under variations in the operating point, outperforming classical adaptive control approaches such as gain scheduling.
References
Åström, K. J., Hägglund, T., 2006. Advanced PID control. IEEE Control Systems 26 (1), 98–101.
Bequette, B. W., 1991. Nonlinear control of chemical processes: A review. Industrial & Engineering Chemistry Research 30 (7), 1391–1413.
Bertsekas, D. P., 2025. Neuro-dynamic programming. In: Encyclopedia of optimization. Springer, pp. 1–6.
Bloor, M., Ahmed, A., Kotecha, N., Mercangoz, M., Tsay, C., del Rio-Chanona, E. A., 2025a. Control-informed reinforcement learning for chemical processes. Industrial & Engineering Chemistry Research 64 (9), 4966–4978.
Bloor, M., Mowbray, M., del Rio Chanona, E. A., Tsay, C., 2025b. A survey and tutorial of reinforcement learning methods in process systems engineering. Computers & Chemical Engineering, 109515.
García, J., Fernández, F., 2015. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research 16 (1), 1437–1480.
Gil, J. D., Chanona, E. A. D. R., Guzmán, J. L., Berenguel, M., 2026. Reinforcement learning meets bioprocess control through behavior cloning: Real-world deployment in an industrial photobioreactor. Engineering Applications of Artificial Intelligence 164, 113326.
Koiš, R., Pataro, I. M. L., Gil, J. D., Záková, K., Guzmán, J. L., Berenguel, M., 2026. A web-based virtual laboratory for PID control of CSTR processes. In: Proceedings of the 23rd IFAC World Congress. Accepted for publication / In press.
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N. M. O., Erez, T., Tassa, Y., Silver, D., Wierstra, D. P., Sep. 15 2020. Continuous control with Deep reinforcement learning. US Patent 10,776,692.
Luo, B., Liu, D., Wu, H.-N., 2017. Adaptive constrained optimal control design for data-based nonlinear discrete-time systems with critic-only structure. IEEE Transactions on Neural Networks and Learning Systems 29 (6), 2099–2111.
Monteiro, M., Kontoravdi, C., 2024. Bioprocess control: A shift in methodology towards reinforcement learning. In: Computer Aided Chemical Engineering. Vol. 53. Elsevier, pp. 2851–2856.
Parambil, S. K., Vasantha, D. R., 2026. Advanced temperature control of a continuous ethanol fermentation bioreactor using deep reinforcement learning. The Canadian Journal of Chemical Engineering.
Rajasekhar, N., Radhakrishnan, T., Samsudeen, N., 2025. Exploring reinforcement learning in process control: a comprehensive survey. International Journal of Systems Science, 1–30.
Sachio, S., del Rio-Chanona, E. A., Petsagkourakis, P., 2021. Simultaneous process design and control optimization using reinforcement learning. IFAC-PapersOnLine 54 (3), 510–515.
Seborg, D. E., Edgar, T. F., Mellichamp, D. A., Doyle III, F. J., 2016. Process Dynamics and Control. John Wiley & Sons.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Antonio Martínez-Roa, Juan D. Gil, Igor M.L. Pataro, Antonio del Rio Chanona, José L. Guzmán, Manuel Berenguel
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
