Reinforcement Learning of Planning Processes for Tool-Augmented LLM Agents
PDF

Keywords

LLM planning
process-level reward
tool use
reinforcement learning
industrial agents

Abstract

 This study focuses on optimizing planning and tool-use processes of LLM agents when final answer verification is unreliable. A planner is initialized via knowledge distillation and rejection sampling, followed by reinforcement learning using a process-level reward measuring tool invocation completeness and execution validity. Training is conducted on 2,900 industrial-style agent trajectories. The approach reduces invalid action sequences by 41.6% and improves task completion reliability by 29.8%, demonstrating that process-oriented RL yields stable improvements independent of answer correctness. 

PDF

References

Pahune, S., & Akhtar, Z. (2025). Transitioning from MLOps to LLMOps: Navigating the unique challenges of large language models. Information, 16(2), 87.

Fu, Y., Gui, H., Li, W., & Wang, Z. (2020, August). Virtual Material Modeling and Vibration Reduction Design of Electron Beam Imaging System. In 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA) (pp. 1063-1070). IEEE.

González-Santamarta, M. Á., Rodríguez-Lera, F. J., & Matellán-Olivera, V. (2025). Cognitive Architectures in Autonomous Robotics: A Systematic Review of Behavior Generation Approaches and Evaluation Strategies. IEEE Access, 13, 191619-191644.

Viale, R., Gallagher, S., & Gallese, V. (2023). Bounded rationality, enactive problem solving, and the neuroscience of social interaction. Frontiers in psychology, 14, 1152866.

Yue, L., Xu, D., Qiu, D., Shi, Y., Xu, S., & Shah, M. (2026). Sequential Cooperative Multi-Agent Online Learning and Adaptive Coordination Control in Dynamic and Uncertain Environments.

Peltomaa Åström, S., & Winoy, S. (2024). Automating Software Development Processes Through Multi-Agent Systems: A Study in LLM-based Software Engineering.

Chen, F., Liang, H., Yue, L., Xu, P., & Li, S. (2025). Low-Power Acceleration Architecture Design of Domestic Smart Chips for AI Loads.

Pozzi, A., Incremona, A., Tessera, D., & Toti, D. (2025). Mitigating exposure bias in large language model distillation: an imitation learning approach. Neural Computing and Applications, 1-17.

Chen, H., Li, J., Ma, X., & Mao, Y. (2025, June). Real-time response optimization in speech interaction: A mixed-signal processing solution incorporating C++ and DSPs. In 2025 7th International Conference on Artificial Intelligence Technologies and Applications (ICAITA) (pp. 110-114). IEEE.

Dey, S., Marzullo, T., Zhang, X., & Henze, G. (2023). Reinforcement learning building control approach harnessing imitation learning. Energy and AI, 14, 100255.

Yang, M., Wu, J., Tong, L., & Shi, J. (2025). Design of Advertisement Creative Optimization and Performance Enhancement System Based on Multimodal Deep Learning.

Lööw, A., Nantes-Sobrinho, D., Ayoun, S. É., Cronjäger, C., Maksimović, P., & Gardner, P. (2024). Compositional Symbolic Execution for Correctness and Incorrectness Reasoning (Extended Version). arXiv preprint arXiv:2407.10838.

Peng, H., Dong, N., Liao, Y., Tang, Y., & Hu, X. (2024). Real-Time Turbidity Monitoring Using Machine Learning and Environmental Parameter Integration for Scalable Water Quality Management. Journal of Theory and Practice in Engineering and Technology, 1(4), 29-36.

Rahman, M., Bhuiyan, A., Islam, M. S., Laskar, M. T. R., Mahbub, R., Masry, A., ... & Hoque, E. (2025). Llm-based data science agents: A survey of capabilities, challenges, and future directions. arXiv preprint arXiv:2510.04023.

Hu, W. (2025, September). Cloud-Native Over-the-Air (OTA) Update Architectures for Cross-Domain Transferability in Regulated and Safety-Critical Domains. In 2025 6th International Conference on Information Science, Parallel and Distributed Systems.

Majid, A. Y., Saaybi, S., Francois-Lavet, V., Prasad, R. V., & Verhoeven, C. (2023). Deep reinforcement learning versus evolution strategies: A comparative survey. IEEE transactions on neural networks and learning systems, 35(9), 11939-11957.

Xu, K., Du, Y., Liu, M., Yu, Z., & Sun, X. (2025). Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features. arXiv preprint arXiv:2509.16629.

Majid, A. Y., Saaybi, S., Francois-Lavet, V., Prasad, R. V., & Verhoeven, C. (2023). Deep reinforcement learning versus evolution strategies: A comparative survey. IEEE transactions on neural networks and learning systems, 35(9), 11939-11957.

Tan, L., Liu, X., Liu, D., Liu, S., Wu, W., & Jiang, H. (2024, December). An Improved Dung Beetle Optimizer for Random Forest Optimization. In 2024 6th International Conference on Frontier Technologies of Information and Computer (ICFTIC) (pp. 1192-1196). IEEE.

Setlur, A., Nagpal, C., Fisch, A., Geng, X., Eisenstein, J., Agarwal, R., ... & Kumar, A. (2024). Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146.

Gao, X., Chen, J., Huang, M., & Fang, S. (2025). Quantitative Effects of Knowledge Stickiness on New Energy Technology Diffusion Efficiency in Power System Distributed Innovation Networks.

Mao, Y., Ma, X., & Li, J. (2025). Research on API Security Gateway and Data Access Control Model for Multi-Tenant Full-Stack Systems.

Salvi, R. M., & Barman, P. K. (2025). Evolving Architectures and Long-Horizon Planning in Multi-Agent Conversational Ai: A Decade in Review. Emerging Frontiers Library for The American Journal of Interdisciplinary Innovations and Research, 7(07), 106-122.

Liu, S., Feng, H., & Liu, X. (2025). A Study on the Mechanism of Generative Design Tools' Impact on Visual Language Reconstruction: An Interactive Analysis of Semantic Mapping and User Cognition. Authorea Preprints.

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright (c) 2026 Lukas Müller, Hannah Wagner, Maximilian Weber (Author)

Downloads

Download data is not yet available.