Reassessing Reinforcement Learning for Optimal Trade Execution

Abstract

Recent research has explored reinforcement learning (RL) for optimal trade execution because learning-based policies can adapt execution decisions throughout the liquidation horizon rather than following a predetermined schedule. However, implementation shortfall depends not only on execution decisions but also on realized intraday price paths, stochastic training effects, and the composition of the out-of-sample evaluation set. This study reassesses the performance of RL-based execution policies against Time-Weighted Average Price (TWAP) using a microstructure-aware simulation framework and a statistically controlled evaluation protocol. All strategies are evaluated on identical fixed out-of-sample intraday price paths, enabling paired comparisons that separate execution effects from market drift. Transaction costs are modeled explicitly, and robustness is assessed across multiple training seeds. Unconstrained RL policies frequently generate unstable liquidation schedules and fail to provide reliable improvements over TWAP. To address this instability, this study introduces a residual formulation in which the learning agent makes bounded adjustments around the TWAP schedule. The residual constraint eliminates extreme execution patterns and improves policy stability and interpretability. Across up to 1,000 test episodes, residual RL remains close to TWAP performance without achieving statistically reliable improvements. A drift–friction decomposition further shows that realized price movements dominate outcome variability, while frictional costs remain broadly similar across strategies.



Received: 24 December 2025 | Revised: 14 April 2026 | Accepted: 16 August 2026



Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.



Data Availability Statement

The data that support the findings of this study are openly available in GitHub at https://github.com/EBarbierato/Reassessing-Reinforcement-Learning-for-Optimal-Trade-Execution.



Author Contribution Statement

Matteo Tosi: Conceptualization, Methodology, Software, Investigation, Resources, Data curation, Visualization. Enrico Barbierato: Conceptualization, Software, Formal analysis, Writing – original draft, Writing – review & editing, Supervision, Project administration, Funding acquisition. Alice Gatti: Validation, Writing – review & editing.