Why Your DRL Agent Trades Tremendous in Backtest and Falls Aside Dwell: The Bridge Parity Drawback – Neural Networks – 24 July 2026

In case you have educated a deep reinforcement studying agent in Python and tried to run it in opposition to a reside MT5 terminal, you’ll have seen this: the agent converges cleanly, the backtest Sharpe appears good, after which on a paper account it behaves prefer it by no means educated. No error, no warning. Simply quietly dangerous choices.

In most of those circumstances the mannequin is ok. The issue is the bridge — the layer that computes the commentary vector from uncooked market information. If that computation differs even barely between coaching and manufacturing, the agent receives inputs it by no means noticed throughout coaching, and a coverage is just nearly as good because the distribution of inputs it was educated on.

The failure that produces no error message

One concrete instance. The coaching pipeline match Z-score normalization parameters over the complete historic dataset. These parameters have been by no means saved. The manufacturing bridge as a substitute computed a rolling Z-score anchored to every time the reside system began. Identical characteristic names, completely different imply, completely different normal deviation. Each normalized worth shifted exterior the coaching distribution, and the coverage produced near-random actions with full confidence.

The explanation that is so arduous to catch is that nothing throws. The tensor shapes match. OnnxRun()  returns a sound motion. The order goes out. The one symptom is efficiency that’s worse than random.

Three parity failures value checking first

• Normalization parameter drift. Freeze μ and σ throughout coaching, save them to a file, and cargo them within the EA. Recomputing them at runtime from reside information defeats the whole function of normalization consistency.
• Place state encoding mismatch. Coaching encoded state as flat/lengthy/quick = {0, 1, 2}; the MQL5 bridge used {-1, 0, 1}. The dimension matches, so the mannequin accepts it — however “lengthy” in manufacturing now has the numeric signature of “flat” in coaching. Directionally inverted choices, full confidence.
• Session flags on the incorrect clock. The Python aspect used datetime.now()  with no timezone and defaulted to OS native time, whereas bar timestamps got here again in UTC. London/NY/Asian session flags landed six hours off, so session-conditional insurance policies fired within the incorrect classes all day.

The bar-boundary lure

This one is restricted to how MQL5 indexes bars. Shut[0]  is the present, still-forming bar; it modifications on each tick. Shut[1]  is the final accomplished bar. Throughout coaching, each commentary is constructed from accomplished bars solely — the agent by no means as soon as noticed a partially-formed bar. In case your reside commentary builder reads Shut[0]  for any price-derived characteristic, you might be feeding the coverage a worth no coaching pattern ever contained.

The distribution of Shut[0]  over a bar is genuinely completely different from the distribution of a accomplished shut: increased intra-bar volatility, and in quiet regimes it sits biased towards the open. The repair is to gate the entire inference step on a brand new bar:

The identical precept applies to each indicator you feed the agent. Learn from the final accomplished bar, not the present one:

Validate the bridge earlier than any cash strikes

Not one of the checks under require reside buying and selling:

1. Historic replay parity take a look at. Take a 500-bar phase from the take a look at interval, run it by means of the manufacturing bridge, and log the complete commentary vector at each bar. Diff it element-by-element in opposition to the coaching pipeline output for a similar bars. The utmost absolute distinction must be beneath 1e-6 . Something bigger is a parity failure — discover it characteristic by characteristic earlier than continuing.
2. Random agent take a look at. Earlier than loading the educated coverage, run a coverage that picks purchase/promote/flat uniformly at random on a paper account for per week. Cumulative reward ought to sit close to zero, minus transaction prices. Whether it is badly unfavourable, the surroundings or reward has a bias the random agent is exposing — the coverage just isn’t the trigger.
3. Canary account. Solely then deploy the true coverage on 5–10% of meant capital at minimal lot measurement for 2 weeks, and evaluate Sharpe and drawdown in opposition to the paper baseline.

Hold the guardrails out of the mannequin

The coverage maps observations to actions. It has no idea of “the Python course of simply crashed” or “that is an illiquid 3 AM session.” These belong within the EA: a tough stop-loss on each place, a tough position-size ceiling that overrides the agent’s lot requests, a each day loss circuit breaker, and a heartbeat watchdog that treats a silent Python course of as unavailable and stops opening positions. The agent may be incorrect concerning the market; it mustn’t ever be capable to go away an unmanaged place open throughout a connectivity failure.

Deploying a DRL agent to MetaTrader is an engineering downside, not a machine studying downside. Get the commentary parity proper, show it in opposition to historic information, and implement danger within the execution layer independently of the mannequin. That covers nearly all of reside deployment failures.