The LR Engine
Every stage of the pipeline runs the same simulator. Understanding it once explains the search space, the metrics, the generated MQL5, and why the holdout matters — so it is worth reading carefully.
The one-sentence version
A binary logistic regression on a small vector of price-derived features, re-trained on a rolling window that ends strictly before the current bar, trading one position at a time with ATR-based stop-loss and take-profit levels, entered at the next bar's open.
The whole design exists to answer one question honestly:
The label
Training targets are binary, and depend only on the close price lookahead bars ahead:
label(k) = 1.0 if close[k + lookahead] > close[k] else 0.0
So the model is not predicting magnitude, or return, or direction over a fixed
horizon in a regression sense. It is predicting whether the close will be
higher a few bars later. With the stage-1 default
lookahead = 3, that is a three-bar-ahead up/down call.
That is also all the model learns. It never sees P&L, never sees the stop-loss, and is never rewarded for a trade that happened to work out. It optimises classification accuracy on the log-loss of a static binary target.
Why there is no lookahead
This is the part worth understanding properly, because it is a handful of index expressions that carry the entire credibility of the results.
Batch training stops before the current bar
k_start = current - lookahead - training_bars + 1
k_end = current - lookahead + 1 # exclusive
for k in range(max(1, k_start), k_end):
...
labels.append(1.0 if close[k + lookahead] > close[k] else 0.0)
For every training sample k, the largest index the label touches is
k + lookahead. Because the loop stops at
k_end - 1 = current - lookahead, the largest label index read is
current. The label uses close[k + lookahead] where the
bar at k + lookahead has already closed by the time we are at
current.
Training also explicitly skips samples whose label would reach past the end of
the data: if k + lookahead >= n: continue.
Online learning uses a stale feature vector
def online_learn(i):
if i < p["lookahead"]:
return
stale = i - p["lookahead"] # the bar whose outcome is now known
x = valid_feats(stale)
pr = predict(x)
actual = 1.0 if close[i] > close[stale] else 0.0
error = pr - actual
# one SGD step on weights / bias
Each bar contributes exactly one online gradient step, and it is a step on a
fully resolved prediction: the features of bar
i - lookahead, compared against the close of bar i.
By construction the outcome is known. The model never takes a gradient step
based on a prediction whose truth is still in the future.
current. That consistency
is what makes "no lookahead" a property of the system rather than a claim about
one function.
The model
Standard logistic regression with an L2 penalty:
z = X·w + b, then the sigmoidlam = 0.001 — mirrored in the EA as #define LAM 0.001max_iter — 600 during search, 2,000 at final verification(x − mean) / std, with std floored to avoid division by zero
Before the first successful train, predict() returns exactly
0.5 — deliberately in the dead zone between the entry thresholds,
so an untrained model cannot accidentally trade.
Three separate update paths
| Path | When | Effect |
|---|---|---|
| Initial train | Once, at warmup = training_bars + 1 |
Full batch gradient descent over the first window. Before this, nothing trades. |
| Periodic re-train | Every retrain_interval bars |
Discards the weights and re-fits from zero on the shifted window. A hard reset rather than a fine-tune. |
| Online step | Every single bar | One SGD step on momentum, using the now-resolved prediction from lookahead bars ago. |
The online step uses momentum with a decay factor, so repeated corrections in the same direction accumulate — a small amount of adaptive responsiveness between the hard re-trains.
weights = 0 and re-derives them by
gradient descent. This is not warm-started from the previous fit. It is why
retrain_interval is a genuinely meaningful hyper-parameter rather
than an incremental one.
The search space
Three groups of parameters get grid-searched, in two stages.
Stage 1 — trade geometry (64 combinations)
| Parameter | Values | Meaning |
|---|---|---|
sl_mult | 1.5, 2.0, 2.5, 3.0 | Stop-loss distance as a multiple of ATR(14). |
rr | 1.0, 1.5, 2.0, 3.0 | Take-profit as a multiple of the stop distance. |
lookahead | 2, 3, 5, 8 | Label horizon in bars, and the online-learning lag. |
4 × 4 × 4 = 64 combinations, each run at max_iter = 600.
Stage 2 — training behaviour (243 per stage-1 winner)
| Parameter | Values | Meaning |
|---|---|---|
training_bars | 500, 1000, 1500 | Length of the rolling training window. Also sets warmup. |
retrain_interval | 250, 500, 1000 | How often weights are re-fitted from scratch. |
batch_lr | 0.005, 0.01, 0.02 | Gradient-descent learning rate for batch training. |
long_thresh | 0.50, 0.55, 0.60 | Probability above which a long is taken. |
3 × 3 × 3 × 3 = 81 combinations, run for each of the top 3 diverse stage-1 survivors — 243 runs.
The three stage-1 survivors are deliberately kept diverse: the list is
de-duplicated on the whole (sl_mult, rr, lookahead) tuple, so the
three are not three near-identical variants of one good point.
Final verification
The stage 1 and stage 2 results are merged, filtered to those that pass the
acceptance test, sorted by score, and the top 40 are re-run at
the full budget max_iter = 2000. This matters because the search
ran at 600 iterations: a point that looked good under-trained may not survive
proper convergence, and vice versa.
The acceptance test and the score
score = net_profit / (1 + abs(max_dd_pct) / 100)
ok = trades >= 30 and profit_factor > 1.0
A configuration that fails ok is assigned a score of
-1e18, which effectively removes it from the ranking. So the search
cannot select a high-net configuration that achieved it in twelve trades or with
a profit factor below 1.
1 + |DD%|/100 penalises drawdown but not linearly —
a −50% max drawdown divides the score by 1.5, not by 2. It is a ranking device,
not a risk model.
Slot selection
After verification, survivors are sorted by score and fed to a greedy selector that picks up to 5 slots subject to a correlation constraint:
N_SLOTS = 5MAX_CORR = 0.85 — a candidate is rejected if correlation with an already-picked slot exceeds this1e-12, or the pair is not comparedThe correlation is computed on the first difference of equity, not the level. Levels are dominated by the shared price trend of the instrument and would make almost everything look correlated; differencing isolates the per-bar behaviour, which is the thing being diversified.
Trade execution
One position at a time per slot. The sequence for a given signal bar i:
i's server-time hour is not in the allowed set, return immediately. This gates entries only — training is unaffected.max_spread, skip. Default is use_spread_filter = False, and the search derives max_spread from the median spread of the data file.long_thresh → long. Below short_thresh → short. In between → no trade.entry = open[i + 1]. Not the current close — the decision is made from closed-bar data, and the fill happens at the next opportunity.sl_pts = sl_mult × ATR14[i], then tp_pts = sl_pts × rr. Applied symmetrically around entry according to side.How positions close
On every subsequent bar the position is checked against its levels. The order is significant:
# long
if low[i] <= sl: closed = ("SL", sl)
elif high[i] >= tp: closed = ("TP", tp)
# short
if high[i] >= sl: closed = ("SL", sl)
elif low[i] <= tp: closed = ("TP", tp)
If neither level is touched, the position carries over. Any position still open
at the end of the data is closed at the final close with the reason
"EOData".
Breakeven and trailing stops exist in the engine but are off by default, matching the generated EAs. The pipeline describes them as "NOBE" builds.
Money and costs
P&L is computed in dollars by a single expression:
pnl = (exit_price - entry_price) × fixed_lot × lot_mult × side
side is +1 for long, −1 for short. fixed_lot is 0.01 —
one micro-lot — everywhere in the pipeline. lot_mult converts a price
move into money for that instrument:
| Symbol | lot_mult | Meaning at 0.01 lot |
|---|---|---|
| XAUUSD | 100 | $1.00 per $1.00 price move |
| EURUSD | 100000 | $0.10 per pip |
| DowJones30 | 1 | $0.01 per index point |
The EURUSD value is not merely a table entry — it is asserted. On startup the
engine derives dollars-per-pip for EURUSD at 0.01 lot and raises a
SystemExit if it deviates from $0.10 by more than 5%. A units
mistake would otherwise inflate or deflate every figure silently.
Spread is not charged on fills
median_spread × number_of_trades,
always showing raw and net-of-spread figures side by side.
The convention is
POINT_DOLLARS = 0.01 — dollars per point per 0.01
lot. This is why a zero-filled or missing spread column is treated as a fatal
data error: it would make every trade cost $0 and every result look better than
reality.
Magic numbers
Every slot has a unique magic number, built from the feature set's base plus a timeframe offset:
magic = LR_MAGIC_BASE[set] + magic_offset + slot_index
LR_MAGIC_BASE[set] = FEATURE_SETS[set]["magic_start"] + 100
With magic_start = 301 for base9, the base is 401 and its five slots are 401–405. The timeframe offsets let several timeframes run on one account simultaneously without collision:
| TF | Offset | TF | Offset |
|---|---|---|---|
| M1 | 4000 | M30 | 5000 |
| M2 | 2000 | H1 | 6000 |
| M5 | 0 | H4 | 7000 |
| M10 | 3000 | D1 | 8000 |
| M15 | 1000 |
So base9 on M5 occupies 401–405, while the same set on M15 occupies
1401–1405. The EA scans for its own positions matching on both magic
and symbol.
The features
Features are computed from a small derived-primitive library. Each set selects a
subset. All divisions are guarded against 1e-10, and the whole set
is standardised before use.
| Primitive | Definition |
|---|---|
logret{n} | log(close[i] / close[i + n]) — log return over n bars |
atr14, atr10, atr20, atr50 | SMA of true range over that period |
atr_ratio | atr14 / atr50 — short vs long volatility |
vol10, vol_ratio20 | atr14 / atr10, atr14 / atr20 |
atr_sma50 | (atr14 − atr50) / atr50 |
rng, rng_atr | high − low, and that divided by atr14 |
body_ratio / doji | |close − open| / (high − low) |
upper_wick, lower_wick | Wick extents relative to the bar range |
close_pos | (close − low) / (high − low) — where in the range it closed |
ma_cross | (sma5 − sma20) / atr14 |
ma_cross9 | (sma5 − sma9) / atr14 |
sma_dist20, sma_dist9, sma_dist50 | (close − smaN) / atr14 |
rsi14 | Relative strength index |
mom10 | close − close[i + 10] |
ma_align | Count 0–3 of satisfied orderings among sma5 > sma9, sma9 > sma20, sma20 > sma50 |
ma_align_dist | (sma5 − sma50) / atr14 |
hh50, ll50 | Distance to the 50-bar high / low, in ATR units |
rng50_atr | 50-bar high-low range divided by atr14 |
open, co | Raw open, and close − open |
hour, dow | Server-time hour of day and day of week |
dow follows pandas' Monday=0..Sunday=6 convention. The MQL5 mirror
converts from MetaTrader's Sunday=0 with (DayOfWeekOfBar(t[i]) + 6) % 7.
The eleven feature sets
| Set | Abbr | magic_start | LR base | Features |
|---|---|---|---|---|
base9 | BAS | 301 | 401 | logret5, logret10, atr14, hour, dow, body_ratio, rng, open, co |
momentum | MOM | 311 | 411 | logret3, logret5, logret10, logret20, hour, dow, body_ratio |
volatility | VOL | 321 | 421 | atr14, atr_ratio, rng_atr, body_ratio, hour, dow, logret5 |
price_action | PA | 331 | 431 | body_ratio, upper_wick, lower_wick, close_pos, rng_atr, hour, dow, logret5 |
trend | TRD | 341 | 441 | logret5, logret10, ma_cross, sma_dist20, atr14, rng_atr, hour, dow |
full15 | FUL | 351 | 451 | 15 features — the union of the momentum, volatility and price-action ideas |
micro | MIC | 361 | 461 | logret1, logret3, logret5, body_ratio, rng_atr, hour, dow |
voltrend | VT | 371 | 471 | atr_ratio, vol10, rng_atr, atr_sma50, vol_ratio20, logret5, logret20, hour |
meanrev | MR | 381 | 481 | sma_dist20, sma_dist50, ma_cross9, sma_dist9, rsi14, close_pos, mom10, hour, dow |
mafan | MAF | 391 | 491 | ma_align, ma_align_dist, ma_cross, ma_cross9, sma_dist20, sma_dist50, logret5, logret20, atr14, rng_atr, hour, dow |
breakout | BRK | 401 | 501 | hh50, ll50, rng50_atr, rng_atr, atr_ratio, logret5, close_pos, body_ratio, hour |
The ALL9 portfolio uses the first six sets plus
micro, voltrend and meanrev — nine slots
from nine distinct feature definitions. ALL6 uses only the first
six.
Metrics
All metrics are computed from the equity curve and the closed-trade list:
| Metric | Definition |
|---|---|
net_profit (NP) |
equity[-1] − equity[0] — dollars, at 0.01 lot per slot |
max_dd_pct (DD) |
The minimum of (equity − running_peak) / peak, in percent. A negative number. |
profit_factor (PF) |
gross_profit / gross_loss, with the denominator floored at 1e-10 |
win_rate (WR) |
wins / trades × 100 |
trades (T) |
Count of closed trades, with buys / sells split |
sharpe (S) |
Annualised from per-bar equity returns |
score |
net_profit / (1 + |max_dd_pct| / 100) — the search ranking device |
The Sharpe calculation, precisely
rets = diff(equity) / equity[:-1]
sharpe = mean(rets) / max(std(rets), 1e-10)
× sqrt(365 × 24 × 60 / n_bars)
The annualisation factor assumes the bars are minute bars spanning a 24/7
calendar: 365 × 24 × 60 minutes in a year, divided by the number of
bars. For an M5 series that over-states the number of bars per year, since the
n used is the actual bar count rather than a trading calendar. Treat the
Sharpe figure as comparable between slots in the same run rather than as
an absolute annualised Sharpe ratio.
initial_balance = 10000.0, and the equity array's first entry is
that value before any trade. All drawdown figures are therefore relative to a
$10,000 notional at 0.01 lot per slot — a very small position size, which is why
the dollar figures look modest next to the percentages.
Three modes the engine supports for the holdout
The engine has two hook parameters used only by the tail-holdout stage. They are implemented as plain flags rather than as a separate code path, which is why the holdout is trustworthy: it is the same simulator, restricted.
| Hook | Effect |
|---|---|
_trade_from |
No new entries before this bar. The model still learns and re-trains normally. This is "warm". |
_cold_start |
Suppress all learning — both online steps and periodic re-trains — until _trade_from. Then train once on the pre-tail window and resume online learning normally. This is "cold". |
Warm mirrors what a live EA actually does: it has been training and trading before the tail begins, so it arrives at the tail with an already-adapted model. Cold is the stricter test: the model has never seen anything before the tail boundary and receives one batch train from the pre-tail history before trading. See Step 5.
A third mode: signal recording
When record_signals is set, the engine stops simulating trades
entirely and instead collects raw entry signals — bar, side, entry price, SL, TP,
probability — without ever holding a position.
This exists because of a structural property: the hour filter gates entries only. Training reads its own separate mask, and online learning uses the bar's own forward return rather than position P&L. The weight trajectory is therefore identical for every hour subset.
lr_hours_engine.py exploits, and it is why the reports can quantify
the 24-hour-versus-compiled-window difference by exact replay rather
than by filtering the trade list — which would be wrong, since gating entries
changes which trades exist at all, not merely which ones are counted.