Files & Outputs
Where everything lives, what every artefact contains, and the naming rules that determine whether two runs collide or coexist.
The two parts of the system
There are two separate trees, and it is worth being clear about which is which.
The GUI package. Settings, pages, widgets, the runner. Knows how to drive the pipeline; contains none of the trading logic.
The research pipeline. scripts/, data/,
logs/ and the generated output roots. This is what the app points at.
How the repository is located
On startup, candidate folders are tested in order and the first one that looks like the repo wins:
1. the parent of the package folder # <repo>/FxMath_LR_Studio -> <repo>
2. the package folder itself # exe dropped into the repo root
3. two levels up
4. the current working directory
5. the parent of the working directory
A folder counts as the repository only if both of these files exist:
scripts/feature_search.py
scripts/gen_ea.py
scripts/ is not the repository. This is deliberate — it prevents the app
from silently driving the wrong folder.
Application layout
FxMath_LR_Studio/
├── run.py # double-clickable launcher / frozen entry point
├── __main__.py # python -m FxMath_LR_Studio
├── __init__.py # exports APP_NAME, APP_VERSION, Settings
├── app.py # the window, navigation, output routing
├── config.py # settings, discovery, path conventions
├── runner.py # Task, JobManager, Chain, Presets
├── state.py # read artefacts off disk, judge freshness
├── mt5.py # compile, parse the log, deploy
├── mt5env.py # find terminals, pair install ↔ data dir
├── pyrunner.py # the frozen interpreter stand-in
├── theme.py # palette and fonts
├── widgets.py # Card, KPI, LogConsole, charts
├── pages/ # one module per screen
├── FxMath_LR_Studio.spec # the PyInstaller build recipe
├── build_exe.bat # build the standalone exes
├── run_studio.bat # run from source
├── requirements.txt
├── settings.json # persisted settings
└── README.md
The pages
| Module | Screen | Sidebar group |
|---|---|---|
dashboard.py | Dashboard | Overview |
pipeline_page.py | Full pipeline | Overview |
data_page.py | Data | Pipeline |
optimize_page.py | Optimize | Pipeline |
portfolio_page.py | Portfolio | Pipeline |
hours_page.py | Trading hours | Pipeline |
holdout_page.py | Holdout | Pipeline |
ea_page.py | Expert Advisor | Deliverables |
reports_page.py | Reports | Deliverables |
settings_page.py | Settings | Setup |
The sidebar order is the pipeline order, which is deliberate — the navigation doubles as a workflow diagram.
Repository layout
<repo>/
├── scripts/ # the pipeline
│ ├── download_ohlc_all.py
│ ├── check_ohlc_csv.py
│ ├── feature_search.py # hard marker for repo detection
│ ├── build_allstar.py
│ ├── best_hours.py
│ ├── holdout_tail.py
│ ├── gen_ea.py # hard marker for repo detection
│ ├── report_m5.py
│ ├── report_tail.py
│ ├── broker_run_report.py
│ ├── backtest_lr.py # the engine
│ ├── feature_sets.py # the 11 sets + primitives
│ ├── lr_hours_engine.py # exact hour-subset replay
│ ├── ExportOHLC_MQL5.mq5 # broker-side exporter
│ └── … # study scripts
├── data/ # OHLC CSVs
│ └── XAUUSD_M5.csv
├── logs/ # caches and run logs
├── strategies_lr_m5/ # XAUUSD M5 output root
│ ├── <set>/ # one folder per feature set
│ ├── all9/ # the portfolio
│ ├── best_hours.json
│ └── report*.html
└── README.md
Output roots and naming
| Symbol | Output root |
|---|---|
XAUUSD | strategies_lr_<tf> — e.g. strategies_lr_m5 |
| Any other symbol | strategies_<symbol>_<tf> — e.g. strategies_dowjones30_m5 |
strategies_lr_<tf> form predates multi-symbol support. The app
mirrors it exactly rather than imposing its own convention, so files generated by hand
and by the GUI land in the same place.
Magic numbers
Every slot has a unique magic number, built from the set's base plus a timeframe offset plus the slot index:
magic = (LR_MAGIC_BASE[set]) + TF_MAGIC_OFFSET[tf] + slot_index
where LR_MAGIC_BASE[set] = FEATURE_SETS[set]["magic_start"] + 100
| Set | magic_start | LR base | Slots (M5) |
|---|---|---|---|
base9 | 301 | 401 | 401–405 |
momentum | 311 | 411 | 411–415 |
volatility | 321 | 421 | 421–425 |
price_action | 331 | 431 | 431–435 |
trend | 341 | 441 | 441–445 |
full15 | 351 | 451 | 451–455 |
micro | 361 | 461 | 461–465 |
voltrend | 371 | 471 | 471–475 |
meanrev | 381 | 481 | 481–485 |
mafan | 391 | 491 | 491–495 |
breakout | 401 | 501 | 501–505 |
| Timeframe | Offset | Timeframe | Offset |
|---|---|---|---|
| M1 | 4000 | M30 | 5000 |
| M2 | 2000 | H1 | 6000 |
| M5 | 0 | H4 | 7000 |
| M10 | 3000 | D1 | 8000 |
| M15 | 1000 |
So base9 on M5 occupies 401–405 and the same set on M15 occupies 1401–1405.
Several timeframes can therefore run on one account without their positions being
confused.
Artefact schemas
data/<SYMBOL>_<TF>.csv
time,open,high,low,close,tick_volume,spread,real_volume
timeYYYY-MM-DD HH:MM:SS, oldest → newest<root>/<set>/summary.json
A per-set result. The header keys, shared with portfolio summaries:
| Key | Contents |
|---|---|
engine | Always "lr" for this pipeline. |
set | The set name — or the portfolio folder name for a portfolio summary. |
abbr | The abbreviation used in filenames. |
data | The data file's basename. This is how downstream stages find the CSV. |
bars | Row count of that file. |
magic_start | Lowest magic in the group. |
n_features | Feature count. |
union_features | The full feature list — what this set actually models. |
max_spread | The median spread, used as the cost basis. |
generated | Timestamp. |
slots[] | The slot array — see below. |
Each entry in slots[]:
| Key | Contents |
|---|---|
magic | This slot's magic number. |
set | Which feature set it came from. |
features | The feature list used. |
params | Every tuned parameter: sl_mult, rr, lookahead, training_bars, retrain_interval, batch_lr, online_lr, long_thresh, short_thresh, max_iter, and the base money settings. |
metrics | pf, np, tr, buys, sells, wr, dd, sharpe, score. |
combined, containing the portfolio's np and max_dd.
Otherwise the schema is identical, which is why the same readers work for both.
<root>/<set>/configs.json
A list of parameter dicts — one per slot, including its magic. This is the input the EA generator consumes, and it is the file to inspect when you want to know what a run actually selected.
<root>/<set>/slots.txt
The human-readable equivalent of configs.json: a metrics line per slot, then
every parameter on its own line. Useful for diffing two runs.
<root>/<set>/equity_<magic>.csv
One column, equity. Used by the correlation filter and the charts.
<root>/<folder>/equity_combined.csv
The combined portfolio curve, after warmup alignment and summation.
<root>/best_hours.json
{
"engine": "lr",
"symbol": "XAUUSD",
"tf": "M5",
"data": "XAUUSD_M5.csv",
"<set>": {
"best_windows": [
{ "hours": [...], "len": 5, "pnl": 812, "spec": "08-12", "n": 210,
"in_pnl": 671, "in_n": 158, "oos_pnl": 141, "oos_n": 52 }
],
"hour_pnl": { "0": ..., "1": ..., …, "23": ... },
"hour_n": { "0": ..., …, "23": ... },
"slot_magic": 401
}
}
best_windows[0].spec is what the EA generator bakes into
InpSlot<N>Hours.
<root>/<set>/trades.csv
The trade statement for a set's best slot. Columns: side,
entry, exit, pnl, reason,
prob, bars_held, bar_entry,
bar_exit.
reason is "SL", "TP" or "EOData" —
the last meaning the position was still open at the end of the data and was closed at
the final price.
<root>/<folder>/tail<N>_<mode>/tail_report.json
{
"folder": "all9",
"mode": "warm",
"tail_bars": 2500,
"n_bars": 10000,
"tail_start": 7500,
"combined": { "np": 57, "max_dd_pct": -0.41, "sharpe": 1.23,
"buys": 400, "sells": 414, "trades": 814 },
"slots": [
{ "magic": 401, "set": "base9", "np": 9, "dd": -0.12, "sharpe": …,
"trades": 142, "wins": 76, "buys": 70, "sells": 72,
"pnl_per_trade": 0.06,
"sl": 2.0, "rr": 1.5, "la": 3, "tb": 1000, "ri": 500,
"blr": 0.01, "th": 0.55 }
]
}
n_bars is the staleness key
It records the CSV's row count at the time the holdout ran. If the current CSV has a
different count, the holdout is flagged stale — because the frozen configurations were
selected on different data and the result no longer describes the file you have.
The generated EA
| Pattern | Example |
|---|---|
| Portfolio EA | <root>/<folder>/FxMath_AST_allstar_<FOLDER>_<TF>.mq5 |
| With a tag | FxMath_AST_allstar_ALL9_24X_M5.mq5 |
| Per-set EA | <root>/<set>/FxMath_<ABBR>_<set>_LR_<TF>.mq5 |
| Non-XAUUSD | FxMath_AST_allstar_ALL9_DOWJONES30_M5.mq5 |
| With a prefix | FU_FxMath_AST_allstar_ALL9_M5.mq5 |
Reports
| File | Producer |
|---|---|
<root>/report.html | report_m5.py |
<root>/tail_holdout_report.html | report_tail.py |
<root>/tail_holdout_report_<LABEL>.html | report_tail.py --label |
<root>/report_<tf>_performance.html | broker_run_report.py — e.g. report_m5_performance.html |
Settings
settings.json lives next to the app, or next to the frozen executable, so a
rebuild does not lose it. It is a flat JSON object, sorted by key.
| Key | Meaning |
|---|---|
repo | Path to the pipeline repository. |
python | The configured interpreter for source runs. |
use_bundled_python | Prefer the frozen runner when one is present. |
mt5_dir | The selected terminal's install folder. |
mt5_data_dir | Its %APPDATA% data directory — where MQL5\Experts lives. |
symbol, symbol_suffix | The clean instrument name and the broker's decoration. |
auto_suffix | Keep the suffix in step when the terminal changes. |
tf | Selected timeframe. |
tail | Tail length for the holdout. |
cores | Worker count. |
use_wine | Follows the platform, not the file — a settings file copied from Linux cannot make Windows shell out to wine. |
wine_prefix | For non-Windows hosts. |
appearance | dark or light. |
autoscroll, confirm_jobs, open_report_after, deploy_after_compile | Behaviour toggles. |
Repairs applied on load
Stale values are corrected every time settings load, and the corrections explain several surprising behaviours:
| Condition | Repair |
|---|---|
repo or python points at something that no longer exists | Re-detected. |
repo exists but is not a repository | Cleared and re-detected. |
use_wine does not match the current platform | Corrected to the platform. |
mt5_dir is a stale or POSIX path | Re-detected. |
mt5_data_dir belongs to a different terminal | Corrected — so a deploy cannot silently target the wrong terminal. |
Which files are stale when
The app derives stage status from artefact modification times. This table is the complete rule set.
| Stage | Artefact | Stale when |
|---|---|---|
| Data | data/<SYM>_<TF>.csv | Missing. |
| Search | <set>/summary.json | Older than the data CSV. |
| Portfolio | <folder>/summary.json | Older than the per-set summaries it was built from. |
| Hours | best_hours.json | Older than the portfolio summary. |
| Holdout | tail_report.json | Its recorded n_bars differs from the current CSV row count. |
| EA | *.ex5 | Older than its .mq5 — reported as an error, not a warning. |
| Reports | report*.html | Older than the summaries they describe. |
.ex5 sitting next to a new
.mq5 looks exactly like a successful build.
Collision rules
What protects runs from overwriting each other:
| Dimension | Separated by |
|---|---|
| Symbol | The output root — strategies_lr_m5 vs strategies_dowjones30_m5. |
| Timeframe | The output root's suffix, and the EA's filename suffix. |
| Feature set | The <set> folder and the magic number range. |
| Slot | The slot index within the magic range, and equity_<magic>.csv. |
| Holdout mode | The tail<N>_<mode> folder. |
| EA variant | --tag, which is inserted into the filename. |
| Dataset variant | --prefix on the generator, and --label on the tail report. |
FU, PU)
that share a symbol and timeframe with the primary data. Generating both into
strategies_lr_m5/ overwrites. Use --prefix to keep them
distinct, and be aware that the app always drives the single configured
data/<SYMBOL>_<TF>.csv path.
Which files are safe to delete
| Path | Safe? | Consequence |
|---|---|---|
build/ | yes | PyInstaller scratch. Rebuilt on the next build. |
dist/ | yes | The built executables. Rebuilt by build_exe.bat. |
<root>/<set>/equity_<magic>.csv | with care | Needed by the correlation diagnostics and the charts. Recreated by re-running the search. |
<root>/<set>/summary.json | no | Every downstream stage reads it. Deleting means re-running the search for that set. |
<root>/<folder>/tail* | yes | The holdout can be re-run in minutes from the frozen portfolio. |
<root>/report*.html | yes | Regenerated from the artefacts. Keep the final delivery report for the record. |
data/*.csv | no | Everything is derived from these. Re-export from the terminal if deleted. |
settings.json | with care | Deleting resets the app to auto-detected defaults. Harmless, but you will re-pick the terminal. |