A high-throughput predictive machine learning system generating automated trading signals from 5 years of tick-level and 1-minute EUR/USD OHLCV historical data.
The core deep learning architecture is a custom Gated Transformer Network (GTN) featuring a dual-tower structure (channel tower for feature interactions and temporal tower for sequential dependencies), benchmarked against gradient-boosted decision trees (LightGBM, CatBoost) and deployed with an automated MetaTrader 5 execution pipeline.
1. Research Motivation & Problem Framing
Financial time-series data at high frequencies (such as 1-minute foreign exchange bars) exhibits non-stationarity, regime shifts, and extremely low signal-to-noise ratios. Standard classification models trained on arbitrary fixed-horizon price targets (e.g., predicting whether price increases in exactly 15 minutes) fail to capture realistic trading dynamics because price trajectories often hit stop-loss boundaries before reaching targets.
To overcome these limitations:
Barrier-Aware Labeling: Training labels are constructed using a triple-barrier method (Take-Profit hit vs. Stop-Loss hit vs. maximum holding period), matching real execution constraints.
Dual-Tower Representation: Separating cross-feature correlation attention from temporal attention enables the network to isolate microstructural signals from temporal trend components.
Execution Rigor: Entry prices are explicitly shifted to the next-candle open ($T+1$), eliminating lookahead bias and unrealistic slippage assumptions.
2. Experimental Benchmark Results
Every candidate architecture was evaluated across multi-year out-of-sample test splits using a balanced Composite Precision Score and backtested in MetaTrader 5:
Model Architecture Composite Score Key Strengths & Observations LightGBM (Baseline) 0.37 Strong feature-importance split speed; best single run but stochastic variance across seeds (typical: 0.20–0.25). GTN (Dual-Tower Transformer) 0.36 Learned temporal embeddings outperformed static sinusoidal encodings; captured multi-step volatility regimes. CatBoost + Sharpe Optimization — Directly optimized reward-to-risk (Sharpe ratio loss) rather than cross-entropy loss. Multi-Stage Volatility Regression — Volatility-adaptive position sizing based on rolling Average True Range (ATR). Regime-Aware Ensemble — Dynamically switches TP/SL thresholds based on market regime detection (trending vs. mean-reverting). Direct P&L Neural Network — End-to-end differentiable P&L loss with integrated fractional Kelly criterion sizing.
$\text{Composite Score} = \frac{\text{Precision}{\text{buy}} + \text{Precision}{\text{sell}}}{2} - 0.25 \cdot \text{Precision}{\text{buy}} - \text{Precision}{\text{sell}} $
Core Research Finding: In quantitative trading, higher cross-entropy classification accuracy does not correlate linearly with live strategy profitability. Models trained with asymmetric cost matrices and TP/SL-barrier targets produced significantly lower maximum drawdowns during high-volatility macro news releases.
3. End-to-End Pipeline Architecture
Feature Engineering: Multi-timeframe momentum indicators (5m, 15m, 1h, 4h), rolling support/resistance levels, spread-adjusted volatility metrics, and London/New York session overlap flags.
Data Normalization: Rolling z-score normalization to prevent future data leakage across temporal splits.
4. Engineering Gotchas & Expensive Lessons
Encoding Requirements: MetaTrader 5 requires UTF-16LE character encoding for signal CSV ingestion; standard UTF-8 silently fails to load trade instructions without error logs.
Label Alignment: sklearn.LabelEncoder defaults to alphabetical ordering (buy, keep, sell $\to$ 0, 1, 2), which requires explicit mapping to directional integers (1, 0, -1) before MT5 MQL5 scripts parse them.
Sequence Overhead: Sequence flattening is unnecessary for gradient-boosted tree models (CatBoost/LightGBM); building large sequential buffers for trees wasted memory without predictive gain.
Class Imbalance: Market data naturally defaults to 85%+ "keep/flat" states. Applying a weighted cross-entropy cost matrix yielded superior precision over naive downsampling or standard focal loss.
5. Repository Contents & Resources
datapipeline/: Canonical TP/SL triple-barrier signal generator and tick-level cleaning routines.
model/: PyTorch implementation of the dual-tower Gated Transformer Network (GTN).
backtesteval/: Comprehensive backtesting evaluation suite with composite score calculation and drawdown analytics.
mql5/: Live execution and signal-listener scripts for MetaTrader 5.
View on GitHub