Has anyone actually backtested their EA over 10+ years of data, or are we all just gambling with code?

[ThaddeusFinch]

Everyone claims their EA is profitable long-term, but has anyone actually backtested theirs across multiple market cycles (10+ years)? Or are we just curve-fitting recent data and calling it an edge?!

 

[RolloTape2025]

Answer is yes to the backtesting and no to the 'just curve fitting'. If you are curve fitting it sounds like you are adjusting settings to try make a system work. Which is ok, but also has to be tested out of sample, right?

Second answer is that some things are not easily backtestable but still work.

 

[Opak]

Answer is yes to the backtesting and no to the 'just curve fitting'. If you are curve fitting it sounds like you are adjusting settings to try make a system work. Which is ok, but also has to be tested out of sample, right?

Second answer is that some things are not easily backtestable but still work.

When you do out-of-sample testing, do you use walk-forward analysis or something else?

 

[RolloTape2025]

Walk forward analysis is one way yes. It doesn't always have to be 'forward'. As long as you are dividing historical data into distinct parts that don't include the period you used to optimise. You could for example have a system optimised on this past year. But how about stress testing it? How would it have performed during 2007-2009 financial crisis? You want the strategy to still survive and work under different conditions.

I am not a fan of curve fitting at all though. I think it gives a false sense of edge. Conceptually it doesn't seem like a good idea either, but each to their own. There is something to adapting a strategy to recent conditions, but I've personally only found it useful to adapt to volatility, or sometimes just to step away if the news is a bit wild.

Irrespective of all of that, the one thing that matters is actually trading it for real (do it in the minimum size possible at first), and see how it performs. So many things come up in real trading, lots of slippage, orders that you think would get filled but don't, overnight fees, widened spreads etc.

Best of luck

 

[TickCOM]

It makes little sense to backtest more than 2 years back, depending of the timeframe. It makes not sense to make an automated strategy in higher timeframes than 1 hour, even above 15 minutes, definitely not using daily bars.

The recent data is the most important, since they show if the strategy works right now. Forward testing is also a curve fitting, actually the longer you forward test a working strategy, the higher probability it will stop working.

Which is why you cannot operationally develop any kind of strategy using daily bars - it will take years to forward test it, or test if it is doing what it is supposed to do.

You would test the strategy in different ways for robustness etc, but set it live immediately.

 

[RolloTape2025]

It makes little sense to backtest more than 2 years back, depending of the timeframe. It makes not sense to make an automated strategy in higher timeframes than 1 hour, even above 15 minutes, definitely not using daily bars.

The recent data is the most important, since they show if the strategy works right now. Forward testing is also a curve fitting, actually the longer you forward test a working strategy, the higher probability it will stop working.

Which is why you cannot operationally develop any kind of strategy using daily bars - it will take years to forward test it, or test if it is doing what it is supposed to do.

You would test the strategy in different ways for robustness etc, but set it live immediately.

🙁

 

[Opak]

Slippage, missed fills, fees—those things just don’t show up on backtests. Curious though—how far back do you guys usually test when optimizing?