AriaS
Well-known member
- Messages
- 363
- Likes
- 121
When we backtest trading strategies, we must be careful and do it in a way that makes our backtests reliable.
For example, we design a strategy and want to backtest it. We might think that the more historical data we use, the better the result will be. So we take three years of data, optimize the strategy, choose the best setup, and assume it will work well going forward on a live account.
This is a common trap. At this stage, we don’t know whether the strategy is genuinely robust or just another curve-fitted result. We haven’t validated it, and we haven’t developed a recurring backtesting protocol that itself has been tested.
The main step we missed earlier is validation (also known as out-of-sample validation). We don’t want our live trading to be the validation period. What if it fails? Isn’t it better to validate on historical data first? The answer is obvious.
So our next steps should be the following.
First, we choose a period to optimize on. This requires experimentation. On one hand, the period must be long enough to produce a sufficient number of trades so the results are statistically meaningful. On the other hand, it must not be too long, because market behavior changes, and we want our tests to be as relevant to the present as possible.
Let’s say we decide on a three-month period. Now we are going to backtest our backtesting protocol. One possible approach is:
My strategy for finding a profitable strategy (which I will attach to all my posts):
Borrow or design a strategy → backtest and optimize to see if it truly works (not MT4!) → validate out-of-sample → start with the smallest possible account → scale → adapt to market changes by repeating steps 2 and 3 as often as possible.
Wishing you all success!
For example, we design a strategy and want to backtest it. We might think that the more historical data we use, the better the result will be. So we take three years of data, optimize the strategy, choose the best setup, and assume it will work well going forward on a live account.
This is a common trap. At this stage, we don’t know whether the strategy is genuinely robust or just another curve-fitted result. We haven’t validated it, and we haven’t developed a recurring backtesting protocol that itself has been tested.
The main step we missed earlier is validation (also known as out-of-sample validation). We don’t want our live trading to be the validation period. What if it fails? Isn’t it better to validate on historical data first? The answer is obvious.
So our next steps should be the following.
First, we choose a period to optimize on. This requires experimentation. On one hand, the period must be long enough to produce a sufficient number of trades so the results are statistically meaningful. On the other hand, it must not be too long, because market behavior changes, and we want our tests to be as relevant to the present as possible.
Let’s say we decide on a three-month period. Now we are going to backtest our backtesting protocol. One possible approach is:
- Optimize on January 2025 – March 2025 (I suggest ranking results by Recovery Factor).
- Validate out-of-sample on October 2024 – December 2024. Test each optimization result, from best to worst, preferably with more trades rather than fewer, until we find a setup with a satisfactory Recovery Factor (for example, above 2.0).
- Test this chosen setup on April 2025 – June 2025, as if it were the future. Did it work? Did it make a profit? If yes, then we may be onto something real.
- Repeat steps 1–3, shifting everything one month forward.
- Repeat steps 1–4.
My strategy for finding a profitable strategy (which I will attach to all my posts):
Borrow or design a strategy → backtest and optimize to see if it truly works (not MT4!) → validate out-of-sample → start with the smallest possible account → scale → adapt to market changes by repeating steps 2 and 3 as often as possible.
Wishing you all success!
Last edited:
