Forward Testing.

brewski1984

Senior member
Messages
2,067
Likes
355
Ok so the forum had a huge banner across it telling me I hadn't posted for a while and to make a contribution so although I'm sure it's been asked before, after how much forward testing can you assume a statistically significant result to show an edge?

A time frame of say 1 month, 3 months or 6 months?
An amount of trades like 10, 50 or 100?
 
The simple answer to this is that you need enough trades to determine that the result you achieved could not have happened just by random chance. Unfirtunately, The number depends on the characteristics of your system

Using extremes as an example, if you had a system that used a 1 point target with a 1000 pip stop, then you'd obviously end up with a very high win rate. Under those circumstances, a sample size of 10 trades with 10 wins is fairly meaningless. Conversely, if you swapped the stop and target around and achieved 10 wins from 10 trades, a sample size of 10 would be sufficient !

So the number is dependent on the distribution of gains and losses,

I guess most people use a Monte Carlo approach
 
Ok so the forum had a huge banner across it telling me I hadn't posted for a while and to make a contribution so although I'm sure it's been asked before, after how much forward testing can you assume a statistically significant result to show an edge?

A time frame of say 1 month, 3 months or 6 months?
An amount of trades like 10, 50 or 100?

brewski1984

Which testing system are you using?

When you say forward testing, how does this differ from back testing?
 
The simple answer to this is that you need enough trades to determine that the result you achieved could not have happened just by random chance. Unfirtunately, The number depends on the characteristics of your system

Using extremes as an example, if you had a system that used a 1 point target with a 1000 pip stop, then you'd obviously end up with a very high win rate. Under those circumstances, a sample size of 10 trades with 10 wins is fairly meaningless. Conversely, if you swapped the stop and target around and achieved 10 wins from 10 trades, a sample size of 10 would be sufficient !

So the number is dependent on the distribution of gains and losses,

I guess most people use a Monte Carlo approach

Interesting. I have just googled Monte Carlo approach, it looks too confusing to be looking at properly on a saturday night but I'll have another go somewhen.

I am not testing any system at the moment PollyM, it was just a general topic to get some conversation going as the forum told me to do it(y)

I think forward testing is a lot more valuable than back testing as there's no opportunity to curve fit into a "profitable" system and if using small stakes you at least get a little bit of live trading into the results. That doesn't mean I ignore back testing though.
 
Interesting. I have just googled Monte Carlo approach, it looks too confusing to be looking at properly on a saturday night but I'll have another go somewhen.

I am not testing any system at the moment PollyM, it was just a general topic to get some conversation going as the forum told me to do it(y)

I think forward testing is a lot more valuable than back testing as there's no opportunity to curve fit into a "profitable" system and if using small stakes you at least get a little bit of live trading into the results. That doesn't mean I ignore back testing though.

Back testing, forward testing, are you not dealing with the same data?
 
But you know that time will not change an apple tree into a peach tree.
 
But you know that time will not change an apple tree into a peach tree.

I do see your point that once a trade is closed it becomes past data just like everything else. Have you ever found a system that was proven to be profitable in backtesting but when you traded it you lost?
 
Unless you have built a back-testing system that identified and adapted to the market as it transitioned from one phase to the next, it wont make any difference how far you forward test. Purely because in your forward testing you will not be in a position to identify when the system will be profitable or when it wont. so you chosen time to forward test could fall into the part of the system that couldn't adapt to the market conditions.

I you have identified this aspect then the back-test results will give an idea of forward results. I don't know many system that can do this as the work involved is huge and not understood by most.
 
How did this end up in a back testing vs forward testing argument? I agree that backtesting can have a lot of value. I do happen to stumble across the phrase "past performance is not indicative of future results" rather a lot though.
 
How did this end up in a back testing vs forward testing argument? I agree that backtesting can have a lot of value. I do happen to stumble across the phrase "past performance is not indicative of future results" rather a lot though.

Your right. Sorry for diverting,;)
 
Top