Forward Testing.

brewski1984

Senior member
2,073 356
Ok so the forum had a huge banner across it telling me I hadn't posted for a while and to make a contribution so although I'm sure it's been asked before, after how much forward testing can you assume a statistically significant result to show an edge?

A time frame of say 1 month, 3 months or 6 months?
An amount of trades like 10, 50 or 100?

the hare

Senior member
2,949 1,283
The simple answer to this is that you need enough trades to determine that the result you achieved could not have happened just by random chance. Unfirtunately, The number depends on the characteristics of your system

Using extremes as an example, if you had a system that used a 1 point target with a 1000 pip stop, then you'd obviously end up with a very high win rate. Under those circumstances, a sample size of 10 trades with 10 wins is fairly meaningless. Conversely, if you swapped the stop and target around and achieved 10 wins from 10 trades, a sample size of 10 would be sufficient !

So the number is dependent on the distribution of gains and losses,

I guess most people use a Monte Carlo approach

PollyM

Active member
195 7
Ok so the forum had a huge banner across it telling me I hadn't posted for a while and to make a contribution so although I'm sure it's been asked before, after how much forward testing can you assume a statistically significant result to show an edge?

A time frame of say 1 month, 3 months or 6 months?
An amount of trades like 10, 50 or 100?

brewski1984

Which testing system are you using?

When you say forward testing, how does this differ from back testing?

brewski1984

Senior member
2,073 356
The simple answer to this is that you need enough trades to determine that the result you achieved could not have happened just by random chance. Unfirtunately, The number depends on the characteristics of your system

Using extremes as an example, if you had a system that used a 1 point target with a 1000 pip stop, then you'd obviously end up with a very high win rate. Under those circumstances, a sample size of 10 trades with 10 wins is fairly meaningless. Conversely, if you swapped the stop and target around and achieved 10 wins from 10 trades, a sample size of 10 would be sufficient !

So the number is dependent on the distribution of gains and losses,

I guess most people use a Monte Carlo approach

Interesting. I have just googled Monte Carlo approach, it looks too confusing to be looking at properly on a saturday night but I'll have another go somewhen.

I am not testing any system at the moment PollyM, it was just a general topic to get some conversation going as the forum told me to do it

I think forward testing is a lot more valuable than back testing as there's no opportunity to curve fit into a "profitable" system and if using small stakes you at least get a little bit of live trading into the results. That doesn't mean I ignore back testing though.

PollyM

Active member
195 7
Interesting. I have just googled Monte Carlo approach, it looks too confusing to be looking at properly on a saturday night but I'll have another go somewhen.

I am not testing any system at the moment PollyM, it was just a general topic to get some conversation going as the forum told me to do it

I think forward testing is a lot more valuable than back testing as there's no opportunity to curve fit into a "profitable" system and if using small stakes you at least get a little bit of live trading into the results. That doesn't mean I ignore back testing though.

Back testing, forward testing, are you not dealing with the same data?

the hare

Senior member
2,949 1,283
Back testing, forward testing, are you not dealing with the same data?

No, forward testing is out of sample. Back testing may or may not be out of sample.

brewski1984

Senior member
2,073 356
How can it be the same data when it hasn't happened yet?

the hare

PollyM

Active member
195 7
I don't understand, what is meant by out of sample?

PollyM

Active member
195 7
How can it be the same data when it hasn't happened yet?

What I am writing now will be the past by the time you read it.

2,949 1,283

brewski1984

Senior member
2,073 356
What I am writing now will be the past by the time you read it.

Yeah but I didn't know you were going to write it until afterwards

PollyM

Active member
195 7
But you know that time will not change an apple tree into a peach tree.

PollyM

Active member
195 7

Sounds like a flawed premise to me.

brewski1984

Senior member
2,073 356
But you know that time will not change an apple tree into a peach tree.

I do see your point that once a trade is closed it becomes past data just like everything else. Have you ever found a system that was proven to be profitable in backtesting but when you traded it you lost?

wackypete2

Legendary member
10,229 2,055
Forward testing is when you backtest yesterday's data tomorrow.

Peter

Replies
4
Views
1K
Replies
0
Views
658
Replies
8
Views
2K
Replies
4
Views
761
Replies
101
Views
133K