Backtesting issues

Albert Carey

Member
Messages
88
Likes
3
I read the paper by M. Harris and I wonder what influence wrong backtesting results might have had on trader profitability. Do you have any experiences where the same system showed different results when tested on different programs of this sort?
 
Wrong back-testing results? Back-testing is about the biggest waste of time for any trader AFAIAC so I'll not take the time to read the paper you ref, but if you can't even get the back-testing results 'right' - there's hope.
 
Backtesting can be very useful if done correctly. I wouldn't like to blindly trade a system that had in the past excessive drawdown and a small profit factor. At the same time, backtesting cannot provide any kind of certainty for future performance. Most people abuse backtesting by employing optimization. This is the main cause of losses. In addition, if some programs generate wrong results, things can get much worse.

I have noticed in the past that some backtesting programs did not generate signals where they should but I did not pay any attention to this crap. Most program vendors do not understand what their programs do anyway. Only when you develop your own you can have an understanding of what it does.
 
Top