Market Stochasticity, randomness and system failure

I would just like to posit this question to the group ... especially the more experienced asicionados amongst you.

I have been looking at several technical systems, one of which had a success rate of over 95% spanning 2/3 years

Naturally one would expect this to break down at some stage or rather. But when for instance could one say that the system is no longer effective? I mean it is still possible to get a string of losers even where a system delivers high w/l ratios .... but statistically there must come a point where one must conclude that the relationships have irrevocably broken down & look elsewhere.

Your thoughts would be greatly appreciated


The one who spreads legs

2/3 years...lol...your not a 12 year old French quant are you? The success rate sounds curve fitted don't it? You could make decisions based on some statistics relating to past performance. So maybe doing a Monte Carlo analysis on the historical performance and working out how likely x series of losses are.

I don't really think this is effective though. More likely, you will have to make a subjective judgement about why it has worked in the past, why it works now and why it may not work in the future (I get the impression this is how Renaissance Tech does well. i.e they don't have one system that makes money, they have a lot and adapt them as they go...just speculating though). The only "systems" that work in every market are when you get paid a premium for certain risks as in the endowment model (i.e beta, volatility, liquidity) and, of course, these only "work" over long periods (i.e 20+ years) and for people on here the returns would probablly seem pointless.

cr6196 said:

2/3 years...lol...your not a 12 year old French quant are you?

Click to expand...

I wish I were .... French that is

cr6196 said:

The success rate sounds curve fitted don't it? You could make decisions based on some statistics relating to past performance. So maybe doing a Monte Carlo analysis on the historical performance and working out how likely x series of losses are.

Click to expand...

To be fair it ain't .... I ripped this one off another chap who undertook a 5 year + study (with some adaptations of my own ofcourse )
But I guess you're right. I will need to perform some rigorous testing of my own :smart:

cr6196 said:

I don't really think this is effective though. More likely, you will have to make a subjective judgement about why it has worked in the past, why it works now and why it may not work in the future (I get the impression this is how Renaissance Tech does well. i.e they don't have one system that makes money, they have a lot and adapt them as they go...just speculating though). The only "systems" that work in every market are when you get paid a premium for certain risks as in the endowment model (i.e beta, volatility, liquidity) and, of course, these only "work" over long periods (i.e 20+ years) and for people on here the returns would probablly seem pointless.

Click to expand...

Thanks for the contribution .... but I guess that for a single trader continuously adapting , testing all these different systems would be incredibly time consuming

spreader_legger said:

I wish I were .... French that is



To be fair it ain't .... I ripped this one off another chap who undertook a 5 year + study (with some adaptations of my own ofcourse )
But I guess you're right. I will need to perform some rigorous testing of my own :smart:




Thanks for the contribution .... but I guess that for a single trader continuously adapting , testing all these different systems would be incredibly time consuming

Click to expand...

my point was that your going off two to three years which really isn't enough, it doesn't really prove anything. try 20 years. yes it would be time consuming but if it was easy then everyone would do it. I forgot to ask, how far have you adjusted the parameters in your model? So if your looking at a momentum system what happens if you change time periods or use different definitions of momentum. If returns don't hold then its curve fitted. just to be clear, my last comments were about getting paid risk premiums i.e equity risk premium/beta.

I agree with everything CR said.
I noticed no mention was made of trade frequency or number of trades though.
Personally I would say that 2-3 years would be OK for higher trade freq (3+ per day avg) but trade costs will start to become a significant factor...
So yeah 5+ years backtest as a minimum.

TBH this whole issue of backtesting going back more than 5-10 years does bother me though.
Here's the reason why:

http://www.conatum.com/presscites/HFTMMI.pdf
Everyone is familiar with the effect of HFT algos on markets, its interesting to note
the scale and speed of uptake though.

Viewpoints on future direction of algo trading:
http://www.lampsacus.com/documents/ALGORITHMICTRADING.pdf
Yeah OK nothing concrete, may well all be bo11ox, but the point is the potential
is still there for greater market impact from algos.

Point is, markets are changing and have changed significantly since 2003-2005 due to the gradual increase in HFT.
That is why from my own point of view at least, I am unsure of the merits of backtests longer than 5-8 years.
Just my opinion thats all.

95% strike rate, that would be enough for me to bin it and start again...:whistling
As for when is it no longer working - when drawdown exceeds a realistic level you can cope with or it blows up