which ratios should I use for calibrating system parameters?

[terryzz5]

:idea: I tried many different ways to calibrate system parameters. In-sample performance is really nice but out-of-sample performance is always ****.

Which ratios do you guys use? Or many suggestions? Please!

 

[intradaybill]

:idea: I tried many different ways to calibrate system parameters. In-sample performance is really nice but out-of-sample performance is always ****.

Which ratios do you guys use? Or many suggestions? Please!

Try the Kelly ratio.

 

[the hare]

Any sort of optimisation is doomed to failure. If the method is 100% mechanical and TA based, you need to diversify, because this years star performer will probably be a right dog next year, and vice versa.

 

[terryzz5]

thanks, mate. But I meet new problems. After fixing the parameters, I filtered numbers of strategies which work very well out-of-sample. So I then select one used to trade, but the performance in reality still sucks.. what can I do?

Any sort of optimisation is doomed to failure. If the method is 100% mechanical and TA based, you need to diversify, because this years star performer will probably be a right dog next year, and vice versa.

 

[the hare]

thanks, mate. But I meet new problems. After fixing the parameters, I filtered numbers of strategies which work very well out-of-sample. So I then select one used to trade, but the performance in reality still sucks.. what can I do?

I think you need to understand exactly what market behaviour the system is trying to exploit. Most TA based sytems you see on forums are nothing more than a collection of indicators that just happened to work well over some period just by random chance. There needs to be some basic concept thats being traded.

Once you know what you are trying to exploit, it makes sense to diversify, trading one TA based system mechanically is asking for trouble.

 

[Blaiserboy]

I have found that even though out of sample looks good, unless Sharpe Ratio is near 3 or greater, chances are great that the thing will fail.

 

[terryzz5]

I have found that even though out of sample looks good, unless Sharpe Ratio is near 3 or greater, chances are great that the thing will fail.

how long do u use for testing out-of-sample performance? 3 yr, 4 yr, 5 yr or even longer? :idea:

 

[Blaiserboy]

how long do u use for testing out-of-sample performance? 3 yr, 4 yr, 5 yr or even longer? :idea:

I am trading intraday, I use 15 to 25000 bars, 25 to 35 percent OOS

 

[Joey25]

:idea: I tried many different ways to calibrate system parameters. In-sample performance is really nice but out-of-sample performance is always ****.

Which ratios do you guys use? Or many suggestions? Please!

Try going long the star performer and short the crap performer. It's been tried for moving averages and apparently earns excess returns.

 

[terryzz5]

Try going long the star performer and short the crap performer. It's been tried for moving averages and apparently earns excess returns.

Thank you very much, Joey25, for your suggestions. I'll try that. Now I've sorted my problems out. I ran my system for a out-of-sample simulation, which generates nice and stable returns over the last 6 yr. But I'm still very stressed since I may use it and be going to be a pro-trader. So I still keep looking for stable strategies that are not affected by the market.

 

[sr100m]

Agree with the poster who warns about optimizing. If you are over optimizing then it does not really matter what parameters you use - none of them will tell you how robust the system is. Given your post about being stressed I suspect deep down you know that. Google Tom Basso on the importance of being comfortable

I find most of these parameters tell me more about whether I would be able to trade the damn thing or not. I doubt any parameters can tell you whether a system will be robust or not. A system will be robust if it captures a behavior that will last - and that is primarily a qualitative judgment.

I start looking for basic positive expectancy and what the return would be for different position sizes. Then I look at max drawdown to see whether that return is likely to break the bank. Then I look at some smoothness parameter (risk to drawdown ratio, van tharp's SQN, sharpe). Finally, I look at a whole boatload of the psych parameters - durations of drawdowns, rolling monthly, weekly returns etc. to see if I can make myself stick to it. I also run results through monte carlo simulations in market system analzyer to get some worst case scenarios.

 

[intradaybill]

Win rate is everything.