Backtesting and optimization

[xyza]

Hi,
This is my first thread to this forum, I hope this will be a useful thread for me and everyone as well.

I am new at strategy trading and I have a trading system written on C# which gives buy and sell signals according to 10 parameters. The program retrieves the prices from a txt document. It is a very simple algorithm that gives buy and sell signals according to the last x previous bars' high, low, open and close prices. While writing the code on C#, I did not use any back-testing platform, therefore I created a results sheet which makes my code very slow. When I want to optimize (Simulate) my system according to ten parameters, the combination of ten parameters become very large and the program needs to be run for 30 days, which makes my code unfeasible for me. The program becomes very slow because it goes into lots of loops for calculating the sharpe ratio, drawdown, ulcer index etc for 20.000.000 combination of parameters. Can anyone help me on that which platform should I use to make optimization fast? I do not want to use this platform as an execution platform or charting , I just need a fast backtesting and optimization platform. I read about Tradestation, NinjaTrader, AmiBrokers, and C-Trader( Dr. Katz's platform, the writer of the Encyclopedia of trading strategies).

Thank you in advance.

 

[jcl365]

Ten parameters are too much. This will likely creating an overfitted strategy that performs well in a backtest, but fails in real trading. I think the reasonable maximum are about 6 parameters, 3 for entry and 3 for exit.

When your code already exists in C# then probably Ninjatrader is the easiest solution for testing.

 

[Vicorka]

The fastest way is to have raw data on your computer and run the scan on your computer - no network connection time.

Agree, 10 parameters are too much. I do back-testing directly on raw data in my SQL via my VB net:

1 parameter - 100 combination on 10-year daily history 1-3 sec
2 parameters - 10,000 combinations on 10-year daily history 110-20 sec
3 parameters - 1,000,000 combinations on 10-year daily history couple of minutes
4 parameters - 100,000,000 combinations on 10-year daily history couple of hours

I did not go above 4 parameters.

Recommendations that I may give:
1. Try to reduce the range for each parameter
2. Try to split test into 2-3 steps. Find the best settings for the 3-4 basic parameters. Then taking these parameters as constants scan the other 3-4 parameters and then the rest. I would not go above 4 parameters per scan.

P.S. Check the scan results of other services (free):
Stock Charts Scan
If you use any of the listed indicators it may narrow your scanning range - follow the link to signals to see the best indicators settings.

 

[tanyach]

Hi,
Can anyone help me on that which platform should I use to make optimization fast? I do not want to use this platform as an execution platform or charting , I just need a fast backtesting and optimization platform. I read about Tradestation, NinjaTrader, AmiBrokers, and C-Trader( Dr. Katz's platform, the writer of the Encyclopedia of trading strategies).

Hello! I know that ninja trader is a quite good tool for strategies optimization. Also you can try to use algo studio from protrader platform. It supports c# and mql4 languages and performs fast optimization and backtesting.

 

[AndreasClenow]

Congratulations on writing your first trading system. Starting out in C# is a good idea. It's a powerful language and many of the best software packages are based on CLR.

Tanyach recommended Ninja Trader, and I agree that this is a very good piece of software. I prefer WealthLab, RightEdge or Open Quant, depending on the task at hand, but between all these really strong platforms it's a matter of taste and needs.

Ninja and WL are a little easier to get started with while RE and OQ are a bit more powerful. If you are a strong programmer, you'd probably enjoy the latter two more. Since you already know C#, there is no need to consider the vast amount of consumer platforms around with proprietary scripting languages.

Be very careful in how you model your strategy to avoid data snooping. Set all assumptions on the negative side to avoid nasty surprises when you go live.

I'd also caution you against optimizations. As an industry pro, I would never use them. That kind of functionality mostly serves to curve fit and give you false hope.

Whether or not 10 parameters are too much depends very much on your definition of a parameter in this context.

I wish you good luck and happy trading!