System Still Tradeable?

[AllanXT]

When you start trading a tested system, how do you test it to make sure it is still tradeable? How much data should you use for an intraday system? An end of day system?

thanks and good trading all

Allan

 

[JonnyT]

The answer is you can argue each way until the cows come home.

You have two choices:

1) Go for it
2) Don't

JonnyT

 

[Vaco]

At the end of the day the only way to know for sure is to go for it, however if you have doubts before you start it is unlikely you will stick to the system when it hits a bad run. If you have back tested then you can look at the results in more detail to gain more confidence in particular look at the maximum no of losers in a row, say this is 15 if you trade the system and the maximum no of losers doubles that to 30 it might be a sign that the system has had its day.

ps. Make sure you use good money management to ensure that a run of losers like that will not leave you sweating buckets.

 

[jmreeve]

Some general system testing Guidlines:

1) Try to make sure you test with enough data to give at least 50-100 trades in your testing sample to give a reasonable level of confidence in the results.

2) Even if it is a very short term system, test over at least a years worth of data as markets often change short term trading characteristics suddenly on a longer time frame.

3) If you have any optimisable parameters. Optimise these on the first two thirds of the data and when you have finished and found the best parameters. Test on the last third of data to see if they still work. I would also recommend then optimising on the last 3rd of data and if the optimum parameters are very different from the first 2/3rds or the system is profitable for a lot less ot the parameter settings then be very wary.

4) Don't have more than a few optimisable parameters and optimise for system stability and not for best profit.

5) If the system looked great in the past but is looking like it is starting to perform less strongly on recent data also be very wary.

A useful book on system testing is "Design, Testing and optimisation of trading systems" by Robert Pardo

 

[AllanXT]

Some general system testing Guidlines:

1) Try to make sure you test with enough data to give at least 50-100 trades in your testing sample to give a reasonable level of confidence in the results.

2) Even if it is a very short term system, test over at least a years worth of data as markets often change short term trading characteristics suddenly on a longer time frame.

3) If you have any optimisable parameters. Optimise these on the first two thirds of the data and when you have finished and found the best parameters. Test on the last third of data to see if they still work. I would also recommend then optimising on the last 3rd of data and if the optimum parameters are very different from the first 2/3rds or the system is profitable for a lot less ot the parameter settings then be very wary.

4) Don't have more than a few optimisable parameters and optimise for system stability and not for best profit.

5) If the system looked great in the past but is looking like it is starting to perform less strongly on recent data also be very wary.

A useful book on system testing is "Design, Testing and optimisation of trading systems" by Robert Pardo

thank you for the great post. Is there a time fram that you use? For example, do you keep testing every 3 months to see if the probability of the system is still constatnt? Or would every month be better?

thanks,

Allan

 

[jmreeve]

thank you for the great post. Is there a time fram that you use? For example, do you keep testing every 3 months to see if the probability of the system is still constatnt? Or would every month be better?

I run my systems from Tradestation so I can see how the performance develops on a day-by-day basis and keep and eye on how well it is working.

Deciding when a system is failing is a matter of experience. For instance, if a trend following system is starting to show bad draw-down in a non-trending market it has not necessarily broken. If there are good fundamental reasons why a non-trending market is likely to continue then that would be a good argument to stop trading it or at least reduce position size. If the trend following system starts to show low profitability in a trending market then I would be very concerned and start to investigate what is going wrong.

I run multiple systems on multiple assets and also have systems sitting in reserve that are paper trading. If an active system starts to look suspect I will reduce its weighting in the portfolio and increase the weighting of another active system or start to trade one of the reserve systems. This way failing systems end up fading out of the portfolio and new stronger systems get included. Over time the portfolio slowly evolves as the market changes.

 

[AllanXT]

I so I can see how the performance develops on a day-by-day basis and keep and eye on how well it is working.

.

Thank you JM , day to day, will do.

Allan