Optimising mechanical trading strategies

[JTrader]

I have recently acquired ts2000i, and plan to do all my system writing, backtesting and optimising with this software.

say you were using a day period or intraday period mechanical trading system for example, which you had back-tested over 1,2 or maybe even 3 years. How often would you review the parameters (and optimise the parameters if necessary)? does on a monthly basis sound sensible?

And if paying attention to the most recent past is important. If performing monthly review backtests/optimisations, over how far backs worth of data would you perform the monthly review backtests/optimisations?

Would any of these considerations be affected by the length of the intraday timeframes that you traded within? if so, how?

do you think it is possible to trade basically the same mechanical day or intraday system, using the same indicators, for years on end, with the only alterations that you do make being periodical reviews/optimisations of the included 1 or 2 indicators parameter settings?

Many thanks

jtrader.

 

[Tuffty]

I would optimise on every 'significant number' of trades (could be 30 or higher).
I would backtest back as much as possible.
I would not be bothered about length of trade just the number of trades.
I would not trade on an intraday basis, but then I'm Tuffty and you're jtrader!

 

[TWI]

It is all about how you optimise. The complexity increases with the number of variables. I think TS may lead people into a very false sense of security with its optimisation functionality. I think you cannot rely on this alone as you actually need to find stability points in a system with regard to variables rather than point that produce favourable PnL or max draw or profit factor etc.
I think that simple systems can remain profitable over the long term without changing any parameters. The only problem I have found with daily rather than intra-day systems is the inherent draw down. I do not believe there is a daily system which will not take you through times of real pain along the way to profitability.

 

[mr_cassandra]

Same system & indicators for years on end

The brief answer, in my opinion, is yes. I have to qualify that by saying that what I do & use is very far from 'trading' per se, however. MY mechcnical trading system, know as the mvp signal system (not a commercail product-just for my own use) covers a span of 8.5 years (so far) and applies the exact same logic set across the entire span. All of its triggers and settings have been 'de-tuned' as much as possible without seriusly degrading performance.

I have recently acquired ts2000i, and plan to do all my system writing, backtesting and optimising with this software.

say you were using a day period or intraday period mechanical trading system for example, which you had back-tested over 1,2 or maybe even 3 years. How often would you review the parameters (and optimise the parameters if necessary)? does on a monthly basis sound sensible?

And if paying attention to the most recent past is important. If performing monthly review backtests/optimisations, over how far backs worth of data would you perform the monthly review backtests/optimisations?

Would any of these considerations be affected by the length of the intraday timeframes that you traded within? if so, how?

do you think it is possible to trade basically the same mechanical day or intraday system, using the same indicators, for years on end, with the only alterations that you do make being periodical reviews/optimisations of the included 1 or 2 indicators parameter settings?

Many thanks

jtrader.

 

[mr_cassandra]

General info post

This is just to clarify the current 'non-commercial' stayus of my trading program, called the mvp signal system.

I did have it marketed at collective2.com for several weeks but decided to discontinue it (after its first successful paying trade by the way) in order to simplify my life. One major consideration was the low level of income versus the money I was making on my own.

IE better just to invest in myself.

I'm only stressing this because some boards can be very serious about their TOS and have no wish for unnecessary discussion on this issue.

 

[TWI]

You mention on your website that you will e-mail code of an earlier version. Would you be prepared to post it here?

 

[mr_cassandra]

sorry for delayed response

You mention on your website that you will e-mail code of an earlier version. Would you be prepared to post it here?

The code is in excel with many columns and if/and/or logic referring back and forth. I'm not sure how well it would do if posted here but am open to suggestions.

Its very out of date compared to version 21, but I feel it supplies enough detail for people to get the gist of what I do.

regards, Steve

 

[mr_cassandra]

just posting to activate my thread subscription+

This is just to clarify the current 'non-commercial' stayus of my trading program, called the mvp signal system.

I did have it marketed at collective2.com for several weeks but decided to discontinue it (after its first successful paying trade by the way) in order to simplify my life. One major consideration was the low level of income versus the money I was making on my own.

IE better just to invest in myself.

I'm only stressing this because some boards can be very serious about their TOS and have no wish for unnecessary discussion on this issue.

eom

 

[JTrader]

Optimising V's forward optimising mechanical trading strategies

Hi

does anybody have any methods of "forward optimising" mechanical trading strategies that you use and are prepared to share, methods that you have found work better than basic optimising of historical data over a set period of time?

By basic optimisation I mean - optimising a strategy with perhaps two years of historical data - as one optimisation.

Forward optimising would somehow involve splitting the data into chunks. So for example - if you had for years of historic data - you may backtest and optimise the 2001 data first, then apply the optimised strategy parameters to the 2002 data - as if "peeking into the future" - and so forth.

I have just read an article about optimisation in this months Traders magazine - which has got me interested.

If you do do any optimising other than basic optimising - what exactly do you do?

what software do you use to do this?

And how do you feel your MTS/s benefits from forward optimisation when applied in real-time?


Many thanks

jtrader.

 

[mr_cassandra]

sory no help here

I wrote my own program and didn't use the commercial stuff. Its all in excel and any change in code causes a change in the whole sheet of 8.5 years data so its easy to see whether it was an improvement or a one shot wonder.
I've been using it realtime, realcash since Feb.2004 and results have continued as programmed.
My work can be found by googling mvp signal system

Hi

does anybody have any "forward optimising" strategies that you use and are prepared to share that you have found work better than basic optimising of historical data over a set period of time?

By basic optimisation I mean - optimising a strategy with perhaps two years of historical data - as one optimisation. Forward optimising would somehow involve breaking the data into chunks.

I have just read an article about optimisation in this months Traders magazine - which has got me interested.

If you do do any optimising other than basic optimising - what exactly do you do?

what software do you use to do this?

And how do you feel your MTS/s benefits from forward optimisation when applied in real-time?


Many thanks

jtrader.

 

[the blades]

I wrote my own program and didn't use the commercial stuff. Its all in excel and any change in code causes a change in the whole sheet of 8.5 years data so its easy to see whether it was an improvement or a one shot wonder.
I've been using it realtime, realcash since Feb.2004 and results have continued as programmed.
My work can be found by googling mvp signal system

I received (with thanks) a copy of an older version of your system and promised to give you feedback. As yet, I havent had time to give is a good appraisal - mainly due to the depth of the spreadsheet - very impressive!
I intend to fully appraise it in the new year when I go part time. However, I did wonder if you'd tried the system on any other markets?

UTB

 

[mr_cassandra]

re mvp system and other markets

Mvp makes its decisions based on indicators which look at the usa nasdaq and nyse markets. As such I’ve never looked at other worl markets but only other usa indexes.

I’ve loaded in the nasdaq 100, nasdaq composite and the sp500. All of them still soundly beat buy and hold but the sp400 produces superior results by far.

It is my opinion that is because the sp400 is a more resilient group of companies who try harder, stay leaner and are not fat cat complacent large cap companies, which can lose ‘the eye of the tiger’. Anyway, the more resilient the index, the more predictable it is for any mechanical system.

Regards Steve

I received (with thanks) a copy of an older version of your system and promised to give you feedback. As yet, I havent had time to give is a good appraisal - mainly due to the depth of the spreadsheet - very impressive!
I intend to fully appraise it in the new year when I go part time. However, I did wonder if you'd tried the system on any other markets?

UTB

 

[blackcab]

jtrader - what you describe is standard practice for training neural nets. You split your training data into three sets: training set, validation set and test set. Train the networks on the training data and measure the error against the (out of sample) validation set every so many generations/epochs. Stop training when the error against the validation set stabilises or increases. You use the test set after training a number of populations of networks to completion, to select the networks with the best generalisation capabilities - ie the ones that all score highly on the training set that also score highly on the (out of sample) test set.

More info here if you're interested:

Designing a neural network for forecasting financial and economic time series, Kaastra and Boyd 1995

A new evolutionary system for evolving artificial neural networks, Yao and Liu 1997

 

[mr_cassandra]

Neural nets

Am interested in this subject. Can you point me to a mechincal trading system which looks good in testing like this? (am defining good as superior return to buy and hold, acceptable drawdowns, and goes both mong and short)

jtrader - what you describe is standard practice for training neural nets. You split your training data into three sets: training set, validation set and test set. Train the networks on the training data and measure the error against the (out of sample) validation set every so many generations/epochs. Stop training when the error against the validation set stabilises or increases. You use the test set after training a number of populations of networks to completion, to select the networks with the best generalisation capabilities - ie the ones that all score highly on the training set that also score highly on the (out of sample) test set.

More info here if you're interested:

Designing a neural network for forecasting financial and economic time series, Kaastra and Boyd 1995

A new evolutionary system for evolving artificial neural networks, Yao and Liu 1997

 

[pkfryer]

I was developing systems with TradeStation but dont use it any more as it has limited functionality built in for portfolio trading which is the direction Im doing most of my work towards. I'm currently developing my own testing application that will do all that I want. But its a lot of work.

I believe it is possible to have a simple trading system that doesn't need optimisation but I think that optimisation is very important. I achieved steady and consistent gains (in back testing) with several systems I developed, I thought that I had hit pay dirt but unfortunately the markets had changed somehow about 5 years ago. The behaviour changed so that what was an incredibly successful set of systems that would have bagged consistent profits for 10 years with minimal draw downs started flat lining, with a few gently dropping down.

These systems weren't optimised at all, as I was at the time against optimisation. But I think most systems will self destruct eventually if left unattended. These systems traded large portfolios with small trade sizes. The idea being to select a variety of industries so that bad news in a sector wouldn't cause too much damage overall. But everything changed its behaviour at about the same time.

In a way this backs up the idea of NOT optimising as a deoptimised simple trading system was consistently profitable for 10 years. But you would have to keep a close eye on it and back off if the previous largest drawdown is surpassed! But I'm wondering whether optimisation would have fixed the problem. Its something that I'm going to work on when my testers ready. I notice on the charts that price behaviour changes quite regularly, varying from quiet to aggitated and back again. I would optimise on the last three months and keep those settings for three months, or an approach similar.

Just as long as the optimisation is something that you could have actually traded and not need precognition, I dont think its a problem to incorporate optimised systems in your backtesting. Do not use optimisation to find the most profitable settings for a period and then look at the profit you could have got and say that this optimisation works. But tweak the type of optimisation, so that optimisation becomes a part of your system design, what kind of optimisation was the most effective, what intervals to optimise. Optimise for a recent period, keep to those settings until the results start to slip, then optimise again. Test that approach historically. I dont think we should throw out the baby with the bath water just because optimisation has taken on a dirty feel recently.

 

[Trader333]

In my view optimisation needs to be done with great care. As a basis, any system that is not consistently profitable without any optimisation is unlikely to yield consistent profits after optimisation.

There are a number of systems commercially available that, on closer inspection, are nothing more than optimised curve fitted approaches. They are sold for large amounts of money and then, without exception, lose money. It is interesting to see that a few of these that became available late last year started losing a lot when the Dow was bouncing between 10,000 and 10,400 and all because they were based on trend trading. Since then and without any surprise at all many have re-optimised their package which will again lose money as soon as the markets change again.

I agree with PKF that getting this right is a difficult task.


Paul

 

[triax]

This is a good thread on ET for anyone who wants to find systems that have an edge, as opposed to those which are curve-fitted to past data :

http://www.elitetrader.com/vb/showthread.php?s=&threadid=33654&perpage=6&pagenumber=1

Do a search for posts by the same guy who started the thread and you'll find all sorts of interesting stuff.

 

[mr_cassandra]

Strongly agree on backtest versus reality.

The program I've written is on its 21st version. Of those 21, only two others showed any promise and when they went from the sterile environment of the excel backtest data to real life trading, something 'unprogrammed' always came up.

The program started off as a very simple long or short position switcher. But after 4 years of work now I'd say it only came into its own after two major changes:

1. I changed it to long , cash or short and greatly increased to criteria to be met before going short.
2. I then built in code to allow it to change its mind if a trade turned out premature. IE re buy if it had sold the the index continued higher.
and re-sell if it had bought and the index continued lower.

In my opinion its this last change which got me onto the playing field successfully. Markets love to fool people and machines and unless you're system can change its mind (with rules), it can get blind0sided again and again.

I'm fortuneate to have 8.5 years data and can apply any proposed code change to the entire system and see how it did overall, having learned many years ago that a fix for a today event can implode results from years ago, IE the change must work thru bull, bear and flat markets.

Regards, Steve

I was developing systems with TradeStation but dont use it any more as it has limited functionality built in for portfolio trading which is the direction Im doing most of my work towards. I'm currently developing my own testing application that will do all that I want. But its a lot of work.

I believe it is possible to have a simple trading system that doesn't need optimisation but I think that optimisation is very important. I achieved steady and consistent gains (in back testing) with several systems I developed, I thought that I had hit pay dirt but unfortunately the markets had changed somehow about 5 years ago. The behaviour changed so that what was an incredibly successful set of systems that would have bagged consistent profits for 10 years with minimal draw downs started flat lining, with a few gently dropping down.

These systems weren't optimised at all, as I was at the time against optimisation. But I think most systems will self destruct eventually if left unattended. These systems traded large portfolios with small trade sizes. The idea being to select a variety of industries so that bad news in a sector wouldn't cause too much damage overall. But everything changed its behaviour at about the same time.

In a way this backs up the idea of NOT optimising as a deoptimised simple trading system was consistently profitable for 10 years. But you would have to keep a close eye on it and back off if the previous largest drawdown is surpassed! But I'm wondering whether optimisation would have fixed the problem. Its something that I'm going to work on when my testers ready. I notice on the charts that price behaviour changes quite regularly, varying from quiet to aggitated and back again. I would optimise on the last three months and keep those settings for three months, or an approach similar.

Just as long as the optimisation is something that you could have actually traded and not need precognition, I dont think its a problem to incorporate optimised systems in your backtesting. Do not use optimisation to find the most profitable settings for a period and then look at the profit you could have got and say that this optimisation works. But tweak the type of optimisation, so that optimisation becomes a part of your system design, what kind of optimisation was the most effective, what intervals to optimise. Optimise for a recent period, keep to those settings until the results start to slip, then optimise again. Test that approach historically. I dont think we should throw out the baby with the bath water just because optimisation has taken on a dirty feel recently.

 

[JTrader]

mr_cassandra -
a fix for a today event can implode results from years ago, IE the change must work thru bull, bear and flat markets.

This quote addresses the principles that I want to stick to. Use a large amount of historical intraday data - say 2 years per instrument - so that a curve fitted system becomes a less likely outcome. This way the system will have experienced bull, bear, ranging and flat markets and will hopefully have been profitable or had minimal drawdown within each market type - and profitable overall.


Thanks again

jtrader.

 

[pkfryer]

I think its asking a lot for a system to be profitable in all market states using fixed system components/variables. Ranging markets dont suit trend based systems and vice a versa. Also, the rally correction intervals shift.

I found that the systems I worked on worked better for rallies, resulting in a set of systems that actually only took buy signals. Buy only systems seemed to work best. As an experiment I worked on a sell only system and that profited very well but try reversing the rules to produce buy signals as well and it only broke even. Trying to make a system do well in all market environments is like trying to produce a vehicle that can fly, drive on the roads and sail on the ocean. Possible in James Bond movies but will produce poor results in markets.

I think we would be better off concentrating on systems that do one thing and do that well. And stick to using them in particular environments including logic in them that closes them down when they detect that the market is incompatible with their rules.

At best I'd have a system that doesn't lose money when the market changes and makes efficient trades when it goes in its favour (for automatic systems) for manually traded systems... Use your judgement and use the right tool for the job.