Problems with DIY trading system

[daveb1]

Hi,

After making small losses spreadbetting on shares and subsequently reading a number of trading books and articles, I began to realise that I needed a totally systematic way of trading. I need to know exactly when to enter a trade, how to size a position and when to exit a trade in order to remove as much subjectivity as possible. Also, EOD trading suits me so a system that can signal when to trade, where to place & trail a stoploss, when exit, etc would be ideal as my brain is usually fried after a days' work.

l've coded various trading systems and have then extensively back-tested them. The code can run a trading system 1000's of times with different parameter values to see what settings would have given the largest profit over a given historical period.

If the parameters are optimised over a period, say Jan'08 to Dec'09, the trading system shows substantial profits and a high success rate. However, if those optimised parameters are then used to back-test any period outside of the optimisation period e.g. May'10 back to Jan'10, the system gives very poor results. I've tried lots of combinations of optimisation and back-testing periods but the same thing happens.

In order to have any confidence in a system and stick with it after a string of losses, I really do need to convince myself that it does work. So far I haven't found anything I'm comfortable with. Has anyone had similar experiences? Any advice gratefully received!

Thanks
Dave

 

[pboyles]

I think pretty much everyone here has had a similar experience, you wont find any system that works over time simply by combining a number of indicators. In my opinion there always has to be some discretion involved.

 

[tomorton]

Get hold of Marc Rivalland on Swing Trading - his system offers almost eveything you would need and his website offers free signals.

 

[Rossini]

I disagree with pboyles. But as a suggestion, it sounds to me like your optimisation window is far too small and you're optimising using only extraordinary market conditions.

What you need in my opinion is to pick 5 or so years of data (note: this does not have to be 5 consecutive years: you are best choosing a total of 5 years that show you chosen market in a broad and varied way) then optimising over this period. Forward test over a similar amount of time to see if what you have come up with has merit.

There is a big difference between curve fitting and optimisation. At all costs try to avoid curve fitting (which in my opinion you are doing at the moment) but at the same time optimisation is essential to any system.

 

[daveb1]

I disagree with pboyles. But as a suggestion, it sounds to me like your optimisation window is far too small and you're optimising using only extraordinary market conditions.

What you need in my opinion is to pick 5 or so years of data (note: this does not have to be 5 consecutive years: you are best choosing a total of 5 years that show you chosen market in a broad and varied way) then optimising over this period. Forward test over a similar amount of time to see if what you have come up with has merit.

There is a big difference between curve fitting and optimisation. At all costs try to avoid curve fitting (which in my opinion you are doing at the moment) but at the same time optimisation is essential to any system.

Rossini,

Thanks for your suggestions. I would agree that I've been curve fitting by finding the best parameter values over a period of time. However, I'm not clear how I can optimise the system and how that differs from what I've been doing. Could you explain the difference ?

I am coming around to the idea that there's no single system of indicators that will be consistently successful over time. Having discretion involved though won't work for me at the moment and I'm sure it would lose me money. Perhaps I need to adopt a system that is flexible by applying different rules according to the market conditions although determining these conditions could be a bit subjective.

I'll check out Marc Rivallans' system. Tomorton have you had any success with it ?

Thanks
Dave

 

[amnonco]

Rossini,

Thanks for your suggestions. I would agree that I've been curve fitting by finding the best parameter values over a period of time. However, I'm not clear how I can optimise the system and how that differs from what I've been doing. Could you explain the difference ?

I am coming around to the idea that there's no single system of indicators that will be consistently successful over time. Having discretion involved though won't work for me at the moment and I'm sure it would lose me money. Perhaps I need to adopt a system that is flexible by applying different rules according to the market conditions although determining these conditions could be a bit subjective.

I'll check out Marc Rivallans' system. Tomorton have you had any success with it ?

Thanks
Dave

simply put, if your "out of sample" testing doesn't give you great results, then your system isn't robust enough.
robustness can be achieved by adding degrees of freedom (i.e droping parameters).
if at its base, your system has sound rationality, then you should find the way to "describe" it the best you can with as little parameters as you can.

 

[Rossini]

Good post amnonco.

Finding the best parameter values is not necessarily curving fitting. Optimisation is using the tools that are available to you in the most efficient way.

Instead of using the term ‘curve fitting’ let’s use the term ‘over fitting’. If you ‘over fit’ a system you are very likely to get the results you have described. You’re likely to experience this when using an insufficient testing size or a testing period where one market condition is much more prevalent over any other. This is why I suggested expanding your testing period and making sure the testing period includes all market conditions. Once this has been done out of sample testing is a good way of finding out whether what you have created is robust. If this looks to be the case then forward testing is the nest stage.

If your testing period uses a fair sample and your rules are simple (as amnonco pointed out) you have a much better chance of reproducing the kind of results that back testing showed.

 

[charliechan]

may be useful if you gave specifics on the parameters your writing about. i assume your thinking of profit targets and stops?....

i wouldnt bother with any curve fitting, optimisation etc unless you find a market that never changes. get the drift? like you mention, these methods are text book stuff, and if we could learn from reading a text book, we'd all be....(yawn).... i trust your trying to make money here, not sit an exam....

if lack of time means you must trade mechanically, go for statistics drawn from current market dynamics. eg standard deviations of immediate volatility. these should give results that fit the current market cycle rather than what the market was doing x years ago under very different driving factors.

my 2p