Trading System Evaluation

[Ed James]

I am interested to hear people's views on what they look for when evaluating a trading system and what might be considered to constitute a legitimate body of evidence for the performance of such a system. I've seen on various postings the suggestion that there should be a standard format for presenting the results of a trading system whether those results are from back testing or actual use. What should that standard format contain ?

Goldline for example has a published history of all of its bets but (amoung other things) it doesn't seem to include the effect of the spread or fees charged for making stop losses compulsory in its calculation of profit and loss.

Also I've seen it suggested that systems without an introductory trial period should probably be avoided.

If we could have a thread which brings together some basic guidelines as to what to look for and what criteria to apply then that would be very useful to those of us new to evaluating systems.

 

[donaldduke]

After many years of research on my own trading systems ive come to conclusion that you should
do atleast 5 and ideally 10 years backtesting. If your systems trades the e-minis then you
should go futher back in your backtesting and use the pit contracts.

Ideally the backtest should include atleast a 1000 trades.

If you are happy with the performace (return/drawdown ratios etc) of the system over that period of time then trade it/buy it.

Dont buy a system based on just a few months or a few years back testing.
Even if the system shows more than a few years of back testing dont buy if the total number of
trades shown in the back test is less than 300.

 

[Markus_H]

After many years of research on my own trading systems ive come to conclusion that you should
do atleast 5 and ideally 10 years backtesting. If your systems trades the e-minis then you
should go futher back in your backtesting and use the pit contracts.

I disagree with this statement because trading the e-mini or trading the pit contracts are two completely different animals. Many times you will find in your backtesting that a system works very well on the pit traded contracts but not in the e-minis. Markets change characteristics: The shift of the stock day traders into the e-minis after they introduced the "Pattern Day Trader Rule" dramatically changed the e-mini markets: Suddenly it was flooded by scalpers who use to trade for a penny in the stock markets.

Another reason why markets change is technology: 5 or 10 years ago it was difficult to get live data for daytrading. Today, in the internet age, you can get these data for a few dollars per month. These days you can open an account with less than $5,000, and the daytrading margin for the e-mini S&P is sometimes as small as $500. If you wanted to trade the pit traded S&P you needed $25,000.

All these factors change the markets, and systems that performed well a few years ago don't work in these market conditions any more. That's why in my experience you should limit your backtesting to 2-5 years.

Ideally the backtest should include atleast a 1000 trades.

I agree with this statement. See
http://www.trade2win.com/boards/showpost.php?p=161667&postcount=3

If your system does not produce at least 300 trades on a certain market (e.g. e-mini S&P), then you should run it on multiple markets to get this amount of trades (e.g. e-mini S&P, e-mini NQ, e-mini DOW, FTSE, EuroStoxx, Dax, etc).

Markus

 

[stellmach]

Has anyone used GOGECKO from MX-technologies for screening and selection of trading candidates?
Is there any stats on how the recommendations have fared?

 

[donaldduke]

I disagree with this statement because trading the e-mini or trading the pit contracts are two completely different animals. Many times you will find in your backtesting that a system works very well on the pit traded contracts but not in the e-minis.
Markus

The point of doing extensive back testing was to disprove your system, if you test
with a few years e-mini and find the system works well and then test further back with
the pit contract aswell you might find the system did not perform that well in the past and
you might judge your system to be suspect. If you know of a valid reason for the
difference then fine but if your system should have worked just aswell in the past, it might
be a generic breakout system that should work across all markets then the extra testing
will be valuable to giving you an idea of the drawdowns/returns to expect.

 

[FetteredChinos]

If we could have a thread which brings together some basic guidelines as to what to look for and what criteria to apply then that would be very useful to those of us new to evaluating systems.

two points:-

A) come up with your own system. evaluating off-the-shelf systems is largely pointless. if you devise your own system, then you are aware of how it is likely to perform and are also able to trade it during periods of drawdown..

and non-coherently:-

2) one of the most important considerations when devising a system is not win rate, ave win/ave loss ratios, profit factors or any of that ********. simply the most important number to look for is the time between equity peaks. this is the most important reflection of the tradeability of a strategy. would you trade something that is in a 2 year drawdown etc?

FC

 

[FetteredChinos]

sharky, im disappointed that ******** is in the swear filter.

especially when buggeration makes it through...

 

[Tuffty]

"would you trade something that is in a 2 year drawdown"

Yes...... if a 2 yr draw down is normally to be expected. (Systems that don't trade that often would fit into this category).

 

[Markus_H]

"would you trade something that is in a 2 year drawdown"

Yes...... if a 2 yr draw down is normally to be expected. (Systems that don't trade that often would fit into this category).

You really would? Then I admire your self discipline.
I for one could psychologically not stand a 2 year drawdown; it would drive me crazy.

I am constantly trying to improve my self-discipline, therefore I would like to know:

• How do you do that?
• Have you ever traded a system through a 2-year drawdown?
• How did you overcome the temptation to drop the system?

Your thoughts are much appreciated.

Markus

 

[Roberto]

sharky, im disappointed that ******** is in the swear filter.

******** is in most people's swear-filters, whether buggeration makes it through or not.

 

[FetteredChinos]

what a ****ing joke!

 

[Tuffty]

Marcus, I haven't had to stand a 2 yr drawdown on a system so I can't answer your questions. One can do testing to see the likelihood of such a draw down so one shouldn't be suprised or start to feel nervous when it comes along. Generally I'm more worried about the depth of drawdown rather than it's length.

 

[snappyone]

Goldline trading system

You mention Goldline, I'd stay well away if I were you... Speaking from personal experience...

After over a year, I can say it hasnt worked for me and hasn't produce anything like the results claimed. It would be interesting to hear from anyoone that has been with it for the last 4 years. My guess is, that list would only include the owner. Funny that!!

I am interested to hear people's views on what they look for when evaluating a trading system and what might be considered to constitute a legitimate body of evidence for the performance of such a system. I've seen on various postings the suggestion that there should be a standard format for presenting the results of a trading system whether those results are from back testing or actual use. What should that standard format contain ?

Goldline for example has a published history of all of its bets but (amoung other things) it doesn't seem to include the effect of the spread or fees charged for making stop losses compulsory in its calculation of profit and loss.

Also I've seen it suggested that systems without an introductory trial period should probably be avoided.

If we could have a thread which brings together some basic guidelines as to what to look for and what criteria to apply then that would be very useful to those of us new to evaluating systems.