Backtesting - whats the point exactly?

[wasp]

Whilst trying to discover a great mechanical system to bring in a steady money flow controlled only by my pet monkey Tony B, thus allowing me to travel the world to ponder all my unaswered questions like 'Do English speaking people from Yemen and Oman find their countries name at all amusing?', Where actually is Outer Mongolia and if 'people who move to Coventry, still speak to old friends?, I have a dilemma...

Backtesting...

Part of me and part of T2W says -

Backtest it to hell (or at least as far as it goes) and try it on other things to,

The other part of me and T2W says -

So what if it did well in '96, it is an ever evolving market and yesterday is history, anything could happen tomorrow...

So what do we all think, Just a few months back and a couple forward, or go the whole hog?

Keeping with my 'trend', herein find a poll!

Chris

 

[Trader333]

Backtesting - whats the point exactly?

None whatsoever in my view


Paul

 

[Salty Gibbon]

Yes, I would tend to go along with you there Paul.

I just can't see why some people regard it as a necessary thing to do.

 

[wasp]

Cheers chaps,

very gratifying answers, certainly saves me a lot of pointless hours. I was leaning that way (and hoping) and this will be my choice.

 

[frugi]

The only backtesting that I find to be of use is scrolling forward through a pretty recent -say last quarter at the oldest -intraday chart, or charts if indicators like Tick come into it, bar by bar, looking for my setups and asking myself honestly "What would I do here" before peeking at the next bar(s). It's like trading the hard right edge, only faster and I can see how my discretion is faring for free. This exercise is also quite useful for getting an idea of realistic, high-probability targets for partial outs. Exhaustive data mining that eventually finds a purely mechanical setup that worked in the past has little interest for me as I believe that in general they are lucky coincidences that won't work as well in the future. *flame suit on*

 

[Salty Gibbon]

The only backtesting that I find to be of use is scrolling forward through a pretty recent -say last quarter at the oldest -intraday chart, or charts if indicators like Tick come into it, bar by bar, looking for my setups and asking myself honestly "What would I do here" before peeking at the next bar(s).

Basically, this is what I do also. I find it quite sufficient.

 

[SOCRATES]

Yes, Frugi, the idea is a good one and what you say is correct. You are likely to be flamed, as I have discovered that anything which is correct is flamed, and done so by individuals who ought to either keep quiet, or if they open their beaks to speak should do so to comment intelligently or to ask politely. The idea of using a chart to scroll forward for the purpose of study is very useful to beginners.

 

[SOCRATES]

Backtesting ? Pointless excercise really...

 

[TheBramble]

While I agree with current consensus and I no longer back-test, I'm not so sure the hours (thousands of 'em!) spent back-testing in the early part of my trading development weren't in some way responsible for where I'm at right now.

Developing 'an eye' or an intuition - I don't know...

 

[turtle trader]

My backtesting has turned me from an occasionally profitable, often nervous trader, into a very relaxed consistently profitable one. I'm not at the end of my journey and don't write off the idea that i'll be able to become more discretionary in the future (although for now I don't want to).

Those who discard backtesting out of hand are assuming that the only way to trade succesfully is to do it their way. This, of course, is nonsense. Different things work for different people, and backtesting has definitely worked for me.

Simon

 

[commanderco]

If you sell trading software, backtesting gives you another product to add into your cashflow ... so yes it is a good idea.

If you are a techo who enjoys playing with trading toys ....then yes backtesting is another shiny toy.

If you trade for a living, then backtesting has no practical purpose. In fact I will stick my neck out and say that if you are relying on backtesting, then you are still not quite ready for the markets.

 

[peterpr]

Whilst trying to discover a great mechanical system to bring in a steady money flow controlled only by my pet monkey Tony B, <Snip>
Chris

There isn't a definitive answer to your question IMHO.

If you define back-testing as running a set of trade entry, management and exit rules over historical data, then of course it has some value. After all, that is exactly what you do real time with any such non-back-tested rules isn't it? If they add value to your trading you refine and develop them in the light of experience - same thing really.

The real point is, what is the back-tester looking for from back-testing? If it is a mechanical money-making system he is likely to be very disappointed. If it is a deeper understanding of how a set of rules can inter-act with price/time/volume, the better to inform his judgement and discretion, then it is unlikely to be time wasted - again IMHO.

Personally I have found many hours spent on back-testing with ESignal to be very illuminating indeed. I also find replaying historical data bar by bar and applying my 'rules' to be time well spent.

 

[Trader333]

Those who discard backtesting out of hand are assuming that the only way to trade succesfully is to do it their way.

This is not the reason that I dont use it. Like others I have gone down the route of using backtesting on a very extensive basis but have found that it did not allow for year in and year out consistently profitable methods. This is why, without exception, computer based trading systems will always ultimately fail all in my view of course.


Paul

 

[SOCRATES]

This is not the reason that I dont use it. Like others I have gone down the route of using backtesting on a very extensive basis but have found that it did not allow for year in and year out consistently profitable methods. This is why, without exception, computer based trading systems will always ultimately fail all in my view of course.


Paul

I respectfully suggest that you do not persist. This is because this is exaclty the opposite of what they want to hear. No amount of effort will clear away the fog. Fog is fog. It persists for much longer than could normally be expected.

Kind Regards.

 

[wasp]

The poll results and the thread starter both agree along with the results that backtesting does not answer lifes prayers. Fog, what fog?

 

[turtle trader]

This is not the reason that I dont use it. Like others I have gone down the route of using backtesting on a very extensive basis but have found that it did not allow for year in and year out consistently profitable methods. This is why, without exception, computer based trading systems will always ultimately fail all in my view of course.


Paul

I didn't want to suggest that a trader must backtest in order to be succesful - just that it is a valid method, and will suit some traders. There seem to be quite a few members who dismiss the concept as nonsense without taking the time to understand it properly.

If you are thinking of going down this route and feel that a more mechanical method of preparation may suit your personality, then don't be put off by the 'my way is the only way' brigade.

Simon

 

[wasp]

Not put off by the brigade at all, I still feel a few months back and a couple forward are necessary but adequate. My query was stretched more to the 6 months or, 6 years timescale. For a mechanical system you need to test it to a degree but just cause it worked/didn't work 5 years ago has no real bearing on next week. If it worked for the last month it has a higher chance over the next. And as I'm not going back in time or thinking more than 3 months ahead, my conclusion of a few back suits me down to the ground.

 

[turtle trader]

Chrisw

It depends on the type of strategy you're testing - for short term techniques, trading several times a day, then I think you're absolutely right. However, I have a few strategies that trade once a day at most, that have been consistent for several years, and which provide the backbone for my overall plan. I'm sure they won't last forever, but the fact that they've been working for 3 or 4 years, on several different instruments, gives me a lot of confidence in them. I wouldn't worry too much about their profitability prior to that sort of timescale though.

Simon

 

[ducati998]

No statement is more true and better applicable to Wall St than the famous warning of Santayana:

"Those who do not remember the past are condemned to repeat it"

Cheers d998

 

[Salty Gibbon]

Yes, but in most cases, those who do remember the past always repeat it anyway.