Back testing is it of any value?

Grey1

Senior member
2,186 178
Back testing is it of any value? :?: :?: :?: :?:

I have opened this thread for those who spend hours and hours to design or even buy a trading system based on its past performance..

your views are appreciated..
 

ZigZag

Active member
123 0
Hi Grey1. I am somewhat surprised by your question Reading your previous post my perception of you is of an astute thinker. Surely the advantage of back testing is to identify a strategy which has a higher probability of achieving success in the future.
The alternative is to support the random walkers theory. You will have to excuse I have another couple of hours of research.
 

Mr. Charts

Legendary member
7,370 1,194
A few years ago I started looking at US stocks for intra-day patterns and came up with several that gave me the impression that they seemed to work a high percentage of the time.
Time to analyse.
I back tested initially by looking for 200 examples and calculated the success rate in tradable moves using certain entrance and exit criteria.
I then paper traded those patterns live until I had 50 samples that again produced a high percentage hit rate.
That left me with a total of ten patterns in five groups that worked great back testing and walk forward testing.
Time to trade for real.
To my delight I've found they actually work when combined with my own entry and exit criteria, position management and triggers. What's more they are methods which occur in different market conditions at different times. Certainly some of the patterns work better in good trading environments than in bad ones, but when they don't work well, losses are very minimal or you break even or make small profits and when the do work well, they are excellent.
Most of the time they work very satisfactorily indeed.
So my original research and back testing were the pre-cursors of a trading methodology which led me to the freedom of trading for a living.
 

Grey1

Senior member
2,186 178
Zig Zag,

I will contribute later on this issue... I need to explain the strength of the link between PAST and FUTURE..

these are the questions

1) Does future resemble past?
2) Can the resemblance be modeled in an objective way ..
3) Curve fitting and stuff like that ..
4) Can statistical time series be extended into future?

I 'll be back at around 7 Pm to contribute my thoughts .

This subject is not an abstract issue .. It affects your every day trading because you all look at the past to take a position ...
 

Fluke

Well-known member
456 1
Grey1 and Mr charts.
This is something I am very interested and am currently back testing short period cycles on the ftse index.Periods of say 4 weeks or so.I am looking at bottom to top moves and the retracement and top to bottom ditto.It is time consuming but I am doing it a bit at atime.

I will be following this thread to pick up any new idea's,so please keep posting.I am particularly interested in entry signals and low risk entry set ups,so if you can post any recommended set ups I will be pleased to see them
Many thanks
 

stevet

Established member
917 5
i suspect that it is impossible to create a credible back testing system in a lifetime - since so many parameters and parameters of parameters would have to be called on

however, that does not mean that a methodoldogy cannot be realised from looking at past action - and then a workable profitable system created from realtime experience

but for sure - only someone who is already a trader can add the last part of the equation to get a system to work
 

sidinuk

Established member
624 5
how can you have any confidence at all in what you are doing unless you have seen it work in the past?

Even if you trade bar by bar you have to have confidence based on the fact that you have seen certain bar combinations previously.

I agree that it is very easy to devise systems that have worked brilliantly well in the past but only because the parameters have been 'fixed' in hindsight. Thats why your system needs to be simple, with as few parameters as possible. The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move but losses money unless that move is included.
 

Rognvald

Established member
916 15
OK Grey 1 - I'll jump in (btw were you long MSFT?)

A cogent summary of how market performance is influenced by random events and the difficulties that these place on backtesting. Their influence cannot be denied any more than their random nature can. I would suggest it is impossible to produce a backtest that factors in these items with any more than a very small chance of future success. My conclusion is that I should trade what I see with as much relevant information as I can bring to bear relative to the proposed time frame of my trading.

Backtesting is a very seductive concept as is optimisation. I have dallied with the idea of both.They are both very time consuming and the older I get the more I am aware of the decreasing time I have left. I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.

For those who are interested in indicator optimisation this can now be carried out with the latest edition of Updata's TA software. They believe that this is a first. This is not a puff for UD - merely for information of those who may be interested & is a side issue to the thrust of the thread.
 

Grey1

Senior member
2,186 178
Rog

Quote " I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.


A brilliant comment.

Also . I was not long MSFT


Sidinuk

Quote "The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move "..

I liks this comment a lot , but if you did back test a system over diffenent market or parameter the reliability of the system diminished hugely.. This is why some system developers only design systems for a single market...

Can I also refer you to my post regarding the performance of a back tested system in LONG TERM..
if a poor monay managment is used on these systems the capital is vanished before the system gets the chance to show its reliability..

Either way , back testing does not offer the kind of the edge a trader needs to beat the market on a day to day bases and in a relatively short period..
 

sidinuk

Established member
624 5
How do you know that you have any sort of edge if you haven't seen it work in the past?

I agree there is absolutely no guarantee that anything that worked last year will work next year but surely you must gain some comfort knowing that what you are doing has at least worked in the past.

OK, you trade what you see on a daily basis, but that must correspondence to some sort of behavior that you have previously seen happen. This, surely, is a sort of backtest.
 

Glenn

Experienced member
1,040 118
Most (all ?) backtesting is done on a single chart at a time.
If this is an Equity chart then the impact of what's going on in the market or sector as a whole is not accounted for.
Problem here is to recognise market states in advance and then apply a system which works well in a given market state.
e.g. is market bullish ? If so then apply a method which works in bullish market. etc.

So how then to establish market state in advance, or at least in early stages of that state ?
A simple example: Earlier this year the uk market climbed from a major bottom, made a top and later overtook that top. It also broke a 6 month downtrend to the upside. So from that point on it was arguable bullish (imho). Therefore application of bullish systems to equities should have yielded positive returns.
Likewise the UK Bullish Percent gave a Bull Confirmed signal at the same time. (Early April), and just before, in March the FTSE had hit the 50% retracement (G1) from the all-time high.
In principle the same information could have been used to trade the major indexes (FTSE, Sectors) themselves. Same applies to US market etc.
However, given most peoples fear of markets tanking again, as usual those who got in were those who overcame that fear.
"Markets climb a wall of worry" So whilst you can see the signals, you have to be brave enough to follow them.

The above is largely longer term stuff, true. However I believe that what works at the macro level can also work at the micro level.
In respect of backtesting, I think that all the commonly known chart patterns, higher highs, lower lows, trend lines etc are quite enough to operate with and don't need backtesting. They have existed for a long time and they work most of the time under certain conditions, which is all you need. You just have to recognise those conditions as they arise in your selected timeframe.

If people want to backtest, then I believe that they should do it in the context of the market state at the time and not in isolation.
I have not seen a piece of software which does all this. They all test a single chart and ignore any outside (higher level) influences. They ignore which way the tide is running at the time, which is key imo.
For backtesting to have any value it must surely be used on discrete chunks of data over time periods which relate to certain specific market conditions; not just 1 year or 5 years or whatever but, say 13/3/2003 to 19/9/2003 - a bullish phase.

Personally I don't use backtesting in the conventional sense. It's not necessary for what I do. I know that others use it and that's fine. If it helps them, then good.
Glenn
 

stevet

Established member
917 5
i guess the one thing that "back testing" will show without any query, is that when markets rise with volume - they are trending and tend to continue to trend upward and when markets fall with volume - they are trending and tend to keep falling - and you can see that in seconds of looking at all history - ok - back testing done! - just trade with the trend

game set and match
 

Grey1

Senior member
2,186 178
The reason for posting this thread was to inform my fellow traders not to fall into past performance report of software vendors what so ever..

The best way to test a strategy is to test it in real time.. The system should be profitable from day first with high win/loss ratio to give the trader the feeling of DOMINATION over the market,,

To achieve that one needs to lose few quid in real time and learn fast before his capital is evaporated..
I donot know any short cuts but I can assure you by relying on backtested results u ain't gonna winnnnnnnn
 
 
AdBlock Detected

We get it, advertisements are annoying!

But it's thanks to our sponsors that access to Trade2Win remains free for all. By viewing our ads you help us pay our bills, so please support the site and disable your AdBlocker.

I've Disabled AdBlock