Hi Grey1. I am somewhat surprised by your question Reading your previous post my perception of you is of an astute thinker. Surely the advantage of back testing is to identify a strategy which has a higher probability of achieving success in the future.
The alternative is to support the random walkers theory. You will have to excuse I have another couple of hours of research.
A few years ago I started looking at US stocks for intra-day patterns and came up with several that gave me the impression that they seemed to work a high percentage of the time.
Time to analyse.
I back tested initially by looking for 200 examples and calculated the success rate in tradable moves using certain entrance and exit criteria.
I then paper traded those patterns live until I had 50 samples that again produced a high percentage hit rate.
That left me with a total of ten patterns in five groups that worked great back testing and walk forward testing.
Time to trade for real.
To my delight I've found they actually work when combined with my own entry and exit criteria, position management and triggers. What's more they are methods which occur in different market conditions at different times. Certainly some of the patterns work better in good trading environments than in bad ones, but when they don't work well, losses are very minimal or you break even or make small profits and when the do work well, they are excellent.
Most of the time they work very satisfactorily indeed.
So my original research and back testing were the pre-cursors of a trading methodology which led me to the freedom of trading for a living.
Grey1 and Mr charts.
This is something I am very interested and am currently back testing short period cycles on the ftse index.Periods of say 4 weeks or so.I am looking at bottom to top moves and the retracement and top to bottom ditto.It is time consuming but I am doing it a bit at atime.
I will be following this thread to pick up any new idea's,so please keep posting.I am particularly interested in entry signals and low risk entry set ups,so if you can post any recommended set ups I will be pleased to see them
how can you have any confidence at all in what you are doing unless you have seen it work in the past?
Even if you trade bar by bar you have to have confidence based on the fact that you have seen certain bar combinations previously.
I agree that it is very easy to devise systems that have worked brilliantly well in the past but only because the parameters have been 'fixed' in hindsight. Thats why your system needs to be simple, with as few parameters as possible. The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move but losses money unless that move is included.
OK Grey 1 - I'll jump in (btw were you long MSFT?)
A cogent summary of how market performance is influenced by random events and the difficulties that these place on backtesting. Their influence cannot be denied any more than their random nature can. I would suggest it is impossible to produce a backtest that factors in these items with any more than a very small chance of future success. My conclusion is that I should trade what I see with as much relevant information as I can bring to bear relative to the proposed time frame of my trading.
Backtesting is a very seductive concept as is optimisation. I have dallied with the idea of both.They are both very time consuming and the older I get the more I am aware of the decreasing time I have left. I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
For those who are interested in indicator optimisation this can now be carried out with the latest edition of Updata's TA software. They believe that this is a first. This is not a puff for UD - merely for information of those who may be interested & is a side issue to the thrust of the thread.
Quote " I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
A brilliant comment.
Also . I was not long MSFT
Quote "The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move "..
I liks this comment a lot , but if you did back test a system over diffenent market or parameter the reliability of the system diminished hugely.. This is why some system developers only design systems for a single market...
Can I also refer you to my post regarding the performance of a back tested system in LONG TERM..
if a poor monay managment is used on these systems the capital is vanished before the system gets the chance to show its reliability..
Either way , back testing does not offer the kind of the edge a trader needs to beat the market on a day to day bases and in a relatively short period..
Most (all ?) backtesting is done on a single chart at a time.
If this is an Equity chart then the impact of what's going on in the market or sector as a whole is not accounted for.
Problem here is to recognise market states in advance and then apply a system which works well in a given market state.
e.g. is market bullish ? If so then apply a method which works in bullish market. etc.
So how then to establish market state in advance, or at least in early stages of that state ?
A simple example: Earlier this year the uk market climbed from a major bottom, made a top and later overtook that top. It also broke a 6 month downtrend to the upside. So from that point on it was arguable bullish (imho). Therefore application of bullish systems to equities should have yielded positive returns.
Likewise the UK Bullish Percent gave a Bull Confirmed signal at the same time. (Early April), and just before, in March the FTSE had hit the 50% retracement (G1) from the all-time high.
In principle the same information could have been used to trade the major indexes (FTSE, Sectors) themselves. Same applies to US market etc.
However, given most peoples fear of markets tanking again, as usual those who got in were those who overcame that fear.
"Markets climb a wall of worry" So whilst you can see the signals, you have to be brave enough to follow them.
The above is largely longer term stuff, true. However I believe that what works at the macro level can also work at the micro level.
In respect of backtesting, I think that all the commonly known chart patterns, higher highs, lower lows, trend lines etc are quite enough to operate with and don't need backtesting. They have existed for a long time and they work most of the time under certain conditions, which is all you need. You just have to recognise those conditions as they arise in your selected timeframe.
If people want to backtest, then I believe that they should do it in the context of the market state at the time and not in isolation.
I have not seen a piece of software which does all this. They all test a single chart and ignore any outside (higher level) influences. They ignore which way the tide is running at the time, which is key imo.
For backtesting to have any value it must surely be used on discrete chunks of data over time periods which relate to certain specific market conditions; not just 1 year or 5 years or whatever but, say 13/3/2003 to 19/9/2003 - a bullish phase.
Personally I don't use backtesting in the conventional sense. It's not necessary for what I do. I know that others use it and that's fine. If it helps them, then good.
i guess the one thing that "back testing" will show without any query, is that when markets rise with volume - they are trending and tend to continue to trend upward and when markets fall with volume - they are trending and tend to keep falling - and you can see that in seconds of looking at all history - ok - back testing done! - just trade with the trend
The reason for posting this thread was to inform my fellow traders not to fall into past performance report of software vendors what so ever..
The best way to test a strategy is to test it in real time.. The system should be profitable from day first with high win/loss ratio to give the trader the feeling of DOMINATION over the market,,
To achieve that one needs to lose few quid in real time and learn fast before his capital is evaporated..
I donot know any short cuts but I can assure you by relying on backtested results u ain't gonna winnnnnnnn