Hi Grey1. I am somewhat surprised by your question Reading your previous post my perception of you is of an astute thinker. Surely the advantage of back testing is to identify a strategy which has a higher probability of achieving success in the future.
The alternative is to support the random walkers theory. You will have to excuse I have another couple of hours of research.
The alternative is to support the random walkers theory. You will have to excuse I have another couple of hours of research.
Mr. Charts
Legendary member
- Messages
- 7,370
- Likes
- 1,200
A few years ago I started looking at US stocks for intra-day patterns and came up with several that gave me the impression that they seemed to work a high percentage of the time.
Time to analyse.
I back tested initially by looking for 200 examples and calculated the success rate in tradable moves using certain entrance and exit criteria.
I then paper traded those patterns live until I had 50 samples that again produced a high percentage hit rate.
That left me with a total of ten patterns in five groups that worked great back testing and walk forward testing.
Time to trade for real.
To my delight I've found they actually work when combined with my own entry and exit criteria, position management and triggers. What's more they are methods which occur in different market conditions at different times. Certainly some of the patterns work better in good trading environments than in bad ones, but when they don't work well, losses are very minimal or you break even or make small profits and when the do work well, they are excellent.
Most of the time they work very satisfactorily indeed.
So my original research and back testing were the pre-cursors of a trading methodology which led me to the freedom of trading for a living.
Time to analyse.
I back tested initially by looking for 200 examples and calculated the success rate in tradable moves using certain entrance and exit criteria.
I then paper traded those patterns live until I had 50 samples that again produced a high percentage hit rate.
That left me with a total of ten patterns in five groups that worked great back testing and walk forward testing.
Time to trade for real.
To my delight I've found they actually work when combined with my own entry and exit criteria, position management and triggers. What's more they are methods which occur in different market conditions at different times. Certainly some of the patterns work better in good trading environments than in bad ones, but when they don't work well, losses are very minimal or you break even or make small profits and when the do work well, they are excellent.
Most of the time they work very satisfactorily indeed.
So my original research and back testing were the pre-cursors of a trading methodology which led me to the freedom of trading for a living.
Grey1
Senior member
- Messages
- 2,190
- Likes
- 202
Zig Zag,
I will contribute later on this issue... I need to explain the strength of the link between PAST and FUTURE..
these are the questions
1) Does future resemble past?
2) Can the resemblance be modeled in an objective way ..
3) Curve fitting and stuff like that ..
4) Can statistical time series be extended into future?
I 'll be back at around 7 Pm to contribute my thoughts .
This subject is not an abstract issue .. It affects your every day trading because you all look at the past to take a position ...
I will contribute later on this issue... I need to explain the strength of the link between PAST and FUTURE..
these are the questions
1) Does future resemble past?
2) Can the resemblance be modeled in an objective way ..
3) Curve fitting and stuff like that ..
4) Can statistical time series be extended into future?
I 'll be back at around 7 Pm to contribute my thoughts .
This subject is not an abstract issue .. It affects your every day trading because you all look at the past to take a position ...
Grey1 and Mr charts.
This is something I am very interested and am currently back testing short period cycles on the ftse index.Periods of say 4 weeks or so.I am looking at bottom to top moves and the retracement and top to bottom ditto.It is time consuming but I am doing it a bit at atime.
I will be following this thread to pick up any new idea's,so please keep posting.I am particularly interested in entry signals and low risk entry set ups,so if you can post any recommended set ups I will be pleased to see them
Many thanks
This is something I am very interested and am currently back testing short period cycles on the ftse index.Periods of say 4 weeks or so.I am looking at bottom to top moves and the retracement and top to bottom ditto.It is time consuming but I am doing it a bit at atime.
I will be following this thread to pick up any new idea's,so please keep posting.I am particularly interested in entry signals and low risk entry set ups,so if you can post any recommended set ups I will be pleased to see them
Many thanks
i suspect that it is impossible to create a credible back testing system in a lifetime - since so many parameters and parameters of parameters would have to be called on
however, that does not mean that a methodoldogy cannot be realised from looking at past action - and then a workable profitable system created from realtime experience
but for sure - only someone who is already a trader can add the last part of the equation to get a system to work
however, that does not mean that a methodoldogy cannot be realised from looking at past action - and then a workable profitable system created from realtime experience
but for sure - only someone who is already a trader can add the last part of the equation to get a system to work
- Messages
- 2,325
- Likes
- 17
Something I've noticed over the years: men back-test, and women forward-test. 
how can you have any confidence at all in what you are doing unless you have seen it work in the past?
Even if you trade bar by bar you have to have confidence based on the fact that you have seen certain bar combinations previously.
I agree that it is very easy to devise systems that have worked brilliantly well in the past but only because the parameters have been 'fixed' in hindsight. Thats why your system needs to be simple, with as few parameters as possible. The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move but losses money unless that move is included.
Even if you trade bar by bar you have to have confidence based on the fact that you have seen certain bar combinations previously.
I agree that it is very easy to devise systems that have worked brilliantly well in the past but only because the parameters have been 'fixed' in hindsight. Thats why your system needs to be simple, with as few parameters as possible. The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move but losses money unless that move is included.
Rognvald
Established member
- Messages
- 916
- Likes
- 15
OK Grey 1 - I'll jump in (btw were you long MSFT?)
A cogent summary of how market performance is influenced by random events and the difficulties that these place on backtesting. Their influence cannot be denied any more than their random nature can. I would suggest it is impossible to produce a backtest that factors in these items with any more than a very small chance of future success. My conclusion is that I should trade what I see with as much relevant information as I can bring to bear relative to the proposed time frame of my trading.
Backtesting is a very seductive concept as is optimisation. I have dallied with the idea of both.They are both very time consuming and the older I get the more I am aware of the decreasing time I have left. I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
For those who are interested in indicator optimisation this can now be carried out with the latest edition of Updata's TA software. They believe that this is a first. This is not a puff for UD - merely for information of those who may be interested & is a side issue to the thrust of the thread.
A cogent summary of how market performance is influenced by random events and the difficulties that these place on backtesting. Their influence cannot be denied any more than their random nature can. I would suggest it is impossible to produce a backtest that factors in these items with any more than a very small chance of future success. My conclusion is that I should trade what I see with as much relevant information as I can bring to bear relative to the proposed time frame of my trading.
Backtesting is a very seductive concept as is optimisation. I have dallied with the idea of both.They are both very time consuming and the older I get the more I am aware of the decreasing time I have left. I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
For those who are interested in indicator optimisation this can now be carried out with the latest edition of Updata's TA software. They believe that this is a first. This is not a puff for UD - merely for information of those who may be interested & is a side issue to the thrust of the thread.
Grey1
Senior member
- Messages
- 2,190
- Likes
- 202
Rog
Quote " I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
A brilliant comment.
Also . I was not long MSFT
Sidinuk
Quote "The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move "..
I liks this comment a lot , but if you did back test a system over diffenent market or parameter the reliability of the system diminished hugely.. This is why some system developers only design systems for a single market...
Can I also refer you to my post regarding the performance of a back tested system in LONG TERM..
if a poor monay managment is used on these systems the capital is vanished before the system gets the chance to show its reliability..
Either way , back testing does not offer the kind of the edge a trader needs to beat the market on a day to day bases and in a relatively short period..
Quote " I am convinced that either backtesting or optimisation can provide at best no more than a slightly better fit than random entry when applied to the future. At worst they are probably very misleading.
A brilliant comment.
Also . I was not long MSFT
Sidinuk
Quote "The system should also be profitable across a wide range of those parameters not just one particular setting that happens to catch a massive move "..
I liks this comment a lot , but if you did back test a system over diffenent market or parameter the reliability of the system diminished hugely.. This is why some system developers only design systems for a single market...
Can I also refer you to my post regarding the performance of a back tested system in LONG TERM..
if a poor monay managment is used on these systems the capital is vanished before the system gets the chance to show its reliability..
Either way , back testing does not offer the kind of the edge a trader needs to beat the market on a day to day bases and in a relatively short period..
How do you know that you have any sort of edge if you haven't seen it work in the past?
I agree there is absolutely no guarantee that anything that worked last year will work next year but surely you must gain some comfort knowing that what you are doing has at least worked in the past.
OK, you trade what you see on a daily basis, but that must correspondence to some sort of behavior that you have previously seen happen. This, surely, is a sort of backtest.
I agree there is absolutely no guarantee that anything that worked last year will work next year but surely you must gain some comfort knowing that what you are doing has at least worked in the past.
OK, you trade what you see on a daily basis, but that must correspondence to some sort of behavior that you have previously seen happen. This, surely, is a sort of backtest.
Glenn
Experienced member
- Messages
- 1,040
- Likes
- 118
Most (all ?) backtesting is done on a single chart at a time.
If this is an Equity chart then the impact of what's going on in the market or sector as a whole is not accounted for.
Problem here is to recognise market states in advance and then apply a system which works well in a given market state.
e.g. is market bullish ? If so then apply a method which works in bullish market. etc.
So how then to establish market state in advance, or at least in early stages of that state ?
A simple example: Earlier this year the uk market climbed from a major bottom, made a top and later overtook that top. It also broke a 6 month downtrend to the upside. So from that point on it was arguable bullish (imho). Therefore application of bullish systems to equities should have yielded positive returns.
Likewise the UK Bullish Percent gave a Bull Confirmed signal at the same time. (Early April), and just before, in March the FTSE had hit the 50% retracement (G1) from the all-time high.
In principle the same information could have been used to trade the major indexes (FTSE, Sectors) themselves. Same applies to US market etc.
However, given most peoples fear of markets tanking again, as usual those who got in were those who overcame that fear.
"Markets climb a wall of worry" So whilst you can see the signals, you have to be brave enough to follow them.
The above is largely longer term stuff, true. However I believe that what works at the macro level can also work at the micro level.
In respect of backtesting, I think that all the commonly known chart patterns, higher highs, lower lows, trend lines etc are quite enough to operate with and don't need backtesting. They have existed for a long time and they work most of the time under certain conditions, which is all you need. You just have to recognise those conditions as they arise in your selected timeframe.
If people want to backtest, then I believe that they should do it in the context of the market state at the time and not in isolation.
I have not seen a piece of software which does all this. They all test a single chart and ignore any outside (higher level) influences. They ignore which way the tide is running at the time, which is key imo.
For backtesting to have any value it must surely be used on discrete chunks of data over time periods which relate to certain specific market conditions; not just 1 year or 5 years or whatever but, say 13/3/2003 to 19/9/2003 - a bullish phase.
Personally I don't use backtesting in the conventional sense. It's not necessary for what I do. I know that others use it and that's fine. If it helps them, then good.
Glenn
If this is an Equity chart then the impact of what's going on in the market or sector as a whole is not accounted for.
Problem here is to recognise market states in advance and then apply a system which works well in a given market state.
e.g. is market bullish ? If so then apply a method which works in bullish market. etc.
So how then to establish market state in advance, or at least in early stages of that state ?
A simple example: Earlier this year the uk market climbed from a major bottom, made a top and later overtook that top. It also broke a 6 month downtrend to the upside. So from that point on it was arguable bullish (imho). Therefore application of bullish systems to equities should have yielded positive returns.
Likewise the UK Bullish Percent gave a Bull Confirmed signal at the same time. (Early April), and just before, in March the FTSE had hit the 50% retracement (G1) from the all-time high.
In principle the same information could have been used to trade the major indexes (FTSE, Sectors) themselves. Same applies to US market etc.
However, given most peoples fear of markets tanking again, as usual those who got in were those who overcame that fear.
"Markets climb a wall of worry" So whilst you can see the signals, you have to be brave enough to follow them.
The above is largely longer term stuff, true. However I believe that what works at the macro level can also work at the micro level.
In respect of backtesting, I think that all the commonly known chart patterns, higher highs, lower lows, trend lines etc are quite enough to operate with and don't need backtesting. They have existed for a long time and they work most of the time under certain conditions, which is all you need. You just have to recognise those conditions as they arise in your selected timeframe.
If people want to backtest, then I believe that they should do it in the context of the market state at the time and not in isolation.
I have not seen a piece of software which does all this. They all test a single chart and ignore any outside (higher level) influences. They ignore which way the tide is running at the time, which is key imo.
For backtesting to have any value it must surely be used on discrete chunks of data over time periods which relate to certain specific market conditions; not just 1 year or 5 years or whatever but, say 13/3/2003 to 19/9/2003 - a bullish phase.
Personally I don't use backtesting in the conventional sense. It's not necessary for what I do. I know that others use it and that's fine. If it helps them, then good.
Glenn
i guess the one thing that "back testing" will show without any query, is that when markets rise with volume - they are trending and tend to continue to trend upward and when markets fall with volume - they are trending and tend to keep falling - and you can see that in seconds of looking at all history - ok - back testing done! - just trade with the trend
game set and match
game set and match
Grey1
Senior member
- Messages
- 2,190
- Likes
- 202
The reason for posting this thread was to inform my fellow traders not to fall into past performance report of software vendors what so ever..
The best way to test a strategy is to test it in real time.. The system should be profitable from day first with high win/loss ratio to give the trader the feeling of DOMINATION over the market,,
To achieve that one needs to lose few quid in real time and learn fast before his capital is evaporated..
I donot know any short cuts but I can assure you by relying on backtested results u ain't gonna winnnnnnnn
The best way to test a strategy is to test it in real time.. The system should be profitable from day first with high win/loss ratio to give the trader the feeling of DOMINATION over the market,,
To achieve that one needs to lose few quid in real time and learn fast before his capital is evaporated..
I donot know any short cuts but I can assure you by relying on backtested results u ain't gonna winnnnnnnn
Grey1 said:
Mr Charts described above his use of backtesting to make money. Others of us on this board use backtesting to make money. Read a book by Jack Schwager and read the interviews with people who have made millions with backtested systems. Why say the above when it is clearly factually incorrect?
There are many ways of making money. Every individual has to play to their own strengths in order to beat the market. Saying that one way is better than another or that one way doesn't work is just silly.
Last edited:
GreyingSurfer
Active member
- Messages
- 173
- Likes
- 0
I don't accept that there is any logical difference between the application of experience and the application of backtesting. I don't think that anyone would seriously argue that trading with a clean slate (no reading, no experience, no system) is likely to end up making a profit. Once you accept that then you accept that past behaviour may help predict future behaviour, and that what has worked in the past may have a higher probability of working in the future that what hasn't worked in the past.
Once you reach that point there is no logical reason for rejecting backtesting. Of course that is a completely different issue to the question of how you back test, how you interpret and apply the results. Of course sloppy backtesting over limited data sets applied with little critical appreciation of what the results actually tell you is not going to get you very far, but that's no different from saying that uncritical application of what you may have read in a book about trading, or observed from limited trading in specific market conditions may not get you far either.
Backtesting is like any other tool in trading. Applied sensibly it can be of value. Applied badly it can lose you money. But whatever we do no one is going to make a good trader by ignoring what's happened in the past.
Once you reach that point there is no logical reason for rejecting backtesting. Of course that is a completely different issue to the question of how you back test, how you interpret and apply the results. Of course sloppy backtesting over limited data sets applied with little critical appreciation of what the results actually tell you is not going to get you very far, but that's no different from saying that uncritical application of what you may have read in a book about trading, or observed from limited trading in specific market conditions may not get you far either.
Backtesting is like any other tool in trading. Applied sensibly it can be of value. Applied badly it can lose you money. But whatever we do no one is going to make a good trader by ignoring what's happened in the past.
I don't see why back testing can be classified as worthless when it has helped so many people like myself.
Without a foundation you cannot build.
Previous data is a traders foundation- without it there is no where for them to start from.
Especially for traders with an idea for a system.
It appears to me that every so often someone starts criticizing something that they are not happy with i.e TA, charts etc.
I think it would be better if people stick to what they are comfortable with and let others do the same.
If they have a method that they think is superior and they want to share it then fine otherwise say nothing.
Without a foundation you cannot build.
Previous data is a traders foundation- without it there is no where for them to start from.
Especially for traders with an idea for a system.
It appears to me that every so often someone starts criticizing something that they are not happy with i.e TA, charts etc.
I think it would be better if people stick to what they are comfortable with and let others do the same.
If they have a method that they think is superior and they want to share it then fine otherwise say nothing.
Rognvald
Established member
- Messages
- 916
- Likes
- 15
The difficulty with backtesting is that while it is fitted to operate on the consequences of the random events that influence the market it was tested on it can never do more than start to indicate what is likely to be successful in a future which may or may not immediately (or at a later date) contain a random event or sequence of events that the backtesting could never cover as the future event or events is/are totally random.
The fact that trading based on backtesting is for a while successful is is no more conclusive than the success for a while of any other system IMHO
I don't want to engage in a protracted argument nor do I claim that my views are correct - merely that they are my opinions. We all make our own decisions based on our own views and mine are not critical of those of others
The fact that trading based on backtesting is for a while successful is is no more conclusive than the success for a while of any other system IMHO
I don't want to engage in a protracted argument nor do I claim that my views are correct - merely that they are my opinions. We all make our own decisions based on our own views and mine are not critical of those of others
Last edited:
B
Baker2bravo
If backtesting does work, then I wouldn't expect perfection.
A reasonable probability of success is good enough.
Random events, black swans etc are why people use stops presumably.
A reasonable probability of success is good enough.
Random events, black swans etc are why people use stops presumably.
