Metatrader backtests flawed

This is a discussion on Metatrader backtests flawed within the Trading Software forums, part of the Commercial category; Most of data used for backtesting is gotten from bucketshop's non executable quotes/prices as opposed to live executable prices supplied ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Sep 27, 2009, 2:20pm   #1
Joined Mar 2007
Metatrader backtests flawed

Most of data used for backtesting is gotten from bucketshop's non executable quotes/prices as opposed to live executable prices supplied by the banks.

If trader's try to execute trades at bucketshop non executable quotes , they will not be filled.The bucketshop is merely quoting the interbank rate without a spread.The bucketshop has to add its spread to the non executables prices.

Thats a problem if you need to get filled in fast trending markets.

The other major problem is the bucketshops use dealing desks ,and in the absence of a dealing desk ,automated dealer plugins are employed to carry out dirty tricks.

Dealing desks execution results in execution delay , because order has to be verified by dealing desks .This verification time delays execution and fills in fast trending markets.

If you are a serious trader ,there are big disadvantages in dealing with most metatrader brokers.The serious traders deal directly with the banks.

There are a few honest and reliable brokers using metatrader.

The data supplied on Metatrader demo accounts is flawed.Here are comparisons between live and demo accounts .A 500 pip more loss on real accounts ,the demo had 500 pip less loss.

In the attachment s demo account lost 337 pips @0.10 per pip*£1.63 , and live account lost,and live accounts lost 881 pips .These tests used same expert advisors and set ups on both live and demo accounts

The Forex Non-Dealing Desk Trader

As a result of the above information , We can deduce most of the backtesting using metatrader demo data feeds is flawed.One can not rely on it.

OILDAYTRADER
Attached Files
File Type: zip live versus demo.zip (27.3 KB, 43 views)
oildaytrader is offline Trading arcade or proprietary trading company   Reply With Quote
Old Sep 27, 2009, 10:43pm   #2
 
£10kLoser's Avatar
Joined May 2008
This is a valid point and another nail in the coffin for the farce of back testing.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...
£10kLoser is offline Trading system vendor   Reply With Quote
Old Sep 28, 2009, 7:35am   #3
Joined Mar 2007
oildaytrader started this thread There are benefits of backtesting on many years of data.The backtests will show how the market behaved in different types of market conditions, and backtests will also show how system performs in different market conditions.

To use any backtests to trade with metatrader brokers is certain to result in losses.In live trading these bucketshops deliberately hit traders with slippage, slippage is not shown in backtester.

These bucketshops have virtually zero liquidity,the liquidity they claim of $ billions would be of no use ,liquidity is provided when not required and required liquidity is never available.Backtests and demo accounts show unlimited liquidity.

There are other problems like platform failure,power failure,internet connection failure and computer failure etc
oildaytrader is offline Trading arcade or proprietary trading company   Reply With Quote
Old Sep 28, 2009, 1:56pm   #4
 
Rossini's Avatar
Joined May 2008
Quote:
Originally Posted by £10kLoser View Post
This is a valid point and another nail in the coffin for the farce of back testing.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...
Quite simply, that is wrong.
__________________
Do, or do not, there is no such thing as try.
__________________
I'm easily pleased with perfection, and if trading is an art, then I'm painting my masterpiece.
__________________
If the margin between your goal and failure is too small then any shortfall will inevitably lead to failure.
It's far better to aim high because if you don't quite achieve your ambition you may very well be left with something worthwhile.

__________________
Rossini is offline   Reply With Quote
Old Sep 28, 2009, 2:01pm   #5
 
£10kLoser's Avatar
Joined May 2008
Quote:
Originally Posted by Rossini View Post
Quite simply, that is wrong.
Too simply, I would say...

Why is it wrong?
£10kLoser is offline Trading system vendor   Reply With Quote
Old Sep 28, 2009, 3:39pm   #6
 
Rossini's Avatar
Joined May 2008
Quote:
Originally Posted by £10kLoser View Post
This is a valid point and another nail in the coffin for the farce of back testing.

No its a nail in the coffin for back testing with incorrect data, using a live accounts data from someone like FXCM is going to be much more accurate.

Even if it wasn't flawed as above, back testing is based on the incorrect assumption that the market will behave tomorrow like it did yesterday. In a totally random set of figures, there will be patterns (or it wouldn't be totally random) and embryo traders seek out these patterns in the naive belief that they will repeat in the future. Obviously, markets change even if these past patterns had meaning.

How can people make this statement? Markets do not change; they go up, down and sideways. They cannot do anything else; the only thinks that can change is volatility and liquidity, both I would argue are more important factors that contribute to system profitability than whether the market is going up down or sideways. After all there is not point having a system that alerts you to potential profit making opportunities if you cannot get filled.

There have been literally millions and millions of back tests and yet not one has produced a 100% reliable system for making a profit in real life (over any length of time long enough not to be luck). Yet people still insist on doing it!

Well thatís not true either.

People still waste countless hours searching for a system that works by persistent back testing. What they are really doing is adjusting the system until it fits the curve. I know at least one person will reply by saying that you have to back test a system. Argue all you like - back testing is mostly a waste of time and has never produced anything worthwhile. There will be people who think they have found a system that works as a result of back testing but come back in six months time and admit that your account has been blown...
What you are referring to here is curve fitting and over optimization. We both agree that doing this is very likely to result in system losses. But to say that any system back testing is done to over optimize the parameters to ovoid and minimise losses is obviously a gross generalisation.
__________________
Do, or do not, there is no such thing as try.
__________________
I'm easily pleased with perfection, and if trading is an art, then I'm painting my masterpiece.
__________________
If the margin between your goal and failure is too small then any shortfall will inevitably lead to failure.
It's far better to aim high because if you don't quite achieve your ambition you may very well be left with something worthwhile.

__________________
Rossini is offline   Reply With Quote
Old Sep 28, 2009, 3:56pm   #7
 
£10kLoser's Avatar
Joined May 2008
Of course markets change. You can create a set of indicators that work well and then fail when the market changes - that can be seen frequently.

Markets change because of sentiment and so they then behave differently. What previously triggered a rise, no longer does so, for example. Chart patterns that usuually meant one thing will then mean either nothing or perhaps something else. This was very obvious after October last year and the 'credit crunch'. Most markets behaved entirely differently from when people believed in unchecked growth.

Systems and indicators that worked well in one set of conditions fail when the market changes.

If you believe that back testing has succeeding in producing a truly profitable system, then kindly reveal it to the rest of us! There have been hundreds if not thousands come and go and every one has failed to live up to its claims. Just flick through ClickBank as a starting point. You are too quick to just say 'that's not true' with no evidence...

Systems have to be continually updated to allow for market changes if they are to remain profitable. Not only do markets change but markets are different from each other. A system may work well on Forex but not gold, for example.

Last edited by £10kLoser; Sep 28, 2009 at 4:03pm.
£10kLoser is offline Trading system vendor   Reply With Quote
Old Sep 28, 2009, 4:16pm   #8
 
Rossini's Avatar
Joined May 2008
Quote:
Originally Posted by £10kLoser View Post
Of course markets change. You can create a set of indicators that work well and then fail when the market changes - that can be seen frequently.

You can, but that is probably because you are curve fitting the 'system' to certain conditions that do not last forever. Take a broader view and this becomes harder to do.

Markets change because of sentiment and so they then behave differently. What previously triggered a rise, no longer does so, for example. Chart patterns that usuually meant one thing will then mean either nothing or perhaps something else. This was very obvious after October last year and the 'credit crunch'. Most markets behaved entirely differently from when people believed in unchecked growth.

I’m sorry I see no evidence of that, to me the markets remain the same post credit crunch, just more volatile.

Systems and indicators that worked well in one set of conditions fail when the market changes.

If you believe that back testing has succeeding in producing a truly profitable system, then kindly reveal it to the rest of us! There have been hundreds if not thousands come and go and every one has failed to live up to its claims. Just flick through ClickBank as a starting point. You are too quick to just say 'that's not true' with no evidence...

You say ''thousands come and go and everyone has failed to live up to it claims’’ So what?? Thousands of mechanical systems have failed, just like thousands of traders have and will fail. Does that mean it is not possible for traders to succeed? No. Does that mean is not possible for mechanical systems to succeed? No. All it means is that you have no heard of any mechanical system succeeding, and why would you? What would be the need for someone who has such a system to show you?

Systems have to be continually updated to allow for market changes if they are to remain profitable.
Again this is just a generalisation, how can you say this if you do not know the system. Surly it is totally dependent on the system??
__________________
Do, or do not, there is no such thing as try.
__________________
I'm easily pleased with perfection, and if trading is an art, then I'm painting my masterpiece.
__________________
If the margin between your goal and failure is too small then any shortfall will inevitably lead to failure.
It's far better to aim high because if you don't quite achieve your ambition you may very well be left with something worthwhile.

__________________
Rossini is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
Intraday Systems Flawed tommog Trading Systems 21 Jul 9, 2009 10:19am
Long chart history for backtests.. dbinvest Forex 0 Dec 20, 2007 1:01pm
Bollinger Bands: Calculation flawed ? BSD Technical Analysis 23 Nov 14, 2007 8:51pm
Dow Jones Index calculation flawed Directional Indices 2 Nov 22, 2006 5:01pm

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)