Problems associated with using intraday mechanical trading systems in real-time

[JTrader]

Hi

I thought this topic warranted a thread of its own .

on another thread it has been pointed out to me by a more experienced mechanical systems trader, that I may come accross unexpected difficulties when trying to implement my mechanical trading system in real-time.

It was also pointed out that trying to adapt successful EOD mechanical trading systems to intraday real-time data and time frames may produce less successful backtesting results/forward results.

I am interested to here about any problems that people know of or have come up against with regard to any of the two above possibilities, or any other problems associated with implementing mechanical trading strategies in real-time.

Here is a brief description of how I intend to trade................
I classify myself as an intraday momentum trader and not a scalper - my style of intraday trading involves identifying and trading when the instrument is moving in a pattern during the day, in an attempt to perhaps buy at bottoms and sell at tops, or to follow and stay on the right side of the trend.
During sideways/flat markets my aim is to minimise entries and losing trades.
I expect to trade off between real-time 5-30 minute time interval charts when I actually start trading with a mechanical trading system.
I will not hold positions overnight - purely intraday trading.
I will be trading forex and prehaps emini futures contracts.
I do not intend to fully automate my trades with a broker at this time. I simply want my systems to produce entry and exit signals that I execute on a manual basis.
My strategies will be written in easy language using ts2000i. I will probably progress to using TS8 for real-time trading.


All feedback is very welcome. I simply want to be aware of as much as I can be - problems, warts and all - before I invest further significant amounts of time and resources to my developments of mechanical trading systems.


Thanks a lot.

jtrader.

 

[sidinuk]

In traders magazine last month there was an article about why real trading results won't match backtested results. Basically once you enter a trade you are altering the market dynamics by actually participating in the market, your order may itself affect the future behaviour of the market. It's all to do with the butterfly effect, apparently.

 

[JTrader]

Thanks sidinuk for getting the ball rolling. An interesting theory that you mention there. I can see how this effect could affect smaller, exchange based markets if trading large amounts.............


Cheers

jtrader.

 

[Doctor Leo]

The described effect of altering the market upon entrance is worth considering when you open a position about 10 million base currency if speaking about forex on "major" currencies (that is: euro, chief, cable and yen). On less liquid currencies this effect comes into force from a 1 million position maybe. This happens just because smaller positions are cleared by the broker itself and do not reach the market at all. So, suppose you trade with a leverage of 100:1, so you'll need at least 10000 base currency to open such position and assuming you obey basic money management rules this 10000 should be 1/50 of your account, so you'll have to have at least 500000 in your account to start affecting the market. And if you prefer a more safe approach, you trade with a leverage of 30:1 to 50:1 and on major currencies, so your account should be about 10 million itself. Moreover, when you come to these larger amounts, you'll face great problems with a broker, I doubt if it allows you to open and hold such position. In this case you'll have to look for a bank which really participates in forex market.
So, assuming you have an account less than 10 million. In this case you don't have to worry about affecting the market by opening your positions. You asked about the differences between backtesting and real trading. Most of them arise from instability of datafeed and occasional quotes from your broker. This effect makes sence if you trade timeframes from 1 minute to 15 minutes approx. (I assume we're still speaking of forex market). On larger timeframes this effect is not so significant.
But the main problem arises when you try to "trade within a bar" on backtesting. This means that you enter and exit the market not at Close price of the bar, but at at SOME price WITHIN a bar, be it a stop or a limit order or anything alike. You might not have a mere opportunity to enter the market at this price in real conditions (suppose it was a single tick, or your datafeed is not clear enough, or your internet connection is not fast enough, etc., etc.) and your broker simply wouldn't allow you to enter or exit the market at this price or wouldn't execute a GTC order or execute it at a price much worse, etc. So... do not try to apply an even backtested technique which uses such intra-bar trading.
From what you said I dare assume that you may want to stick to 30-minutes and 1 hour timeframes with entries and exits on closing prices. According to my investigations on forex market, 30 minutes timeframe is even more useful for an intraday trading than 1H. Usually about the middle of an hour the price movement slows don a bit and you might be quite sure that you could make your entry or exit at the right price.
All aforesaid except for the basic forex market principles description is only my humble oppinion.

 

[TheBramble]

An interesting point (switching timeframes) and one I've tested empirically, but only over a short period of time (less than a year).

I initially started out 8 years ago trading EOD (options on UK stocks he he he...) and moved right through the card to attempting to scan all 6 billion US stocks and indices and day trade them all using LII...

There's an imaginary 'line' for me: At one end is "Impatient" and at the other is "Overload/Overwhelm".

If I'm feeling 'impatient' I'll decrease my timeframe. If I'm in 'overload/overwhelm', I'll increase it.

For instance, at the moment I have a couple of time and effort intensive non-trading related projects on the go, day trading is simply not going to get the attention it needs, so I've switched to a daily chart. I use EXACTLY the same indicators and setups and trade management (which rather belatedly answers the question you ask JT).

The thing is, it doesn't have to be at such gross levels of granularity (5min OR Daily). Even if you're day trading a 5min chart normally and you know it's going to be a 'busy' day - you can choose to switch to say a 15min or even a 30min and just experience the extra breathing space you give yourself. I hasten to add, you can only do this BEFORE you enter your trades - NEVER switch timeframes whilst in a trade! (I know YOU all know this, but just in case...).

As for back-testing - I no longer do it. Waste of time for me - IMVHO. It has a tendency toward curve-fitting and over-optimisation and real results will NEVER equal BT results. Never. Which can be a negative psychological hit. So now I just forward-test. Nothing quite like the hard right edge for playing with your ideas and for those who claim this might result in waste of time (following non-starters), you can never really waste time in research on the markets.

To answer your initial questions more succinctly JT:-

Moving your EOD system to a different timeframe should present you with few mechanical difficulties and none at all code-wise.

Differences in back-test results based on timeframe shifts are meaningless.

If you're trend following I'd agree with you on minimising (staying out!) of consolidations and add that it might be worth considering buying the pullback on an upward trending market and selling the rally on a down-trending market rather than trying to buy bottoms and sell tops...

As for not scalping, it's worth considering. As long as there's volume (or tick-pressure if you're into FX) and a broad channel to play with. It's certainly worth considering if you're planning on concentrating on just a few instruments.

FX ain't my favourite (no volume and end-of-session respite to regroup), but it does have incredibly attractive volatility!

 

[Doctor Leo]

I'm sorry to say that when applying a successful daytrading system to an intraday timeframe less than 4H turns the system quite unprofitable, if not ruins it at all - on forex.

 

[MartinD]

there is nothing wrong with back testing and optimising to rapidly test and improve upon idea, but in order to avoid curve fitting it is essential that once you have completed your test that you then run the system on out of sample data - data the system hasnt seen before - this will simulate walk forward testing and give a good indication of whether the system is robust.

Other factors to consider are correctly accounting for slippage and commissions on every trade. If your system backtest cannot tolerate adding tick per side of slip for example on the ES then it is not robust.

Also a system that uses limit orders may not produce reliable results as you may miss a reasonable % of your winning trades thru not being filled as the market pulls away but you will always be filled on all of your losing trades as the mkt has to trade thru your limit every time - far better to use market orders to gurantee entry rather than try and save paying up the spread when you may miss 30-40% of your winning trades.

 

[TheBramble]

I'm sorry to say that when applying a successful daytrading system to an intraday timeframe less than 4H turns the system quite unprofitable, if not ruins it at all - on forex.

You're talking about one system you did this with - or ALL systems you've tried to do this with?

It's important to realise the 'equivalence' of switching timeframes. By that I mean the longer the timeframe chosen, the greater the absolute points/pips, the fewer the swings and the bigger the stop.

If you take an EOD system with all the absolute parameters unchanged then yes of course, you'll come a cropper.

 

[Doctor Leo]

You're talking about one system you did this with - or ALL systems you've tried to do this with?

It's important to realise the 'equivalence' of switching timeframes. By that I mean the longer the timeframe chosen, the greater the absolute points/pips, the fewer the swings and the bigger the stop.

If you take an EOD system with all the absolute parameters unchanged then yes of course, you'll come a cropper.

I tried about 7 systems, all based on different principles, and for myself I have concluded that there are day-trading systems, and intra-day trading systems, and they differ much in their ideology, not parameters.

 

[JTrader]

The Bramble -
It's important to realise the 'equivalence' of switching timeframes. By that I mean the longer the timeframe chosen, the greater the absolute points/pips, the fewer the swings and the bigger the stop.

If you take an EOD system with all the absolute parameters unchanged then yes of course, you'll come a cropper.

Thanks The Bramble for making this point clear - where perhaps I didn't. Switching such parameters from an EOD to an intraday system is something that is necessary, and I would refine through back-testing.

I think Technical Analysis is Technical Analysis - on any time frame - whether it is on a daily chart or a 20 minute chart.

Although I understand and appreciate the idea of curve-fitting back-test results - such as a daily system with 6 months worth of back-test data - I fail to comprehend how a 5-60 minute intraday system can be curve-fitted, if back-tested with 1-2 years of data.

As a quick show of hands - How many of the traders amongst us, actually trade mechanical trading systems on intraday time frames, using Tradestation or similar software to generate straightforward entry and exit signals? If you do, what are the main problems that you have stumbled upon with regard to such a trading approach/set-up, and how did you solve/accommodate this problem?

Thanks for the replies so far

jtrader.

 

[Zeos6]

Hi

I thought this topic warranted a thread of its own .

on another thread it has been pointed out to me by a more experienced mechanical systems trader, that I may come accross unexpected difficulties when trying to implement my mechanical trading system in real-time.

It was also pointed out that trying to adapt successful EOD mechanical trading systems to intraday real-time data and time frames may produce less successful backtesting results/forward results.

I am interested to here about any problems that people know of or have come up against with regard to any of the two above possibilities, or any other problems associated with implementing mechanical trading strategies in real-time.

Here is a brief description of how I intend to trade................
I classify myself as an intraday momentum trader and not a scalper - my style of intraday trading involves identifying and trading when the instrument is moving in a pattern during the day, in an attempt to perhaps buy at bottoms and sell at tops, or to follow and stay on the right side of the trend.
During sideways/flat markets my aim is to minimise entries and losing trades.
I expect to trade off between real-time 5-30 minute time interval charts when I actually start trading with a mechanical trading system.
I will not hold positions overnight - purely intraday trading.
I will be trading forex and prehaps emini futures contracts.
I do not intend to fully automate my trades with a broker at this time. I simply want my systems to produce entry and exit signals that I execute on a manual basis.
My strategies will be written in easy language using ts2000i. I will probably progress to using TS8 for real-time trading.


All feedback is very welcome. I simply want to be aware of as much as I can be - problems, warts and all - before I invest further significant amounts of time and resources to my developments of mechanical trading systems.


Thanks a lot.

jtrader.

Hi
I have done this. I have built a trading platform (still working on making it better). Lots of tricky coding involved but watch out for 2 problem areas:
1. slippage. At higher timeframes say 10 min or above slippage will kill you - especially if the trade moves against you. 5 min is better 3 min very good.
2. Tradestation limits you where you can enter/exit trades - i.e. on next bar. With a gap or limit up/down situation - you will be dead. Look at the recent yahoo stock overnight gap. You will have to hedge with options or CFDs or with a position in a different market.
Good luck
Paul