Backtesting With eSignal

Simon

Active member
Messages
133
Likes
1
The text below is copied from a PM I recently sent to mmillar, followed by his response. This should prove useful to anyone contemplating what software to utilise for backtesting their strategies.



quote:
--------------------------------------------------------------------------------

Simon wrote on 09-06-2003 08:35 PM:
Martin,

With reference to your 'updated' review of eSignal, would you mind elaborating on its unsuitability for backtesting? I have recently taken out a subscription to this service & although I have no immediate need for a backtesting facility, I do anticipate having a requirement for it in the future.

Furthermore, I noticed in your 'profile' that you have previously backtested strategies with eSignal, yet why no longer? Has the service been reduced/downgraded, are the strategies you are studying too advanced to be tested on eSignal, etc.? Your thoughts would be greatly appreciated.

Many-thanks in anticipation.

Regards,

Simon




Re: eSignal
Hi Simon,

Sorry I didn't get back to you sooner - I've been away for a few days. This is a long one.

eSignal is a great real time data feed and a great charting package - in fact I think it is the best - but backtesting is just a big no-no. I wasted several months learning efs and backtesting strategies before I realised what the true picture was. I am now seriously pissed off with eSignal. I am still using them for real time data but will probably switch to TradeStation as soon as they release their next version.

1. You can currently only backtest on 60 days of historical data. This is shortly to be increased to 100 days. This is nowhere near enough for backtesting. eSignal's plan is to eventually get to 2 years of backtest data but they wont buy data in, and they only started collecting data in Feb 2003, so it will be 2005 at the earliest before this 2 years of data becomes available. Compare this with TradeStation 7.x which currently allows 10/20 years of backtesting, or TradeStation 2000i which allows unlimited backtesting.

2. Time Zone problems. I am sure you are aware of the general issue about changes in Time Zones. In the UK we change from GMT to BST (and back again) a week before the US make their equivalent change. It means that instead of the US opening at 2.30pm UK time it opens during the changeover week at 3.30pm. You can have problems if you have time conditions around your trading such as ‘Don’t trade before 3.00pm’ or ‘Close at 8.30pm’. Data providers are supposed to normalize this data so you don’t get these problems but eSignal don’t. It means for two weeks per year eSignal will produce rubbish backtesting results.

3. Data holes. If there is data missing from an instrument, for whatever reason, it is ‘technically impossible’ to backfill that data. As an example on the EuroStoxx50 (ET M3-DT) on the morning of Monday 19th May 2003 there is 40 mins of data missing. I asked why this was and if eSignal could backfill it. They told me their servers had crashed that morning – which is fair enough, it happens to the best of us – but that they could not fix it. I have no idea how often this issue happens but for them not to be able to fix it is unacceptable to me.

4. 'Zero Volume Bar' problem. I regard this problem as a major issue, but eSignal seem to regard it as a minor technical glitch that they have no plans to fix. If you bring up three Advanced Charts – one for $COMPQ, one for ‘YM #F’ and one for ‘ES #F=2’, set them all to 5 min bars. Create the following function and apply it to all three charts.

function main() {
return close("$COMPQ");
}

Go back to 29th April at 3.00pm. According to the $COMPQ chart the $COMPQ closed at 1462.21. According to the YM #F chart the $COMPQ closed at 1458.69. According to the ES #F=2 chart the $COMPQ closed at 1521.27.

Although eSignal has the correct data (1462.21) they are unable to return it correctly in all instances. When you backtest one symbol against another it means the results you get are basically random!

To backtest a system and then use it for trading you have to have 100% confidence in what you have done. Because of the above issues I have zero confidence in eSignal. Maybe the issues I have are big, maybe they are small, but with no confidence in eSignal’s data I cannot have any confidence in the systems I have produced. I have had to junk all the work I did with eSignal over the past few months and am currently rewriting everything in TradeStation’s EasyLanguage.

HTH. If you have any other questions feel free to ask.

Cheers

Martin
 
eSignal

Martin

If you are staying with eSignal for the data, where are you consdiering obtaining the historical data for 2year+ backtesting? Did you go for Tradestation 7.1 or have you elected to go for Tradestation 2000i?

DaveG
 
Historica data is available for both TS7 and TS2000. The difference is that TS7 is web based and not stored on your own pc whereas for TS2000 you download the data which is stored on your pc.

Paul
 
Hi DaveG,

I'm not necessarily staying with eSignal. I'm waiting for TradeStation 7.2 to become available before I look at all my options again. If I go with TS7.2 then I will drop eSignal (don't want to pay $100 to TS AND $100+ to eSignal).

I have a copy of TradeStation 2000i which gets its data from historybank.com. However, this is pretty poor quality data as far as I can see. eSignal has very high quality data and there are a couple of products that pull eSignal data into TS2000i but you run into eSignal's 60 day limit again. I could buy data from third parties, but it may be of the same quality as historybank.com.

There are so many ways of putting this all together that it's all a bit confusing and I certainly haven't come to any conclusions yet. I can get real time data from eSignal, TS 7.2 or IB, I can backtest with TS7.2 or 2000i, I can autotrade with TS7.2 or with TS2000i->TradeBolt or DynaOrder -> IB or (in a convoluted way) with eSignal, I can get data to feed into TS2000i from historybank.com, eSignal, IB, or some other third party using various other tools, or I can forget historical data and just use TS7.2 !! Right now I' in the middle of moving house and moving country so can honestly say this isn't top of my todo list. :eek:

There are a number of other posts around this subject where you can see what others have done..

http://www.trade2win.co.uk/boards/showthread.php?s=&threadid=5504

http://www.trade2win.co.uk/boards/showthread.php?s=&threadid=5487

http://www.trade2win.co.uk/boards/showthread.php?s=&threadid=5495

When I sort this out I will let you know what I have done.

Cheers
 
Hi

Currently I am evaluating the free download from Neoticker, this seems to be quite a package. The charts and indicators are in my opinion spot on, plus there are ways to program your own indicators etc. As yet I have not tried the back testing tools but I fully intend to. Should I decide to take this package on board then I shall post a review when I have used it for a few months.

The free 30 day download is well worth a look, if you have a data feed from an approved data-feed then just plug in and play around.

Please note there is a learning curve with this package. If any body else use's this package or tries the download I would be interested in your views.

Cheers

spreadbet
 
Hmm, I have to say that I havent experienced the same problems as mmiller with the quality of data from Historybank.com and this may depend on the instruments that are being backtested. If I have found a bad tick well it can easily and quickly be edited. Occasionally I have seen what appears to be a bad tick on a US stock where the the OHLC may be something like this:

33.09, 33.14, 28.78, 33.11 The low is nearly $5 lower than the open and close and this appears as a bad tick but when watching the Level II screen I have seen this actually happen. The market is trading at around 33.10 and for some bizarre reason there is one trade on the time and sales at 28.78 This then makes the tick look wrong when in fact there was a trade at that level.


Paul
 
Paul,

I was going to ask you what you thought about historybank's data quality. I haven't looked at intra-day data but the EOD stuff has lots of errors. To be fair I'm downloading 2.5 years of data all in one go so what might be a small problem on a daily download has become magnified. I also remember now that fowkesp uses Genesis for his EOD data because of historybank errors. Just as an example on 8th May 2002 the $SPX opens at 1055.97 and closes at 10441.8. I see these kinds of spikes all the way through the data.

I'll spend some time looking at the intra-day data to see if that is better.

Cheers
 
mmiller,

I have only ever downloaded intra-day data and you are limited to 3 months at a time. The only market that I have noticed has poor quality was the FTSE future and I stopped using Historybank for that. I haven't noticed the same problem for US stocks and futures but maybe I just haven't noticed. I will be interested to see how you progress.

Good Luck


Paul
 
Martin - Trader333

Thank you for the detailed explaination over data downloads for Tradestation, I'd completely forgot about historybank.com. Having decided to ditch my current application - Updata TA and needing that additional functionality, I looked to Tradestation. I was concerned about the sources for Intraday data, not just historical but also an RT data provider, but your's and Trader333's expainations have now covered these issues. However, it's my understanding (probably formed somewhere within this forum) that TS 2000i is no longer available and I was particullaly looking for an independent standalone system.

Best Regards

DaveG
 
Dave,

You can get a second hand copy a quite a reasonable price if you search various sites that may sell it. Also with an additional piece of software you can use IB as a realtime data provider which is the most reliable I have yet seen although you cannot use IB for historical data.

Paul
 
I've done some basic checking and it looks like even when the daily OHLC data is wrong the underlying intraday data is accurate. I also haven't seen any bad daily data on individual stocks, only on indices.

Cheers
 
That would explain why I haven't noticed it as I download intra-day data for US based stocks and futures.

Cheers


Paul
 
Top