disktrading.com intraday data

When you work with data as long as I have, you can just tell when something isn't right with the source. But to be sure, I have made it my practice to run a statistical comparison on new sources using overlapping symbols I have in my archive.

I use a variety of vendors as different vendors offer unique markets and/or duration of history. No historical source data is perfect just like no real-time quote feed is 100% accurate. But you want to see a good high 90s (98%+) in reliability.

So let's start with tickdata.com as our primary source. Again, no data is perfect, but tickdata is very reliable but also very expensive. I will then run a correlation between my primary source and another source.

In the case of Disktrading, the correlation was just plain pathetic. From my past comparision of the data, it looked patched; using multiple sources over time to patch together a price history. As Sepp has said, the data is wild: too many gaps, wild price swings, and erratic volume.

The pros of Disktrading of course is price and they do offer a large selection of futures markets. The cons are accuracy and the data not being clean.

My opinion is just that, my opinion. So check it out for yourself. $20 for a single symbol isn't must to run a test.

Before selecting a vendor, you need to narrow down what you are trading: Furtures, forex, stocks, u.s., europe, asia, etc.... Every vendor covers a subset of markets, so what you trade or back-test is important in finding a vendor.

If your want my opinion, let me know what you are looking for first, and I can give you a list of places to try.

Thanks. This is good information. Here's the details of my 2 questions:

Question # 1:
I'd like to know where I can buy data for these CME Group futures: EUR, GBP, JPY, ES, YM, CL, GC, ZN. Obviously, for the same price as disktrading.com data but of a better quality, as you said in your earlier post:
Question # 2:
Thanks, this is very good and precise information. What would you say the percentage rating of the disktrading.com data reliability if you exclude volume (I don't need volume)? The reason I am so interested in knowing how good they are is that I based all of my back-testing on their data (and so far the forward-testing is doing just as well as the back-testing, which would lead me to believe their data is good).

OK sure. But this will not be a comparative test, obviously - I'd need to splash out a couple of grand at tickdata.com to do that. However I'll take your word for it that there are issues with disktrading.com's data, although whether that has any impact on any trading system I'm likely to come up with, I don't know.

One way of looking at it is that mistakes in the backtesting due to bad data will even themselves out.

Of course I'd rather have perfect data, but the money to pay for it would come out of my trading capital. Whether that's a good trade-off or not, I doubt, my account size being relatively modest.

I would not get too hung up about data quality for use with backtesting. Remember that backtesting is only ever an approximation, because you can never know what price you will get when you trade for real. There are some people who believe that all backtesting is a waste of time.

Personally I believe it does have value for two reasons: firstly it gives you an indication of the likelihood of success with the system, but secondly and more importantly, it allows you to discover conditions which you may not have thought about when designing the system.

I would endorse disktrading's data. I have used it for a couple of years. I like the price and the range of data, and as someone said earlier, you can always email the company owner and get a friendly response. As for its accuracy - no idea, but it suits my backtesting needs.

OK, I'm pretty satisfied with that point of view, not having the funds or the time to get hold of more expensive data to compare. Thanks.

It brings me back to my other question now: how are the futures continuous contracts put together? Hopefully there aren't any huge gaps at roll-over, because that would actually cause me concern.

I think I can help with that question as well. When you buy data from them, you get a long email explaining various things. Below is an extract that talks about how he puts together the continuous futures files:

Futures data in our database is in continuous format - there is one file per symbol named after the currently most active contract and containing data from the previous contracts for the time they were most active. We do this by always collecting data for the most active contract in one file and closely comparing the trading volume to the volume of the forthcoming contract. When the new contract becomes more active than the current one, we "rollover" by re-naming the file after the new monthly contract and resuming data collection for the new one, because it is now most active.

Sounds good, although there are some markets which have contract months that you wouldn't want included in the continuous contract due to lack of volume like Crude, or other markets where you wouldn't want to trade the spot month, e.g. Eurodollars where directional trading is better on the 3rd contract out.