Varying performance with different intraday datasets

Hi there,

A system I am working on developing is performing significantly better on one set of intraday data (standard MetaQuote server data provided by IG Markets) compared to another, more realistic dataset (Alpari).

Unfortunately for me, the lacklustre performance is on the more realistic dataset...

I have tried to analyse the datasets and looked at how each candle differs on average (average, max, min). My suspicion is that the unrealistic dataset is less volatile (lower max, higher min), particularly during announcements, however I have not been able to confirm this by looking at the data.

Would you have any idea what could be the difference? Or perhaps what I should look for when a trading system backtests significantly better on one dataset compared to another?

Looking forward to any help !

To give you some context, what I'm hoping to do is a) understand what the difference between the two data sets is and b) change the trading system accordingly to also work well on the realistic data set.

Not sure I'm getting this. Surely if you're trading the IG product on their quotes, then it is 100% realistic with respect to itself - its own charts etc.. It doesn't matter what some other broker quotes (assuming IG maintain a standard relationship to the underlying instrument).

As if you're buying milk at Asda, there's not much point buying extra there when Morrisons have reduced their prices, unless Asda have too.

Sorry, didn't explain myself well enough. Both brokers trade on the same live data. Only one, however, Alpari, provides this accurate data for backtesting. IG's data is somehow skewed.

On Alpari, my backtest results match my real results quite accurately. On IG, there is a significant difference.

Free data feeds are delayed. Be sure to match each open/close with the exact time interval when backtesting, because prices showing in 10-10:05 probably take place 20 mins earlier. The two brokers may have different time delay (Alpari seems to have real time data) and that's why you got different live data. Perhaps you need a time column in your dataset.

3wnl said:

Free data feeds are delayed. Be sure to match each open/close with the exact time interval when backtesting, because prices showing in 10-10:05 probably take place 20 mins earlier. The two brokers may have different time delay (Alpari seems to have real time data) and that's why you got different live data. Perhaps you need a time column in your dataset.

Click to expand...

I see your point, and that may be. But firstly, I am getting the same live data. In fact, on IG, the live data does not match the historical data.

Secondly, my dataset already contains a time column, and apart from some slight timezone adjustments that I've made, they match (insofar that you can relate one bar of IG to one bar of Alpari -- the actual values are different, and it is this that I want to investigate).

"Same live data", then it is impossible that one is unrealistic, as stated in the first post. Your statements contradict each other. You should contact the two brokers to confirm this.
I don't see the point of comparing two same dataset in your analysis. Are you trying to find which broker is more trustworthy? To me this question is irrelevant to your work. If one offers more "realistic" data, use it. Just ignore IG's data.

3wnl said:

"Same live data", then it is impossible that one is unrealistic, as stated in the first post. Your statements contradict each other. You should contact the two brokers to confirm this.
I don't see the point of comparing two same dataset in your analysis. Are you trying to find which broker is more trustworthy? To me this question is irrelevant to your work. If one offers more "realistic" data, use it. Just ignore IG's data.

Click to expand...

Sorry, I may not have made myself clear in my original post — it is only the historical data that differs between the two brokers. Both brokers' live data is the same. However, their historical data differs. Alpari's historical data matches live data very well, whereas IG's data (coming from metaquote servers) does not match live data very well.

Unfortunately for me, my trading system performs very well on IG's historical data (unrealistic) and not so well on Alpari's (realistic). It is thus important for me to understand the difference between the two historical data sets. Hence this post.