Differences in EOD historic data for LSE shares

[guibert]

I was hoping that someone could perhaps shed some light on significant discrepancies in EOD historic data obtained from various sources for FTSE100 and FTSE250 shares. The data sources were Sharescope, ADVFN, Yahoo, and Paritech.
The discrepancies are significant, e.g 16 pence difference in highs on a particular day for a 576p share, 15p difference in opening price for a 620p share etc. They do not seem to be related solely to different methods of assessing Open and Close prices, as High and Low discrepancies occur on days where neither is equal to the Open or Close.

I first noticed the anomalies when modelling a trading strategy using two different data sources, and have now done some random sampling to assess the problem. Of the four sources, only two seem close in data terms, Sharescope and ADVFN. Even then, they show some variation.

I had presumed, obviously wrongly, that all the data was ultimately sourced from the LSE, where there would be standardised definitions of Open, High, Low, and Close. As such, there would be no scope for subjective decisions on whether, for instance, High was the highest mid-point price or instead the highest Ask price during the day. This is obviously not the case judging by the data variability.

I would be interested in any views on this, particularly on which is the most reliable data source in terms of reflecting what would have happened in the real market, if for example CFDs were being traded.

Thanks

 

[FTSE Beater]

Hi Guibert

Welcome to T2W

The discrepancies are a common problem with trading UK shares. No matter what software / data source you get, you'll end up with "dud" data.
Part of the problem is the LSE itself, sending out the wrong data to the suppliers. The other part is what time you class the last trade or close price as. Sharescope take theirs at 4:30 or fairly close. ADVFN however take account of the after hours auctions that go on. Have a look at the Last action on BP's price today. The trading has gone on to 5 o'clock

I'm not sure which is the most accurate, but I've always liked sharescope's data.

Hope this helps

 

[JJL3142]

Paritech EOD is downloaded at 6:30 for UK - it seems cleaner than most

 

[guibert]

Thanks for the replies, FTSE Beater and Pervaz.
The data situation is difficult to say the least with large variability between sources. While the issue of timing and definition of Open and Close prices is obviously part of the problem, that doesn't explain the vastly different Highs and Lows from different data sources on days when neither High or Low is equal to the Open or Close. It's hard to believe that the LSE is providing different Highs and Lows to the data providers, so they must be generating their own Highs and Lows based on different definitions ? Perhaps the LSE is providing the Bid/Ask range at the maximum and minimum of the market ,and also max/ min Mid-Pt prices, and then leaving the rest up to the data suppliers?
The differences can lead to considerable variation if for example you are calculating Average True Range. E.g On a day where Sharescope and ADVFN give a True Range of 7 for an 80p value share, Paritech gives a value of 17. Yahoo may show another figure. This obviously generates a large range of outcomes when backtesting any strategy.
I guess it boils down to which source is the closest mimic to actual market conditions? I was wondering whether anyone out there has tackled the data providers or LSE about this?

 

[barjon]

I think LSE gives data guidance rules but the providers are not bound by them. For example LSE guidance on ftse says in relation to opening and closing value that no account is taken of after hours trading or pre-opening auction (thus todays opening always equals yesterdays close - no gaps). Up until May Sharescope used a provider that applied this rule and there are no gaps to be seen on the charts, but then they changed their data provider and since mid May there are now gaps all over the place (rightly in imo). Unfortunately they haven't recast the historic data yet so the chart looks very strange and is of course inconsistent pre-May to post-May in the values used.

For backtesting particularly, consistency is the key. Ask your provider what "rules" they work by (easier when you're paying for it) and choose the one that fits in with your view of the market and use that. For example, as I've indicated, my own view is that ftse gap opening is right so I'm very pleased that Sharescope have changed tack (just wish they'd recast the historic data by a year or two!!)

ps: the different providers tend to come up with different volume figures too!!

 

[Sirlosealot]

ps: the different providers tend to come up with different volume figures too!![/QUOTE]


An example of this!!

Volume for Shell for last Thursday (22/07/04)

Sharescope quotes 93.5m
Metastock (Paritech) quotes 46.763m
Synergy quotes 44.435m

Sharescope seem to be quoting double the volume for Shell quoted by both Paritech and Synergy for the last few weeks except for this Friday when they all quote around 17m!

As a result Synergy and Paritech show the last rally for Shell to be on markedly less volume than the previous rally. Sharescope's volume message is much less clear for Shell at this time. Its all very confusing for those of us trying to get a grasp on Price/Volume action. Is this all part of the LSE's effort to keep accurate info away from the public?

26/07/04
I've just been in touch with Sharescope and they have confirmed that there has been a problem with their volume figures. They are acting promptly and hope to have it sorted for tonights update. From what they were saying this will bring their values back in line with the others with any variations perhaps being due to the times the various data providers collect their data.

 

[guibert]

Having delved further into the issue of discrepancies in EOD data for LSE shares, I have reached certain initial conclusions which may make disturing reading to some :

Firstly, irrespective of the data source, anyone backtesting using OHLC for FTSE100 or FTSE250 shares should be aware that the currently recorded Highs and Lows can be very misleading. This is because since November 2003, virtually all these shares have been traded under the LSE's SETS or SETSmm systems. Prior to that, only the FTSE100, plus more liquid shares, were traded under SETS. The critical issue is that if Highs and Lows are being properly recorded under these systems, then they do not reflect the mid-point High and Low of the Bid/Ask range during the day, but instead reflect the highest and lowest 'executed trades' of the day. These can be substantially different to the range given by 'mid-point prices', and also the true range of the real market. If anyone is in doubt about this, then simply look at a few FTSE100 or 250 shares, preferably on a CFD provider or broker platform, and compare yesterday's recorded Highs and Lows with the BID/ Ask graphs for the day.
What it boils down to is that Highs and Lows, based on 'mid-point prices', when combined with spread, provide a reasonable indication of the actual market's Bid/Ask range during the day - essential for accurate backtesting. However, Highs and Lows based on 'executed trades' can provide a very distorted picture of this range. This can blow a hole through your backtesting. It also raises questions about the applicability of any technical indicators that use Highs and Lows in their makeup.

I would be interested in any other views on this. Maybe I have got it wrong?

Secondly, on the original issue of discrepancies in OHLC provided by different data providers, I have some initial conclusions re the current situation :

'Close Price' should be consistent between providers as it is provided by the LSE. This seems to be broadly the case, with the exception of Yahoo.

'Open Price' is not as straightforward. While there seem to be guidelines regarding which price to use for SETS and SETSmm shares, the choice is left down to the data provider for SEAQ and SEATS shares.

'High and Low Prices' - guidelines are fairly clearcut with 'executed prices' being used for SETS and SETSmm, and 'mid-point prices' being used for the other systems.

So, as things currently stand, there shouldn't be significant discrepancies between data providers except perhaps in relation to Open prices (providing of course they are following the official guidelines). However, that is now. The bigger problem is when looking at historical data and trying to work out whether 'mid-prices' or ''executed trades have been recorded etc. While it is reasonably easy to find out what definitions and rules are currently in force re the logging of OHLC, it is much harder to find out what was in force in say 1998 or 2000.

Again, any views or futher information on this would be very much appreciated. I'm sure that others have already been through this quagmire.

Thanks

 

[guibert]

Further to my last post regarding the problems with currently available OHLC data on FTSE100 and FTSE250 shares, does anyone know of a source of historic OHLC data where the Highs and Lows are definitely 'mid-prices' rather than 'executed trade prices' ?

Failing that, does anyone know of an affordable source of historic Best Price tick data going back 5 or more years (from which 'mid-price' data could then be extracted) ?

Hope you can help

 

[truffleman]

Hi Guibert,
Surely the data should reflect what really happened during the trading period and therefore the high trade and low trade prices are the correct ones to use even if they give apparently unusual trading ranges.

 

[DaveJB]

Truffleman,
I'd tend to agree with your view, actual trades being the ones to count - however, I wonder if that wouldn't open the door to silly price manipulation? Why you'd do it I'm not sure, but suppose you had a company witha share price of £1.00 and wanted a few column inches, so you go to your online broker and insist on buying £50 worth of shares at £14.95.....<g>

Dave

 

[theknifemac]

Truffleman,
I'd tend to agree with your view, actual trades being the ones to count - however, I wonder if that wouldn't open the door to silly price manipulation? Why you'd do it I'm not sure, but suppose you had a company witha share price of £1.00 and wanted a few column inches, so you go to your online broker and insist on buying £50 worth of shares at £14.95.....<g>

Dave

Dave it could make a difference if you had options which were priced off the minimum / maximum price a share hit during the contract life time, or if you had a option position which was very close to expiring in / out of the money. This type of thing has happened where the market was moved significantly to benefit the option position.

Stew

 

[DaveJB]

Okay,
I was thinking with my shares hat on - the original post being share related - I've never traded options, so it's entirely possible that other instruments/other rules would indeed apply. Ultimately, whatever system you use, there'll always be exceptions and what if's to affect the discussion as well... I'm actually always a little bemused by things like using the highest mid price; when shares trade at specific prices I tend to go with the idea of using what actually happened.

It's all just part of the wonderful world we live in, designed specifically to annoy small traders <g>
Dave

 

[theknifemac]

Okay,
I was thinking with my shares hat on - the original post being share related - I've never traded options, so it's entirely possible that other instruments/other rules would indeed apply. Ultimately, whatever system you use, there'll always be exceptions and what if's to affect the discussion as well... I'm actually always a little bemused by things like using the highest mid price; when shares trade at specific prices I tend to go with the idea of using what actually happened.

It's all just part of the wonderful world we live in, designed specifically to annoy small traders <g>
Dave

Dave I was giving examples of why someone might want to manipulate the price in the market affecting the high / low of the day. I agree that the concept of a highest mid etc seems very bizarre or at least not much use to me....

Cheers

Stew

 

[DaveJB]

Yep,
fine by me - I can see a variety of reasons to manipulate data, not least the one where you place a spreadbet and buy £5 of actual shares to generate a 1000 point rise (how do you work that one again...?<g>)
It's all a bit odd, I wouldn't consider myself unusually paranoid but I do wonder sometimes at how these things work in practise....

Dave

 

[henry766]

i've noticed differences too , seems local to lse