back-adjusted data goes negative for HG Copper

[Adamus]

I've got 10 years worth of EOD data from CRB Trader and their software for compiling continuous history for backtesting is great, with one exception:

for copper, the back-adjusted price series is negative all the way up to 2004.

Checking out what to do on this, the best solution would be to add a constant to all the data so the whole series is lifted above zero.

But the CRB Trader software won't do this.

Does anyone recommend software for compiling continous history?

Or perhaps a different but similar volume market that doesn't have this problem? I'm already using Gold, but not Silver - thought it correlates to Gold too closely.

 

[RichardDale]

Hi Adam,

The reason for this is due to backwardation, and the way that backward adjustment works. Backwardation is the when the price of a future delivery is lower than the current (spot) delivery. When the back adjustment occurs to remove the gap associated with rolling from one delivery month to the next, it causes all historical data to be adjusted downards. Lots of these cause negative values.

Negative numbers shouldn't present an issue in your testing - as long as you are doing a points-only returns test rather attempting to use percentage returns. Futures traders work in points not percentages, because points are easily translatable into profits/losses without having to know anything else about the contract other than its point value.

Other data sources exist (such as ours) but you won't get around the negative number issue if you are using a backadjusting algorithm. You should also understand the roll algorithm too - depending upon when you roll, it will change if/when a negative number occurs. On our standard rolling dates, Crude Oil only goes negative prior to 1990. Some rolling algorithms roll on the last day of trading of the spot month which isn't indicative of when a trader would actually roll. Most traders roll before the First Notice Date.

Other data vendors have other esoteric adjustment methods (proportional adjustments) but be aware you won't be able to work out a points return if you do this.

 

[Adamus]

Hi Richard,

you're right, I shouldn't have problems with the negative values since I'm not actually using percentages. But I am having problems so I guess something has crept into my trading program that shouldn't be there.

In conversation with CRB Trader, they said that adding a constant to the price was something they were looking at but I can't expect to see any action on it any time soon.

I've not come across the roll algorithm you describe. It sounds like a kind of forward-adjustment process.

BTW how many days from expiry is the first notice date?

 

[RichardDale]

Assuming I've described it right, it's just a standard back adjustment algorithm. i.e. the current price of the series is correct and historical data before the most recent roll has been adjusted +- the roll gap (see full document describing the back adjusting method)

The first notice date varies from contract to contract. It's not applicable for cash settled futures (e.g. all index futures), all currency futures, and also not applicable to many deliverable futures where delivery can only be requested after the last trading date (e.g. NYMEX: Crude Oil, Heating Oil, RBOB Gasoline, ICE: Brent Crude, Gas Oil, LIFFE: Cocoa, Sugar). It is applicable to quite a few such as CME Group/CBOT: Gold, mini gold, Corn, Wheat, Soybeans, LIFFE: Robusta Coffee, Long Gilt.

It usually very obvious looking at individual futures contracts where the volume/open interest rapidly diminishes a week or three before FND, so a date-based rolling algorithm usually works fine (e.g. Wheat: Roll from Mar into May on or before Feb 23, roll from May into July on or before April 23 etc. Wheat's first notice day is defined as "the business day prior to the 15th calendar day of the contract month.

Many of the definitions for First Notice Date are very verbose and are not a fixed number of days before expiry/last trading day. e.g. LIFFE Feed Wheat is "First tender day, which is the second business day after Christmas for January delivery, or seventh calendar day prior to first business day of delivery month."

Hope that helps your understanding and doesn't confuse it too much!

I'm currently working on algorithms to give more control on back adjustment (roll X days before LTD, roll Y days before FND, roll when Vol and/or OI are greater in next delivery for X days etc.). However, the choice of roll dates shouldn't have that much effect on your trading system as long as you roll during the liquid times (and don't get stuck trying to take delivery of wheat!)

Cheers,
Richard.

 

[Adamus]

OK, good point re first notice date, I see I should have known that kind of thing - I don't really want to get a call one day telling me I own several tons of wheat sitting in a Chicago warehouse!

I misunderstood what you meant about 'roll algorithm' - I thought you meant there is a special algorithm - but you were using it as a general term for all continuous series compilation methods.

I checked my system through and decided that it's my old version of TradeStation which is causing the problem and just not registering the trades it should take in backtesting when the price is negative.

I'm really not sure what I'm going to do about this but I guess I'll just ditch this one market from my trading portfolio. I'm so far down the line I don't want to stop now.

From what I can tell reading CRB Trader's documentation and help files for their continuous series software, there are ways of compiling historical data that makes me wonder what on earth people are using it for. Anyway, good luck and make sure you write tests for the code