Sentimental Options

This will be about my effort in testing out a sentiment index I concocted to trade IWM, the ETF for the Russell 2000. The main thrust will be to try to make sure the thing isn't curve-fitted, and maybe try a fix or two if it turns out it is.
First up I'm going to go through the risk characteristics of this system and of swing trading in general, since IMO risk is always severely underestimated.
Of course, before any of that I have to go to the mall with my wife, so, later, assuming I have any energy after watching the NY Jets despoil the Indianapolis Colts.

Well, risk was well illustrated in today's game, unfortunately. Jets were up at the half, but already you could see they were getting in trouble: right before halftime, the Colts scored on them. It was downhill all the way from there.
Sigh.

The market can stay irrational longer than you can stay solvent.
- JM Keynes

We've all seen that quote many times, but the truth of it may sometimes get lost on us. I know it was on me until the crisis, when it became vividly real.
So now, I look at just how risky whatever it is I'm about to set out on doing actually is very very thoroughly.
The best thing to do, in my mind, is to look at the movements in whatever it is you're thinking of trading, over the timeframe you're thinking of trading it, with the sign taken out. Basically, the idea is that regardless of whether that move was up or down, that amount represents your maximum risk.
This idea, btw, would only work for indexes and commodities, things that don't ever go to zero. Since stocks of individual companies do in fact go to zero, the best assumptions are:

1 - If you're long, your risk is 100%.
2 - If you're short, your risk is unlimited.

So, looking at what I'm dabbling in for future use right now, I've put together a system that, in terms of days elapsed between signals, averages 13 days. Actually, more accurately, the median is 13; the average is closer to 17. But I like the median better for this purpose.
13 elapsed days translates to 9 trading days, via this formula: 251/365 = .69, 251 being the number of trading days in a year. 13 * .69 = 8.94, which rounded off gives you 9 trading days in a typical period of 13 elapsed days.
So, how much does IWM move in 9 trading days? Well, the maximum move in that space of time was $30.13, which represented a 24% move at the time it happened. So the maximum potential loss on any amount invested in this timeframe is 24%.
I only used prices since 2004 for this test, but I figured that was fine, since it includes the crisis, when volatility went through the roof.
Basically, what that means is that I should simply assume that my risk is unlimited. A 24% loss on a single trade would freeze me in my tracks and stop me from trading for quite some time, if not permanently. Also, the median is not the longest trade: the very longest was 104 days. Imagine the maximum movement and therefore the maximum risk in that timeframe. And then there's maximum drawdown, or how much money the maximum number of losers in your system would cost you. If you took this and used the most costly loser and multiplied that by the maximum losing string, well, that would be an eye-opener too.
All of these exercises make real the truth of that statement by Keynes.
And that explains why I use options: with options, you lay out precisely what your risk and your reward will be. Which means you can define your risk to be well south of 24%, at least in terms of your account size. Limited risk/limited reward strategies have the lowest risk, of course. Some strategies still let you have an unlimited reward, but of course these strategies come at the cost of a higher defined risk.
Stop losses are ineffectual in my opinion in a swing trading system, since an opening gap could easily go right through it, trigger it, and then, to add insult to injury, the direction could reverse, the trading instrument end the day higher, and in the meantime you're stuck sitting on the maximum loss of the day.
Stop losses in swing trading are, IMO, a great way to multiply your losses very quickly.
With options you are in effect setting up a stop loss, but one that is unaffected by things like an opening gap. When you set it, you're setting it for the trade, not for whatever price the trading instrument lands on at some random point during the time you have that trade on.
So, the starting assumption is that risk, unless you control it, is unlimited on every trade. The more practical point of what amount to set for maximum risk on each trade we'll get to next.

The 80/20 rule:

Continuing the inquiry into how much IWM moves in 9 trading days, the better to know the risk of trading it, I did a frequency distribution of that movement. The graph of that looked like this:



That's the shape of a Pareto distribution. Don't get hung up too much on the math: it's just the old 80/20 rule.
In this case, we get to 80% by 6 (as in 80% of the time IWM will move six dollars or less), in between the 5 and the 8 on this graph, which as you can see I scaled on the x-axis by Fibonacci numbers.
So, one way to rationally put on a trade would be to do a spread in the direction in which you think IWM will go with a width of 6 bucks. That would mean a 600 dollar theoretical maximum reward on each trade minus the net premium, and the simplest way to do this would be to buy a vertical spread six strikes apart, where the long is closer to being at the money and the short is six strikes farther away from being at the money.
Six is a long way out, though, and the problem will be that it won't cancel much of the loss if the price moves against you. It will also rapidly lose almost all its efficiency as you get closer to expiration.
Three, half of that, is not bad, and while it does limit you to three hundred dollars minus the premium paid per spread for your maximum (very) theoretical reward, about 60% of the time IWM will move three bucks or less in nine trading days, so it's not like you'll be losing that much opportunity. And it will act as a much better hedge.
The median move is 2.6, which also rounds off to three strikes. Average is 3.98, which rounds off to four strikes. Also not bad, if you're wanting to take on a bit more risk for more reward. (This is a more rational choice for puts rather than calls: down is faster than up, which explains why puts are almost always more expensive than calls. Paradoxically, this also means that put spreads will cost less than call spreads over the same number of strikes, because puts will decline in price more slowly as they get farther out of the money. So, risk/reward is better on put spreads than on call spreads, at least theoretically.)
Let's take a real example from tonight's pricing:

IWM price: 61.80
Mar 62 call: 2.27 ask, .5 delta, .07 gamma, -.02 theta, .09 vega
Mar 65 call: .96 ask, .29 delta, .06 gamma, -.02 theta, .08 vega

If you do a spread where you're long the 62 and short the 65, you'll be pretty much flat on gamma, theta, and vega, while buying 21 deltas. Do two and you'll still be close to flat on the other greeks and have 42 deltas. And so on.
Your max risk is 2.27 - .96, or 1.31, which times 100 gets you to 131 dollars. Your maximum reward is 3 - 1.31 of course, or if you like: (3 - 2.27) plus .96, or .73 + .96, which, finally, is 1.69, or 169 dollars. Not much of a difference, you say? Well, then again, you're not having to take on a lot of risk either: 131 dollars (or 131 times the number of spreads you buy) until the third Friday of March as the absolute maximum you can lose, assuming you actually hold to expiration, which would be nuts anyway, is surely enough to let you sleep peacefully at night. And if you're relaxed about your max loss, you'll be a lot looser about doing interesting things in between to maybe goose your rewards a bit. A relaxed trader is a trader who can take the time to be prepared for whatever surprises the market offers up. A nervous trader is the opposite. Relaxed beats nervous, over time and lots of trades.
Next post, what I like to do to goose rewards a bit.

So, we interrupt this thread to introduce a bit of detail on how I traded in years gone by, and got to doing what I'm doing now.
I got serious in the mid 90s, built a primitive T/A model based mostly on moving averages, made good money, and got out in mid 1999, putting my profits into buying a house. The market at that time looked completely insane to me. Crunching the numbers, the only thing not in a bubble was gold, but I couldn't see that doing anything until the mania subsided.
In 2000, I restarted by slowly building a position in a gold mutual fund; no trading, this was investing. Tech was just beginning to implode, and all around me it was all anyone could talk about. Once I opened my mouth and said I was accumulating a position in gold, and got a look like I was from Mars, so after that I kept my trap shut.
After 2002, having made some good profits on this one, I began diversifying out, and siphoned the profits from the gold trade into a more general account, building that one up from the continuing profits on the gold money (it just wouldn't go down; even today, it still won't go down and stay down) until by the beginning of 2008 it made up most of my money. This was an actively traded account, and in this one I used a model based on my 90's model, but with more bells and whistles, and a mean-reversion assumption at the center of it, said mean-reversion being based on a lognormal distribution of prices.
From this account, in late 2007, I detached a brigade of my money to start a new account strictly for options, based on some good experiences I had collaring stocks whose earnings reports were coming up. More on that soon.
In 2008, the market went into total chaos mode, as we know, and the lognormal model I'd built fell to pieces. By Sept 2008 I'd had enough, and pulled the plug on it.
Meantime, over in options land, I deployed my light brigade of money in March on a strangle on Lehman, buying calls just above its price, and puts just below it. This was about a week before it was to report earnings. I figured that was early enough to prevent overpaying for the volatility that would get priced in once it got too close to the reporting date.
Earnings, as I recall, were to come out on Tuesday of the following week. Friday, Bear Stearns fell sharply on reports it was in serious trouble. Lehman fell sharply in sympathy, and I found myself sitting on some juicy profits by the close of trading. I had to adjust the strangle to keep it delta neutral, which did involve some cost, but hardly enough to notice when placed against the profits I had fallen into.
Saturday, we headed down to Florida to see my parents. Sunday night I got on my parents' computer to see what the news was, and saw that JP Morgan had bought Bear for two or three bucks a share. I was beside myself with joy.
Monday morning, 9:30, I was on my parents' computer again, and watched as more money than I had ever thought possible fell into my lap as Lehman cratered. During the day, I adjusted the strangle again, taking profits and making myself delta neutral for the next day, when earnings were actually going to be reported. My "light brigade" had returned from battle not only intact, but with an entire host of the enemy's men and materiel firmly in its possession.
So, the main, "diversified" account proved that besides the lognormal distribution of prices being nonsense, so too is the drivel about diversifying your account for protection: when TSHTF correlations go to 1 anyway, so it does precisely nothing to protect you from fire, brimstone, and bankruptcy.
The options account got my attention, but I knew that what happened was luck, not trading skill: remembering Bosquet, who said at the actual charge of the Light Brigade: "C’est magnifique, mais ce n’est pas la guerre: c'est de la folie ("It is magnificent, but it is not war: it is madness")."
I had stumbled on madness, and made a nice piece of money, but madness is not something you can use consistently. What I could use was the hedging properties of options to enlarge my possibilities for profit (you can even make money when the market is standing still, which is very nice) while managing risk precisely, and that was and is what I'm focused on.

Benton D Struckcheon said:

So, the main, "diversified" account proved that besides the lognormal distribution of prices being nonsense, so too is the drivel about diversifying your account for protection: when TSHTF correlations go to 1 anyway, so it does precisely nothing to protect you from fire, brimstone, and bankruptcy.:
The options account got my attention, but I knew that what happened was luck, not trading skill: remembering Bosquet, who said at the actual charge of the Light Brigade: "C’est magnifique, mais ce n’est pas la guerre: c'est de la folie ("It is magnificent, but it is not war: it is madness")."
I had stumbled on madness, and made a nice piece of money, but madness is not something you can use consistently. What I could use was the hedging properties of options to enlarge my possibilities for profit (you can even make money when the market is standing still, which is very nice) while managing risk precisely, and that was and is what I'm focused on.

Click to expand...

Excellent stuff

DionysusToast said:

Excellent stuff

Click to expand...

Many thanks!

I'm an options novice but I have certainly experienced the gap through the stop loss during swing trading mentioned in your first post.

I have toyed with the idea of getting into options with a view to placing a single options trades alongside my swing trades. The option trade would be set up effectively to be my stop loss. I have not yet had time to figure out the math behind doing that but the idea is to mitigate against the scenario you outlined.

Perhaps though - as you suggest, I would be better off simply using options to swing trade.

I do have a fair amount of research to get through by the end of this year which I believe will make me more money but I could of course move my plan around. Saying that - it's hard to plan to do something you currently know little about.

Any thoughts ?

DionysusToast said:

I'm an options novice but I have certainly experienced the gap through the stop loss during swing trading mentioned in your first post.

I have toyed with the idea of getting into options with a view to placing a single options trades alongside my swing trades. The option trade would be set up effectively to be my stop loss. I have not yet had time to figure out the math behind doing that but the idea is to mitigate against the scenario you outlined.

Perhaps though - as you suggest, I would be better off simply using options to swing trade.

I do have a fair amount of research to get through by the end of this year which I believe will make me more money but I could of course move my plan around. Saying that - it's hard to plan to do something you currently know little about.

Any thoughts ?

Click to expand...

Options on individual stocks are not usually efficient. As noted above, I stick with ETFs because their options trade in dollar increments with very tight bid/ask spreads. Can't say the same for options on stocks.
There are a few exceptions, such as NYX, for instance (which I'm actually thinking I might add). Mostly not a good idea though.
I'm comfortable with doing sectors rather than stocks. A matter of taste I suppose.

Benton D Struckcheon said:

Options on individual stocks are not usually efficient. As noted above, I stick with ETFs because their options trade in dollar increments with very tight bid/ask spreads. Can't say the same for options on stocks.
There are a few exceptions, such as NYX, for instance (which I'm actually thinking I might add). Mostly not a good idea though.
I'm comfortable with doing sectors rather than stocks. A matter of taste I suppose.

Click to expand...

...which brings up the question of what I was doing in Lehman back then. I was still learning options, and so hadn't yet really understood just what that bid/ask could do to you, not to mention the inconvenience of the usual five-dollar increment. I'd never really thought much about the spread with stocks, since the spread there is usually so thin.
Also, I'd never do a strangle today. I didn't really appreciate how much that strategy depended on high volatility to get away with it profitably. I do now. I'd do a short iron condor if I really thought a stock would move a bunch on earnings day these days, if I were doing stocks. I suppose I'd consider it if I get into NYX again.

A beautiful day. I do wish that every day were like this.
Below, how I do things intraday:



I came into the day with put verticals on GDX. Took my first profits at 9:45, got the rest out at around noon.
Had no position for a while, and then put on a tighter spread, buying 40's and selling 38's, at around 3PM, when the inevitable intraday bounce was peaking.
I use something called "Linear Regression Channel 100", with prices from yesterday and today taken into account. This is a beautiful work of genius, as it uses the linear regression line, and then draws as the extremes not a two standard deviation move, but the most extreme point that prices closed at either on the up or downside. On this chart, the most extreme move in either direction was on close of the 10:00 AM bar, the second red candle for today. It then draws the other side at an equal distance from the linear regression line, giving you the maximum range on either side of the line for price for the range that you choose. What I'll do on a day like today is take profits at first on the extreme move down, then wait to see if it gets even more extreme to take more. If not, I just wait, as I did today, until I see it hit the regression line and take the rest of my profits.
Then I wait for the inevitable bounce, and trade with the trend, but more cautiously. One of the reasons I like options is that I can precisely measure out my degree of bearishness or bullishness. I started the day with 43/39 put spreads, and ended with 40/38s. The tighter range allows both for more profit if we continue down tomorrow, and for less loss if we don't, of course, so that I don't wind up giving up all of today's profits even if it bounces all the way back to where it started the day today.
My plan for tomorrow is to buy more put spreads into any strength that emerges, since I don't think this down move in gold is over by any means. But we'll see. It all hinges on the jobs report, and then the reaction to it.
Next: how I construct my "sentiment" indexes.

This is interesting: it's an article from the BLS on their benchmark revisions for this year. Seems their birth/death model for businesses was way off last year.
In a normal recovery, new jobs are underestimated because generally new businesses start up faster than the BLS thinks, so a lot of new jobs in new businesses aren't captured. The opposite happens in a downturn. This is the opposite.
The revision is through March of last year, but they're extending its effects through December. Does that mean they'll miss more new jobs than they would otherwise? An interesting question, and one which bears on where gold goes: if the US recovery is stronger than Europe, its general direction should be down. But if the BLS masks this by underestimating jobs growth, well, that may not be something to trade on.

The Running of the Stops Down Broadway

Strange day. I'll go through the sentiment index construction later, but today was truly wild. As of now, GDX, which usually, but obviously not always, tracks gold, is down a mere 16 cents from its close two days ago. Gold, meantime, is down 40 bucks, give or take. In percent terms, gold fell 4%, GDX, a mere half a percent, and that's if you round up.
One of these two knows something the other doesn't. I guess we'll find out which one soon enough.
It looks and smells like short-covering. Somebody's stop got hit, and before anyone knew what was going on, stops were falling like Custer's soldiers at Little Big Horn. If this is true, then Monday the downtrend should resume.
My system is telling me that's all it was. No apparent change in sentiment at all between last night and tonight. But of course that's just probabilities.
Still, the busted uptrend in gold is a powerful argument against it being anything other than position squaring before the weekend that got out of hand. This is not a long-term call; we're still in that golden (!!) period after the last Fed move down and before the first Fed move up in interest rates, which is when the real money is made in this sector, so there's still fireworks ahead on the upside, IMO, but as of right now, it looks like that long-term up move is being interrupted by a spell of deep skepticism.
You could point to the liquidity being drained out of the Chinese economy by the authorities there, and the fact that the Eurozone looks worse than the US right now. Either one is a good fundamental reason to back away from this sector for the time being.
A good reason to be in, though, is that the mediocre to bad news on the jobs front in the US means we're a long way away from the Fed making a real move to drain liquidity. Absent that, the world will continue to be awash in greenbacks, and they need some place to rest when the sun sets, after all, just like the rest of us.
So, down, but I'd be surprised not if it breaks 1000, which is almost in the cards now, but if it breaks 850, which is the true and ultimate floor, having been resistance since 1981, before even I was in the markets. Old resistance, as we know, should act as support. If it doesn't, well, I hope I'm on the right side of it is all I have to say.

The US east of the Rockies suffers from a deplorable excess of weather.
The Plains and the southeast get tornadoes, all of it gets violent thunderstorms on a regular basis when it's warm, the eastern coastlines have to worry about hurricanes, while the interior gets gusty, flooding rain from these systems, in the north you can have heavy snow that then gets melted by heavy rains in the spring, causing yet more epic-sized flooding, the southern shore of Lake Erie is notorious for, in some winters, getting crazy amounts of snow from the "lake-effect": cold air passing over warmer water that causes clouds and then precipitation that falls on the southern shore; the city of Buffalo has had winters with 150 inches of snow. Finally, the northeastern coast gets, in the winter, big snowstorms with accumulations measured in feet.
As in the below. First, the view from my son's apartment in Baltimore in balmier times:



The view today:



Baltimore got two feet today, and is just shy of having the snowiest winter on record. Meantime, here in and around NYC, nothing, and except for one Dec storm, not really much snow has fallen so far this winter. Very unusual.

I figure sentiment in the markets is like the weather: high-pressure systems bring bright sunny weather, high sentiment among investors bring bright, sunny markets. The opposite happens on lows.
You can do a much better job of measuring sentiment these days, because the CBOE tracks volatility indices for all kinds of different markets:VIX for the SP500, VXO for the SP100, RVX for the Russell 2000, and the first one I used for this purpose, GVZ for gold, which is relatively new.
On top of that, now that there are ETFs on most indices, where options are traded much more heavily than on the indices themselves, you can get a better idea of what investors' mood is from volume and the put/call ratio.
This is especially easy for gold. Gold bugs are, to be delicate, somewhat bipolar in their enthusiasms, and just as I thought back when I was figuring this stuff out at first, their options trades reflect this bipolarity pretty clearly.
For gold, constructing a sentiment index out of the options data from the CBOE is pretty straightforward. I do this in three steps (these come from options on GLD, etf for gold itself):

1 - Take the sum of call volume plus total volume. This already overweights call volume, since calls would be included in total volume, but on top of that I multiply the call volume by a certain number as well. I then smooth this out through a complicated looking series of steps, but these actually just fell out commonsensically from looking at the bumpiness of the data and knowing the raw data was just not going to give good signals. This gives you two pieces of information in one index: how many calls there are relative to the total, a good measure of speculative fever, and how much options volume there is overall. I consider high volume in options bullish, because, first of all, no one has to be in gold or any other market, and second of all, even if you are, you don't have to be in the options. So, the more volume on the options, the more interest in that market there is, and all other things being equal, the more interest there is in any particular market, the better it is for prices in that market.
At first I traded strictly on this data alone, and it was quite good. But the next two below improve the performance of it all nicely.
2 - Call/put ratio. Yes, I like calls. My reasoning is that puts are more of a hedging than a speculative instrument, but if you see a lot of calls being originated, that's a pretty good indication of speculative enthusiasm.
3 - GVZ.

I add 1 and 2, then subtract 3 from it, weighting each differently. This is because 1 and 2 are bullish indicators if they're going up, while 3 is bearish if it's going up, so subtracting it switches the sign on it, of course, keeping it in tune with 1 and 2.
All of this just seemed to me to be a simple, commonsense way of measuring sentiment. It's turned out to be pretty accurate in the signals it gives, and I have been using it for quite a while now to generate steady profits.
But constructing one for the general market isn't so straightforward.
Next post will be on that.

Interesting stuff. You might want to put in a moving average crossover to attract a wider audience though ;-)

Why is it that options would have some value in terms of sentiment/prediction of future move ? As I understand it, if I buy a call option I am betting that future prices will rise. Of course, if I buy the underlying I am effectively doing the same thing. Once I am in the market I may or may not buy some more options/underlying or I may be done and my sentiment may no longer weigh on the market.

Obviously, the difference is that I may actually call the underlying, which is a buy and MAY call an uncovered seller to go and buy at the market. I am not sure how often this happens but my guess would be that most would actually just sell the option to lock in the gains and not actually call the underlying.

So - do we think that options buyers are more savvy than buyers of the underlying ? I am not against this to be honest as I think the buyers of corporate bonds are a lot more savvy that people that buy the stock (they don't use MA xovers for one).

I'd be interested to hear your thoughts on why option data is better at expressing sentiment than the underlying.

BTW - that view looks lovely.

DionysusToast said:

Interesting stuff. You might want to put in a moving average crossover to attract a wider audience though ;-)

Why is it that options would have some value in terms of sentiment/prediction of future move ? As I understand it, if I buy a call option I am betting that future prices will rise. Of course, if I buy the underlying I am effectively doing the same thing. Once I am in the market I may or may not buy some more options/underlying or I may be done and my sentiment may no longer weigh on the market.

Obviously, the difference is that I may actually call the underlying, which is a buy and MAY call an uncovered seller to go and buy at the market. I am not sure how often this happens but my guess would be that most would actually just sell the option to lock in the gains and not actually call the underlying.

So - do we think that options buyers are more savvy than buyers of the underlying ? I am not against this to be honest as I think the buyers of corporate bonds are a lot more savvy that people that buy the stock (they don't use MA xovers for one).

I'd be interested to hear your thoughts on why option data is better at expressing sentiment than the underlying.

Click to expand...

Well, with gold it made sense, and actually works well, because it's not a large market. The folks who inhabit it tend to have a strong speculative bent. You can actually see this in the options volume: whereas with most other underlyings, puts are more numerous than calls most of the time, with GLD, calls have more volume and open interest than puts the majority of the time. You look at that, and you say to yourself, speculation. Incautious speculation, at that.
Makes it a very easy to read market, sentiment-wise.
In all cases, rising options volume is bullish, IMO. Options are, well, optional, you know. If there's rising interest in the options on an underlying, that has to indicate, to me, rising interest in the underlying itself.
However, options may not measure sentiment better than action on the underlying for most other markets, as you note, and as I suddenly discovered just a week and a half ago. That was actually going to be the main point of my next post. Thanks for the segue.

As for the view, that kid has no idea how good he has it. It's his college apartment, paid for by yours truly. He has a spectacular view of a truly beautiful valley from his living room window. He's lucky he gave me some crucial assistance a while back with a math formula I dragged out of him.

Also, as you can see, my whole point was that the options buyers, in this case, anyway, are actually less savvy, not more. If they were more savvy, they'd be a lot harder to read, which also segues nicely into my post on constructing a sentiment index for the market in general.

...wherein we prove Occam's razor is still sharp.

Luck is a prepared man meeting opportunity.
- Vince Lombardi

The quote is there 'cause today was Super Bowl Sunday, and the New Orleans Saints took a couple of lucky breaks that went their way and used them to turn what had been a losing game into a winning one and trounced the Indianapolis Colts, who had been heavily favored to win.

So, back to how I put together a sentiment index for the general stock market:
I started out by adding together the VIX and the RVX: these two measure two completely different markets, so it seemed to me that to get an overall read on sentiment, you'd want both. The SP500 makes up about three-quarters of total market cap, though, so I overweighted it vs RVX, and the index that came out was pretty accurate.
I tried adding in IWM options, but this is where it got hard. A quick look at the data showed me this was NOT a plaything for speculators: most of the time puts had more action than calls, as you would expect if it was being used for hedging. Still, I tried adding it in. Results were OK, but unspectacular.
So I went on to the VIX options, and results were much better, more in line with what I got with the GLD options. Just staring at it, you could see that a lot of VIX calls were bought for speculative purposes, as volume on calls would spike at points where you could see it might make superficial sense for someone to play for a bounce in the index.
Which means there are a few unsophisticated folks playing these options, as they're not even a direct bet on the VIX, but on the futures on the VIX. While the front month might move reasonably in line with the published index, later months might, and then again, might not. Even on the front month, there's no way to count on it even going in the same direction if the move on the VIX on a given day is relatively small. Oh well.
For the final garnish, as it were, I threw in a formula I'd made my son sweat for a month to come up with. I had shown him, back in the summer, how Wilder's RSI is calculated, then told him all the different things you can get out of it, and then told him exactly what I wanted.
It was a bit of a brain tease for him: after two weeks of trying, he said it was impossible. I said no, it isn't. He went back, emerged from his room again two weeks later, and what he had was almost perfect. Had to tweak it a bit for a bug it had, but after that, when I put it on that previous index for gold, well, the difference was like night and day, really amazing.
When I threw that formula into the analysis I was doing on this one, same result: very accurate signals. So I knew I had something.
It correlates closely to the VIX, but not exactly; in particular, the period just prior to Feb 2007, when volatility first spiked just prior to the onset of the crisis, this index was moving very differently, because at that time the VIX's moves were very small, making it almost useless as a market sentiment measure. The other ingredients in the mix pretty much took over, and gave good signals in the backtest. After Feb 2007, the VIX's importance increased, and the accuracy of the signals continued to be high. I was very satisfied with that result.
Still, it's a complex construction, and that put me on my guard for curve-fitting.
Then, things got hairier still: a week and a half ago, on FOMC day, I was put even more on my guard.
That day I played for an afternoon reversal in direction. In the morning, after thinking about and executing my trades setting up for that reversal (which happened, much to my satisfaction), my mind was still restless, so I thinks to myself, "Hey! Remember that idea you had about the exchanges being maybe a good indicator of market direction? Maybe you should try that?", idea here being that if you buy the stock of one of the exchanges, you're expressing an opinion both about that stock and about the markets the exchange supports. In other words, this too could be a good indicator of sentiment about the markets, and therefore a good base for trading something like IWM.
So, having a little spare time, I downloaded prices for the four exchanges that I could think of that are publicly traded: NYX, NDAQ, CME, and ICE. Then, in the space of an hour, I constructed an index out of these prices, put a little very simple T/A on it, and voila! it was as accurate as that crazy Rube Goldberg thing I put together with the options data.

Oh noooo!!!

Hmm, thinks I, trying to stay objective about maybe losing weeks of thinking and working and testing the options one only to find I should have been looking at the stocks of the exchanges themselves. I tweaked it around a bit, and with different weightings. I overweighted NYX, and out popped an index that's very accurate for NYX. This is nice since I'm thinking of trading it again: just like the etfs, it has options in dollar increments, and those options have very tight spreads during the trading day. Then I went back and diddled around with the index for the market, added IWM to it since that's what I want to trade, and out popped an index better than the one I did with the options data. Besides adding IWM to this one, I also wound up deleting ICE, as I fooled around with it and couldn't find that it correlated at all to movements in IWM.
As a bonus, it gives signals once every six trading days, less than the nine on the options-based index, which cuts theoretical risk somewhat, and also makes it a better candidate for options trading: the shorter the time horizon the better when doing options.
So, I had now come up, without much work at all, with an index that predicted IWM's direction better than all that complicated options stuff I'd been fooling with, and with less risk to it, and better for trading options, all in one.
Sigh.
As a control, I ran the very simple T/A I was running on these new indexes on the instruments themselves: NYX and IWM. In both cases, results were barely positive, with returns about one-fifth what they are with the two indexes for each of these I came up with.
Then came the final total surprise: I added on my son's formula to the T/A, and instead of angels singing the Ode to Joy, figuratively speaking of course, the result was: nothing. Made things less accurate, actually.
It looks like all the complexity is in the construction of the different indexes for different purposes. The T/A done on the result stays simple and stupid.
So, the upshot is we'll trade IWM with very small trades with the options-based system while I continue to look at the exchange-based one, and trade NYX with very small trades with the exchange based one, as a forward-test on both of them, and that's what will be recorded in this journal.
BTW, and just to tie this long mess up, I did try coming up with something exchange based for GDX, but nothing really worked, which is a good thing as far as proving this idea has some value. Gold is a small market, and most of it is OTC, even with the etfs that now exist, so it would have been disconcerting if the exchange stocks predicted anything about gold.

Interesting stuff