Neural net systems

[Hwyman]

As promised, I ran a few tests for large cap US equities using Neuroshell Trader's ability to build a neural network. For those of you not familiar with neural nets, they create a model based on past data where the neural net can spot non linear patterns invisible to the eye. Once a model is build (process called training), the neural net is applied to "out of sample" data, that is data never seen by the model before in the hope that the patterns learned in the past can be applied to create a trading system for a certain period out in the future. The problem is, we have a choice of thousands of input indicators and training objectives when we build these neural nets and months of trying various combinations may never produce a neural network with good predicvive capabilities. In my first example I did not use a true neural net but only neuroshell traders capability to optimize millions of combinations at a high speed (using a genetic algorythm instead of an exhaustive search. the system is optimized for a period of two yeats using daily values then applied to one year of new data. The results represent the out of sample results for LONG trades only with two different optimization criterea A (max return) and B(max #win/#lose)

Symbol: Buyand Hold % Model A % Model B
AET 59.1 7.1 3.0
AA -19.1 -9.5 -16.5
MSFT 3.4 3.5 4.7
INTC 16.1 1.9 -6

Well, results are not very encouraging. So much for bulding a system based on Fisher transforms of High/Low for several periods back.

 

[Pat494]

Good luck with the testing ( I don't understand your figures) Are you attempting EOD figures ?
I feel neural networks and AI are the way of the future and therefore of some interest. We are probably at only the second generation stage and there is plenty to come in this field.

 

[Tuffty]

I've not used neural nets before but I am interested in how I may be able to apply them to the markets. The problem I have is that I don't really understand the technology 'neural nets'. Is it just a posh way of saying 'lets look at hundreds of inputs and see if we can find any that have a strong relationship to a certain outcome?' but done fast by fuzzy learning programmes?

 

[Hwyman]

Neural Net attempt 1

For this test i used Neuroshell Trader. I opened a stock chart and inserted a neural network indicator. For each day it calculated a value between -1 and 1 based on two inputs: Hilbert period (H,L) for that day and the value for the day before. With the software i created the follwing system: " Based to the chart and indicator data available for 2003 and 2004, construct the best possible model you can. Them apply it to data for 2005" The strategy would buy when the neural indicator crosses above a certain level and sell when it crosses a different level. My optimization criterea was "maximize the ration of wins over losses". (long tradesonly). The model took about 6 minutes per stock to build a model and then apply it to the last year's worth of data. here are some of the stocks i tested:

AA.....................B&H................. - 19.5 %............Neuralnet............ -.5%
MSFT...............B&H..................... 3.4%.............Neuralnet............ 5.1%
IBM...................B&H ................... -6.7% .............Neuralnet...........15.2 %
C.......................B&H ................... 9.0% .............Neuralnet.............0.0 %
IFF...................B&H ................... -19.0% .............Neuralnet...........-1.1%
INTC.................B&H ................... 16.1% .............Neuralnet...........8.8%

Average...........B&H.....................-2.8%.............Neuralnet.............4.5 %

From my personal observation, this approach works well if the training and out of sample data is similar that is, if the stock moved sideways during training and continued to do so the results would be better than if characteristics changes from bulling to bearish and vice versa. I will try to build a better network next time.

 

[Hwyman]

Neural Net attempt 2

In this example I used Biocomp Profit 2002.
1. I loaded Open, close, high, low, volume data for the following instruments: TGT, WMT, $INDU and $VIX. Period was 1/1/2002 till 12/31/2004
2. Using Biocomp I selected TGT as my instrument which I'd like to model and WMT, $INDU and $VIX as my inputs or data that influences the stock price og TGT
3. Biocomp's first stem was to apply different transforms to the inputs and decide which ones have predictive power, I assume by was of statistical correlation
4. Next Biocomp evaluated a large number of models and selected 10 which were considered best. The result was a signal line which oscillated between -1 and 1. The default strategy is to buy and sell when the signal line crosses zero.
5. I used Biocomp to optimize the buy and sell thresholds
======================================================================
The model that was built showed the following for its evaluation period:
Model Name: System Model

Trading Start Date: 12/23/2003
Trading Stop Date: 12/31/2005
Number of winning trades: 19
Number of losing trades: 4
Percent winning trades: 82.61%
Highest trade value: 523.00
Lowest trade value: -135.00
Average trade value: 133.4348
Std Dev of trade value: 159.9178
Ave period between trades: 10.48
Lowest Acct Equity: 145.00
Max. Intraday Drawdown: -869.00
Total Equity / Max. Drawdown: 3.6582
Closed Equity: 3,069.00
Open Equity: 110.00

Perfect profit: 24,538.00
Maximum 'desired' profit: 6,980.00 (28.45% of perfect)
Model's Profit: 3,179.00 (12.96% of perfect, 45.54% of 'desired')

Percent of Buy-N-Hold: 11.04%
Significance: Not calculated
=========================================================================
The above performance is not true out of sample performance as I adjusted the buy and sell thresholds after the fact. So I saved the model at this point and closed down Biocomp.

To check the true out of sample performance, I loaded the model again, this time with data for the period 1/1/2005-12/05/2005. This is data that the model had never seen. here are the results:
Trading Start Date: 12/31/2004
Trading Stop Date: 12/31/2005
Number of winning trades: 10
Number of losing trades: 11
Percent winning trades: 47.62%
Highest trade value: 1,063.00
Lowest trade value: -515.00
Average trade value: 74.619
Std Dev of trade value: 437.1485
Ave period between trades: 10.57
Lowest Acct Equity: -75.00
Max. Intraday Drawdown: -741.00
Total Equity / Max. Drawdown: 2.0121
Closed Equity: 1,567.00
Open Equity: -76.00

Perfect profit: 27,674.00
Maximum 'desired' profit: 7,622.00 (27.54% of perfect)
Model's Profit: 1,491.00 (5.39% of perfect, 19.56% of 'desired')

Percent of Buy-N-Hold: 534.47%
Significance: Not calculated
=============================================================

I am not an expert in using Biocomp but this simple model easily produced a return of 16 % compared to a buy and hold of les than 1 % (long trades only)

 

[Pat494]

It would be Interesting to see how the 2 systems treat the same data i.e. compare apples with apples.
Another but very relevent point is how do both systems work on new data ? I mean forecasting long or short for say the next few days. It's one thing to have a nice looking model with nice looking results. It has been my experience that good past results are hard to reproduce when actually trading.
Please don't let my observations upset you. I will be interested in your next posts.
keep going
Pat

 

[Hwyman]

Neural net model for TGT (Target Corp)

This model was created to trade the proce of Target Stores (Symbol TGT). As an input I used the price of Walmart ( WMT, main competitor), $INDU and $VIX. I loaded data for 1/1/2002 through 12/31/2004. Biocomp Profit selected 10 best neural models and automatically created a voting system based on them. After I saved the model, I applied it to data for the period 1/1/2005 through 12/05/2005. The model only looked at long trades and traded a fixed amount of 100 shares with $5 roundturn comission. These are the results:

1. the model produced a profit of $325 with a drawdown of $900
2. During the same period, Buy and Hold would have resulted in a gain of $55

the model beat the buy and hold strategy (GOOD) but the profit compared to the drawdown was very low (BAD). There were 4 winning trades and 2 losing trades. I believe I need to use more relevant inputs in my next model such as a retail sector index, market sentiment index and technical indicators.

 

[blackcab]

Interesting, and thanks for posting the results of these experiments - good work. Did you check the price movement correlation between WMT and TGT over the sample?

 

[Chrisso]

Hwyman,

'Neural Networks for Financial Forecasting' by Edward Gately ISBN 0471112127 gives a good methodology for evaluating the relevance of inputs and generally 'fine tuning' neural networks.

It's a while now since I read it but I seem to remember Gately used a neural network to successfully predict the S&P500 futures contract upto about 5hrs in advance using intraday data.

Although historical intraday data is probably more difficut to obtain than end of day data the shorter timeframes do generally mean lower drawdowns and result in networks that don't require nerves of steel to trade with.

Good luck,

Chris

 

[Hwyman]

approach

Hwyman,

'Neural Networks for Financial Forecasting' by Edward Gately ISBN 0471112127 gives a good methodology for evaluating the relevance of inputs and generally 'fine tuning' neural networks.

It's a while now since I read it but I seem to remember Gately used a neural network to successfully predict the S&P500 futures contract upto about 5hrs in advance using intraday data.

Although historical intraday data is probably more difficut to obtain than end of day data the shorter timeframes do generally mean lower drawdowns and result in networks that don't require nerves of steel to trade with.

Good luck,

Chris

I agree that if I try to find a good approach for building a model I am reinventing the wheel: these topics have been widely explored at academic level and applied in many areas of science. I will get the book and several other books on the subject and go over them when I have the time. By the way one of the programs i am using, Biocomp profit 2002, only works with daily data. Neuroshell Trader which I have as well handles intraday. For the stocks I trade I usually analyze on daily basis: I am pretty busy during the trading day to be following intraday trading. For stocks, daily analysis and trading works well, unlike futures and forex where the drawdowns in daily trading can be substantial. When I have the time I will share a few of the more sophisticated models I built. The problem I have faced mostly is finding inputs that have predictive power. Going back to blackcab's post above. Finding a good correlation between two data sets does not mean they have predictive power, all it means they relate in the past. if correlation was the key, building a good model would be a breeze!

 

[blackcab]

True