Neural Net - Different results every time it is 'trained'

[BigDE]

Hi Folks, I am new to the forum and new to trading...so please go easy !
In my quest for a charting package I decided to purchase 'Tradecision' because it has most of the charting indicators I could ever need, a programmable language for creating new indicators and Nerual Network & GAs. Also it came with good backtesting/walk forward testing capabilities to allow me to test my ideas...fabulous !

Then I started to use it and could not get any kind of consistent results from the Neural Net software. The more I explored the neural net software the more it appeared to be giving me 'random' results.
In the end I created my own sample data where the open, close, high and low were all the same figure and represented a simple 'sine' wave which took 3 months to move from top to bottom and back again on daily charts - always consistent and over many 'years' of data. I then provided a simple input to the model 'the close price'...and expected to see the software be able to 'learn' the pattern very quickly. Well suffice to say it didnt and it consistantly gave an 'out of phase' forecasted price, wrong wave amplitude and worse yet, the result figures moved around quite a bit each time the model was re-trained despite the fact that inputs reamined fixed and very simple.

When I create and use a model on real world data the forecasted price, input indicator importance and other such model summary results appear wildly inconsist despite the fact that the input data is unchanged (and I'm not using GA's - rather exhaustive search).

I have attached a screen shot showing 3 different result outputs after each model 're-training'.

$2000 is a lot to pay for a random number generator so I would like some opinions as to what might be going on here and if anyone has another neural net software package (like Neuroshell)...does that do the same thing?

I have attached the screenshot for your reference.

Thanks you in advance for any insight you may be able to provide.

 

[dcraig1]

Depends on the initial weights, I think. ANNs require study and knowledge.

 

[BigDE]

I guess I'm just trying to find out if this type of result is unique to Tradecision because of their poor software implementation of an artifical neural network..or if the implmentation is good and the results are also random on other software packages ...like Neuroshell

Thanks

 

[dcraig1]

I guess I'm just trying to find out if this type of result is unique to Tradecision because of their poor software implementation of an artifical neural network..or if the implmentation is good and the results are also random on other software packages ...like Neuroshell

Thanks

General "feature" of ANNs.

 

[kaciara]

input data? hidden layers #?

 

[Hwyman]

(1)exhaustive search (2) one network

Tradecision is not a dedicated neural network tool. It does have that capability but in scientific terms it is sort of primitive. A lot of what you describe you will find with much more expensive pieces of software let alone Tradecision (yes I know Tradecision is not free). In a more scientific environment a model is created by combining several hundred neural networks. When you average them out you get a fairly consistent result and performance. One program that follows this approach is Biocomp but even with that program I don;t know how practical the implications are if you try to model stock price or other similar movements. One more note regarding Tradecision. You are not using a genetic algorithm but that does not mean that yoyr search is exhaustive. I am sure the software has a built in limit (by time, size or something else) that cuts short an exhaustive search. If your exhaustive search takes a few minutes, I guarantee you it is not exhaustive and it may explain the wild differences from one model to the next. I have performed similar tests with Neuroshell and the results are somewhat consistent, not great but you can see that the neural network is trying to create a model and arrives at similar conclusions independantly