3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

This is a discussion on 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines within the Trading Software forums, part of the Commercial category; Originally Posted by fralo Class problems require transformation of raw data into 'feature space' transformation of feature space into (usually ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Oct 30, 2010, 4:45am   #57
 
5 Posts
Joined Oct 2010
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by fralo View Post
Class problems require
transformation of raw data into 'feature space'
transformation of feature space into (usually higher dimensional) decision space
placement of hyperplane in decision space to make decisions
In this sense all classification problems can be solved with a linear classifier.

The hard part is to find the transformations from data space to feature space and from feature space to decision space.
Agreed.

Quote:
Since we know algorithms to search for the appropriate transforms (backprop, QP, etc), it makes sense to avoid indicators, and search for the transforms using raw data as the input.
I'm a skeptical of this because of the high noise level and very high dimensionality.

I mean you can already do something dumb, and just put all your observations of a big length M vector into a PCA and extract out the principal components. They'll probably look like Lagrange polynomials (PCA on random time series), and so your features are dot-producting a few of these into history. Will this have out of sample predictive value substantially better than EMA's at various timescales? Not sure. Or what about just making some time-space to wavenumber space transform manually?

Just because it's not obvious at all what these deep networks will extract given various inputs doesn't mean that they're finding something profound. I see them used in mimicry of natural learning tasks where the inputs (e.g. from sensory neurons) are very structured because of reality of the world and sensory channels have structure.

Of course I can be wrong.

Quote:
But this data is notoriously non-stationary, since e.g. it trends. So we need to make some simple transform to introduce at least quasi-stationarity. First differences of OHLC are quasi-stationary in the sense that the mean and stdev are relatively constant over relatively long periods. An alternative is %change, which is closely related, and perhaps more stationary. It would also be reasonable to use detrended data (e.g. x - SMA(x)). None of these destroy information as so many indicators do.
I think that "destruction of information" is exactly what you want to do. This is calling "information" In the information theoretical sense where a time series with maximal information generation is one with the most noise, and least past-to-future predictability.

Quote:
So the first transform is to difference the data. Now we search for transforms on windows of differenced data that will allow linear seperation. This brings us to my reason for joining this thread. Deep Belief Networks seem to have the ability to find features using unsupervised learning. Although they were suggested by models of the brain, that has nothing to do with their value here. In a search for transforms of raw data to features we need something that will operate without supervision, essentially because we don't know just what a good feature is. We do know what a good decision is..or at least we must know before we can design a system, but because the feature-transform is into a high dimensional decision space where we can place a hyperplane to get a decision, we don't know how to characterize the 'goodness' of a feature transform. It seems to me that the unsupervised learning capability of DBN's may help select good features. It is that aspect of these that I wish to explore.


That said, the TradeFX framework has been useful, just to show some of the difficulties of 'indicators' as features. It is indeed a naive undergraduate project, and fraught with errors, but it demonstrates the futility of using classic indicators (MA, MACD, RSI, CCI, etc) with powerful search algorithms like SVM. The SVM cannot undo the damage done when the raw data was converted to binary-valued features. The best case here is called pipmaximizer. That uses 9 binary-valued features called conditions. The total input space is restricted to 512 points. It is not likely that those points contain enough information to make trading decisions.
I guess I have a different bias----I feel that a trading system with a simple, small dimensionality is more likely to work, or at least continue to work outside the training interval. And it takes human insight and experience to craft these features.
drchaos is offline   Reply With Quote
Old Oct 30, 2010, 10:55pm   #58
Joined Jan 2009
some basic tests

Krzysiaczek99 started this thread After some nice posts lets return to reality and check if TradeFx actually works and how it behaves in some stress conditions so trading against the patterns which it didn't learn

First I applied TradeFX against simple sinus series. It has quite long period but training was including three full periods.

From the screenshots is clear that it works well. Both Buy and Sell singals were generated
at the bottom and tops of sinwave.

So far so good
Attached Thumbnails
sin_buy.jpg   sin_sell.jpg  
Krzysiaczek99 is offline   Reply With Quote
Old Oct 30, 2010, 10:59pm   #59
Joined Jan 2009
basic tests

Krzysiaczek99 started this thread Than I applied TradeFX against more advanced signal which contains 1000 bars of
three cycles 20,40 and 100 bars + trend + noise, than it switches to pure downtrend
Attached Thumbnails
gg.jpg  
Krzysiaczek99 is offline   Reply With Quote
Old Oct 30, 2010, 11:08pm   #60
Joined Jan 2009
basic tests

Krzysiaczek99 started this thread 1st test was done 775 Bars trainig and 1200 Bars OOS. It means system is trained on uptrend part (1st 1000 bars) but trading on last 225 bars of uptrend and 1000 of downtrend

From the results is possible to see that:

for sell orders - sell orders were not executed during downtrend, most likely because system was trained for uptrend

for buy orders - even after switchnig to donwtrend system generate continous buy signal at donwtrend

so very bad !!
Attached Thumbnails
775is_1200oos_buy.jpg   775is_1200oos_sell.jpg  

Last edited by Krzysiaczek99; Oct 30, 2010 at 11:29pm.
Krzysiaczek99 is offline   Reply With Quote
Old Oct 30, 2010, 11:14pm   #61
Joined Jan 2009
basic tests

Krzysiaczek99 started this thread than i tried another combination: 1200 bars for training and 775 for OOS

From results you can see that

for buy orders - system don't generate any buy signals very good !!
for sell orders - system generate continous sell signal : very good !!
Attached Thumbnails
1200is_775oos_buy.jpg   1200is_775oos_sell.jpg  
Krzysiaczek99 is offline   Reply With Quote
Old Oct 30, 2010, 11:20pm   #62
Joined Jan 2009
Basic tests

Krzysiaczek99 started this thread Than final tests I made setting 1st 500 bars of downtrend for training than last 500 for OOS. So it is trained on downtrend and trading on downtrend

from result you can see

for buy orders - no buy orders generated very good
for sell orders - it started to generated sell orders, stopped and resume again
Attached Thumbnails
500is_500oos_buy.jpg   500is_500oos_sel.jpg  
Krzysiaczek99 is offline   Reply With Quote
Old Oct 30, 2010, 11:27pm   #63
Joined Jan 2009
conclusion

Krzysiaczek99 started this thread So from all those post above is possible to conclude that basic funcionality of TradeFX for
sell/buy orders is working however in order to have good results test patterns must be included in training patterns set, otherwise TradeFX gets confused. I used PipMaximizer strategy for this test.

Lets see if DBN and RBM will do better than SVM
Krzysiaczek99 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
Neural Networks - Genetic Algorithms - Boltzmann Machines - FOREX crystal balls? gtatix Forex 27 Apr 30, 2012 9:27am
How deep is your Level II? insight2 Technical Analysis 6 Apr 29, 2008 10:41pm
In at the deep end! superspur First Steps 12 Dec 24, 2007 10:34am
Deep insight Disqplay Trading Software 1 Feb 6, 2006 11:01pm
Deep INsight NKE Trading Software 0 Apr 8, 2005 10:57am

LinkBacks (?)
LinkBack to this Thread: http://www.trade2win.com/boards/trading-software/105880-3rd-generation-nn-deep-learning-deep-belief-nets-restricted-boltzmann-machines.html
Posted By For Type Date
Better NN EA development - Page 82 - Forex-TSD This thread Pingback Oct 17, 2010 12:51pm

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)