3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

This is a discussion on 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines within the Trading Software forums, part of the Commercial category; Indicator builder uses open for BBands and all MA's. If we use close for entry/exit (as we should) then I ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Oct 17, 2010, 5:20am   #17
Joined Jan 2007
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Indicator builder uses open for BBands and all MA's. If we use close for entry/exit (as we should) then I suggest that all indicators use close as input.
fralo is offline   Reply With Quote
Old Oct 17, 2010, 7:29am   #18
Joined Nov 2001
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by Prawnsandwich View Post
Listened to the first few minutes of one of these out of curiosity..and I have to say it sounds like a pile of sh1t..... deliberately vague and pseudo -complicated ..maybe it gets better further on in...anyone else have more patience ?


I like the way you circled the problem and then decided to call a spade a spade!

They'll be feeding us prawnsandwiches next!
Splitlink is offline   Reply With Quote
Old Oct 17, 2010, 10:53am   #19
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread
Quote:
Originally Posted by fralo View Post
Indicator builder uses open for BBands and all MA's. If we use close for entry/exit (as we should) then I suggest that all indicators use close as input.
Yes it must be changed otherwise is a mess for example in this code
he compares pSAR with Close and checks MACDSign which is calculated on Open

Code:
cond(1) = pSAR(i) < close(i); % parabolic SAR below
            cond(2) = (price(i) > stop) || (price(i) < limit);
            cond(3) = (MACDSign(i)) > 0 && (MAhist(i) > 0); %TODO: cross over above
I will make a document where i will collect all changes and solutions
Krzysiaczek99 is offline   Reply With Quote
Old Oct 17, 2010, 10:59am   #20
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread
Quote:
Originally Posted by Highfreq View Post
You should not waist your time on this kind on nets. I have implemented several kinds and realized that the reasons they work so well at classifying stuffs are the very reasons they donít work when applied to markets:

They are very sensitive to the number and the size of layers which make them impractical in market environments.

For the training stage to converge properly they need many thousands input occurrences:
- Even in optimized C++ training takes several hours,
- Even with feature extraction, it is hard or even impossible to find such high number of input occurrences from market data.

These networks have several hundred million neurons the majority on them placed on the first layer. The reason Deep networks work so well in classification is that they encode each possible solution to the problem in the first layer. It is then just a matter to pick the most appropriate one from higher level representations just as it happens in the brain. Deep networks fit the problem at hand very well though raising the question of generalization. Neural nets do not adapt but I was expecting them to pick recurrent market states so that I could trade the deviation. Well, it was quite a disappointment for me to realize that they donít. It was argued that deep networks infer new solutions after learning but I did not see it.

My conclusion after 3 years investigating deep architectures. They are not appropriate to trading at the moment. To address the generalization issue we need incremental versions with online learning. I am currently looking into dynamic factored structures which use similar RBM building blocks. I believe they are more promising but still in a very early stage of development.

My advice: carry out due diligence before spending time on this.

My 2 pips.
Very interesting mail. Is your research a public domain ?? Is it possible to have a look ??
What sort of models you examined ?? Did you try Hilton's autoencoder model with pretraining, training and fine tuning ?? What learning algorithms you used ??

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Old Oct 17, 2010, 11:44am   #21
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread interesting post from barnix from forex-tsd

http://www.forex-tsd.com/expert-advi...tml#post365921

on his site

http://sites.google.com/site/bettereaclone/

there is his SVM EA together with some infor about LDS. Another system to compare and methods to investigate

Last edited by Krzysiaczek99; Oct 17, 2010 at 11:54am.
Krzysiaczek99 is offline   Reply With Quote
Old Oct 17, 2010, 1:47pm   #22
Joined Nov 2008
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by Splitlink View Post


I like the way you circled the problem and then decided to call a spade a spade!

They'll be feeding us prawnsandwiches next!
Maybe I was just being lazy and jumping to conclusions.....or maybe not..
Prawnsandwich is offline   Reply With Quote
Old Oct 17, 2010, 5:10pm   #23
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread
Quote:
Originally Posted by fralo View Post

Incidentally, I have been trying to understand the hasline.m code. It seems to me that this is a count of the number of bars that cross a line defined by the mid of a bar. This is not the same as the article referenced in the code comments, where the idea is to find price lines with the minimum number of crossings. Furthermore, the result is counted over a period from 1 to t-1 which increases with t. So as t increases this count will increase, and the likelihood that it will exceed M will increase, until after a couple of hundred samples it seems that this indicator is very likely to be 1 and carry no information. Can someone check me on this please?
Yes, the code is wrong

Code:
its called from here in explosivePip

54  for t=1:length-1
          price = open;
          cond = zeros(1,NUM_COND);
          cond(1) = hasLine(low,high,t,'SUP'); 


function x = hasLine(low, high, t, type)
    % adopted from: http://articles.mql4.com/369
    % A Method for Drawing Support and Resistant Line
    % a suport line is present if there at least previous M bar that cross
    % its price.
    % We added that past high must not below current line for support
    % and past low must not above current line for resistant
    % In this simplified version, we choose:
    M = 4;   
    x = 0;
    if (t >= M)        
        cross_count = 0;
        mid = (low(t) + high(t))/2;
        last = t-1;     
        for i = 1:last  <----- range will increase on every call
            if (low(t-i) < mid) && (high(t-i) > mid)
                cross_count = cross_count + 1; % first cross is itself
                if (cross_count >= M)
                    break;
                end
the range in 'for' statement will increase but it should be fixed. Perhaps instead of this we can use another solution. See this.

http://www.breakoutfutures.com/Newsl...letter0303.htm

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Old Oct 18, 2010, 12:47pm   #24
Joined Jan 2009
Convolutional Neural Networks

Krzysiaczek99 started this thread I believe we should extend scope of our project for Lechun Convolutional Neural Networks.
Seems to have error lower by 50-70% than BP and SVM. Relevant info below.

http://www.iro.umontreal.ca/%7Elisa/...hapter2007.pdf
http://www.cs.nyu.edu/~yann/talks/le...3-convnets.pdf

http://www.mathworks.com/matlabcentr...-network-class

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
Neural Networks - Genetic Algorithms - Boltzmann Machines - FOREX crystal balls? gtatix Forex 27 Apr 30, 2012 9:27am
How deep is your Level II? insight2 Technical Analysis 6 Apr 29, 2008 10:41pm
In at the deep end! superspur First Steps 12 Dec 24, 2007 11:34am
Deep insight Disqplay Trading Software 1 Feb 7, 2006 12:01am
Deep INsight NKE Trading Software 0 Apr 8, 2005 10:57am

LinkBacks (?)
LinkBack to this Thread: http://www.trade2win.com/boards/trading-software/105880-3rd-generation-nn-deep-learning-deep-belief-nets-restricted-boltzmann-machines.html
Posted By For Type Date
Better NN EA development - Page 82 - Forex-TSD This thread Pingback Oct 17, 2010 12:51pm

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)