3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

This is a discussion on 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines within the Trading Software forums, part of the Commercial category; Originally Posted by Dommo Oddly, that is a very precise description of my response to technical analysis. Ah your just ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Oct 16, 2010, 1:29am   #9
Joined Nov 2008
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by Dommo View Post
Oddly, that is a very precise description of my response to technical analysis.

Ah your just not doing it right..
Prawnsandwich is offline   Reply With Quote
Old Oct 16, 2010, 1:31am   #10
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by Prawnsandwich View Post
Ah your just not doing it right..
Well, you're quite right; indeed, I'm not doing it at all.
Dommo is offline   Reply With Quote
Old Oct 16, 2010, 1:32am   #11
Joined Nov 2008
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Quote:
Originally Posted by Dommo View Post
Well, you're quite right; indeed, I'm not doing it at all.
Welp you have to be doing it for it to make sense...
Prawnsandwich is offline   Reply With Quote
Old Oct 16, 2010, 8:47am   #12
 
Pat494's Avatar
Joined Mar 2004
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Or to take arms against a sea of bubbles
...etc (thanks, Bard)


err thx
haven't a clue what he's on about
__________________
Bonus fortuna
I look forward to future successes
and learn from the past.
Peace & prosperity to all.
Pat494 is offline   Reply With Quote
Old Oct 16, 2010, 5:22pm   #13
Joined Aug 2008
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

You should not waist your time on this kind on nets. I have implemented several kinds and realized that the reasons they work so well at classifying stuffs are the very reasons they donít work when applied to markets:

They are very sensitive to the number and the size of layers which make them impractical in market environments.

For the training stage to converge properly they need many thousands input occurrences:
- Even in optimized C++ training takes several hours,
- Even with feature extraction, it is hard or even impossible to find such high number of input occurrences from market data.

These networks have several hundred million neurons the majority on them placed on the first layer. The reason Deep networks work so well in classification is that they encode each possible solution to the problem in the first layer. It is then just a matter to pick the most appropriate one from higher level representations just as it happens in the brain. Deep networks fit the problem at hand very well though raising the question of generalization. Neural nets do not adapt but I was expecting them to pick recurrent market states so that I could trade the deviation. Well, it was quite a disappointment for me to realize that they donít. It was argued that deep networks infer new solutions after learning but I did not see it.

My conclusion after 3 years investigating deep architectures. They are not appropriate to trading at the moment. To address the generalization issue we need incremental versions with online learning. I am currently looking into dynamic factored structures which use similar RBM building blocks. I believe they are more promising but still in a very early stage of development.

My advice: carry out due diligence before spending time on this.

My 2 pips.
Highfreq is offline   Reply With Quote
Old Oct 16, 2010, 5:47pm   #14
Joined Jan 2007
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzystof has noticed several bugs in the TradeFX programs instantpip.m, instantpipexit.m, and explosivepip.m. These all involve future leaks in which a future price is used to make a current calculation. I assume that he will eventually propose fixes for these bugs.. in fact he has below.

I am concerned with future leaks caused by the use of open prices as entry and exit prices when the conditions causing entry and exit depend on closing prices. This is a massive future leak, and means that all of the TradeFX conclusions about the performance of the strategies must be re-tested. This can be fixed by using closing prices instead of open prices for entry and exit.

This post is intended to warn members that the code in TradeFX should be examined very carefully before it is used.

As these bugs are removed we can begin testing the hypotheses in the TradeFX papers.

Welcome to you thread-poets. Hope that you contribute comments or code as well as poetry.

Last edited by fralo; Oct 16, 2010 at 6:21pm.
fralo is offline   Reply With Quote
Old Oct 16, 2010, 5:52pm   #15
Joined Jan 2009
TradeFX bugs

Krzysiaczek99 started this thread Here are three bugs what i found in TradeFX scripts

1) Future leak from InstandtPip line 64

Code:
64        cond(2) = high(t+1) < high(t) && low(t+1) > low(t);
index t+1 is used what is not allowed


2) code in instantPipExit lines 50-71

Here index t+2 is used. So limit and stop vars are based on this index


Code:
stop = price(t+2);
    if high(t) > stop
        stop = high(t);
    end
    limit = price(t+2) - LIMIT;
   
    cond = zeros(1,3);
    exitTime = 0;
    status = -1;
    for i = t+1:length <------ !!!!! but here it start from t+1
        cond(1) = pSAR(i) < close(i); % parabolic SAR below
        cond(2) = within(lwBol(i),price(i));
        cond(3) = (price(i) > stop) || (price(i) < limit);
        if sum(cond) >= 1
            exitTime = i;
            break;
        end
    end
so you can not compare data t+1 with data t+2 (stop and limit) because t+2 is in the future and you don't know it !!!!! it is future leak

in this code cond(3) = (price(i) > stop) || (price(i) < limit); loop is running over all prices but stop/limit don't adjust to new prices because of fixed t at this moment


3) Future leak from explosivePip line 72.

Code:
72   cond(3) = open(t+1) < close(t); % price breaks
index t+1 is used what is not allowed

Comments are welcome. Maybe somebody will find another bug so please post it

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Old Oct 17, 2010, 12:31am   #16
Joined Jan 2007
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Initially I wondered why we use the SVM on such apparantly trivial estimation problems. Bayes shows us that the best decision we can make uses the conditional pdf p(y/x) where y is output and x is input. So in effect, we are using the SVM to estimate this pdf. Another way to estimate it would be to count the number of times x results in y (Num(y/x)), and count the number of times x occurs. Then a good estimate i s Num(y/x)/Num(x). This is probably pretty good if we have 60 counts or so in each counting bin. So, for 1000 samples, as used in these examples, we could accommodate maybe 16 bins.

In fact, for the Explosivepip strategy there are only 3 conditions and 2 outputs, so we would need only 16 bins, and 1000 samples are probably enough.

However in the pipMaximizer strategy there are 9 conditions and 2 outputs, so we would need 256 bins, and with 1000 samples we have only 4 counts in each bin on average, so the estimate would probably be poor indeed if the pdf is at all spread out.

It would be interesting to compare the performance of an SVM to a histogram at least for the ExplosivePip strategy.

We might simply use more samples and return to counting things, but then we might run up against the inherent non-stationarity in Fx series. By the time we got to 25600 samples we might have lost the local pdf, and find only a pdf averaged over too much time. So maybe the SVM is a good idea after all. It will be a particularly good idea if we ever get to continuous inputs and outputs.

Incidentally, I have been trying to understand the hasline.m code. It seems to me that this is a count of the number of bars that cross a line defined by the mid of a bar. This is not the same as the article referenced in the code comments, where the idea is to find price lines with the minimum number of crossings. Furthermore, the result is counted over a period from 1 to t-1 which increases with t. So as t increases this count will increase, and the likelihood that it will exceed M will increase, until after a couple of hundred samples it seems that this indicator is very likely to be 1 and carry no information. Can someone check me on this please?
fralo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
Neural Networks - Genetic Algorithms - Boltzmann Machines - FOREX crystal balls? gtatix Forex 27 Apr 30, 2012 9:27am
How deep is your Level II? insight2 Technical Analysis 6 Apr 29, 2008 10:41pm
In at the deep end! superspur First Steps 12 Dec 24, 2007 11:34am
Deep insight Disqplay Trading Software 1 Feb 7, 2006 12:01am
Deep INsight NKE Trading Software 0 Apr 8, 2005 10:57am

LinkBacks (?)
LinkBack to this Thread: http://www.trade2win.com/boards/trading-software/105880-3rd-generation-nn-deep-learning-deep-belief-nets-restricted-boltzmann-machines.html
Posted By For Type Date
Better NN EA development - Page 82 - Forex-TSD This thread Pingback Oct 17, 2010 12:51pm

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)