3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

This is a discussion on 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines within the Trading Software forums, part of the Commercial category; Originally Posted by Krzysiaczek99 Did anybody try to extract the features for Financial Time Series ?? According to this paper ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Nov 14, 2010, 8:45pm   #81
Joined Aug 2008
Re: Feature extraction

Quote:
Originally Posted by Krzysiaczek99 View Post
Did anybody try to extract the features for Financial Time Series ??

According to this paper error from pruned feature set is lower.

There is also very good video lecture about it

http://videolectures.net/mmdss07_guyon_fsf/

Krzysztof
Feature extraction is a big word for data selection. But it is more involved than naively inputting price and some indis as most amateurs do. The later does not work. I more specifically look for features that are recurring and remove the junk. I also generate negative features that help the discrimination process. These ones alone make me a living. A good example is the shift of cycles to larger time-frames I touched upon before. The choice of technology (NNs, SVMs…) becomes secondary. My two cents.
Highfreq is offline   Reply With Quote
Old Nov 15, 2010, 12:56am   #82
Joined Jan 2009
Re: Feature extraction

Krzysiaczek99 started this thread
Quote:
Originally Posted by Highfreq View Post
Feature extraction is a big word for data selection. But it is more involved than naively inputting price and some indis as most amateurs do. The later does not work. I more specifically look for features that are recurring and remove the junk. I also generate negative features that help the discrimination process. These ones alone make me a living. A good example is the shift of cycles to larger time-frames I touched upon before. The choice of technology (NNs, SVMs…) becomes secondary. My two cents.
I know that feature extraction is quite important, they were even competitions with money prizes in it on some test data sets.

What you mean by 'shift the cycles to larger TF ' ?? Larger TF is just less frequent sampling. Or you mean that the cycles can not be recognised on current TF any more but can be on higher TF so they are shifting to higher TF ??

Are you using any data mining tools like Rapid Miner or Weka ?? Matlab ??

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Old Nov 15, 2010, 1:01pm   #83
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread For people not familiar with Weka here are the links

presentation of Weka and R

http://videolectures.net/bootcamp07_belanche_mldm/

and like 20 or more videos how to use it

http://sentimentmining.net/weka/
Krzysiaczek99 is offline   Reply With Quote
Old Nov 16, 2010, 9:56pm   #84
Joined Aug 2008
Re: Feature extraction

Quote:
Originally Posted by Krzysiaczek99 View Post
I know that feature extraction is quite important, they were even competitions with money prizes in it on some test data sets.

What you mean by 'shift the cycles to larger TF ' ?? Larger TF is just less frequent sampling. Or you mean that the cycles can not be recognised on current TF any more but can be on higher TF so they are shifting to higher TF ??

Are you using any data mining tools like Rapid Miner or Weka ?? Matlab ??

Krzysztof
"Or you mean that the cycles can not be recognised on current TF any more but can be on higher TF so they are shifting to higher TF ??"

Exactly! The cycle shifts to the larger one when institutional traders place huge bets on the daily crossover. For example, if the 5min bar crosses the day open with just the right amplitude, then the orders shift on the day bar causing the current TF to loose equity. It’s time to switch TF.

I occasionally use Matlab and R. Mostly I use NeuroShell from Ward with a few plugins and write my own in C++. I don't know RapidMiner and Weka. They are certainly great tools to get accustomed to the new technologies but I fear they don't offer the level of customization that is often needed to make a generic technology work on market data. The solution often lies in the details.
Highfreq is offline   Reply With Quote
Old Nov 17, 2010, 1:00am   #85
Joined Jan 2009
Weka

Krzysiaczek99 started this thread Yes those tools are very powerful very good for making comparative analysis, most algorithms and methods is already developed either for Weka or for Matlab so in my opinion they must be used. See this. I applied data from TradeFX for diffeent algos, than tried Adaboost to all those algos, than made feature extraction and did it again to see if there are any improvements, sadly no. All it took 1 hour work.....

Any info about Learn++

Quote:
Tester: weka.experiment.PairedCorrectedTTester
Analysing: Percent_correct
Datasets: 2
Resultsets: 18
Confidence: 0.05 (two tailed)
Sorted by: -
Date: 11/15/10 5:13 PM


Dataset (1) rules.Ze | (2) trees (3) trees (4) trees (5) trees (6) funct (7) funct (8) funct (9) bayes (10) meta (11) meta (12) meta (13) meta (14) meta (15) meta (16) meta (17) meta (18) meta
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ipip_buy (1) 48.42 | 50.97 51.00 51.52 51.55 51.23 49.58 48.42 51.90 48.42 50.65 51.29 51.39 51.74 51.61 52.45 48.32 51.71
ipip_sell (1) 51.03 | 51.74 52.19 51.39 51.71 51.71 51.81 48.97 51.90 51.03 51.68 52.10 51.16 52.16 52.16 51.42 51.03 51.90
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
(v/ /*) | (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0)


Key:
(1) rules.ZeroR '' 48055541465867954
(2) trees.ADTree '-B 10 -E -3' -1532264837167690683
(3) trees.J48 '-C 0.25 -M 2' -217733168393644444
(4) trees.RandomForest '-I 10 -K 0 -S 1' 4216839470751428698
(5) trees.SimpleCart '-S 1 -M 2.0 -N 5 -C 1.0' 4154189200352566053
(6) functions.LibSVM '-S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 40.0 -C 1.0 -E 0.0010 -P 0.1 -model \"C:\\\\Program Files\\\\Weka-3-7\"' 14172
(7) functions.VotedPerceptron '-I 1 -E 1.0 -S 1 -M 10000' -1072429260104568698
(8) functions.Winnow '-I 1 -A 2.0 -B 0.5 -H -1.0 -W 2.0 -S 1' 3543770107994321324
(9) bayes.NaiveBayes '' 5995231201785697655
(10) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W rules.ZeroR' -7378107808933117974
(11) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.ADTree -- -B 10 -E -3' -7378107808933117974
(12) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.J48 -- -C 0.25 -M 2' -7378107808933117974
(13) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.RandomForest -- -I 10 -K 0 -S 1' -7378107808933117974
(14) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.SimpleCart -- -S 1 -M 2.0 -N 5 -C 1.0' -7378107808933117974
(15) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.LibSVM -- -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 40.0 -C 1.0 -E 0.0010 -P 0.1 -model \"C:\\\\Program Files\\\\Weka-3-7\"' -7378107808933117974
(16) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.VotedPerceptron -- -I 1 -E 1.0 -S 1 -M 10000' -7378107808933117974
(17) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.Winnow -- -I 1 -A 2.0 -B 0.5 -H -1.0 -W 2.0 -S 1' -7378107808933117974
(18) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W bayes.NaiveBayes --' -7378107808933117974
Quote:

Tester: weka.experiment.PairedCorrectedTTester
Analysing: Percent_correct
Datasets: 2
Resultsets: 18
Confidence: 0.05 (two tailed)
Sorted by: -
Date: 11/15/10 7:52 PM


Dataset (1) rules.Ze | (2) trees (3) trees (4) trees (5) trees (6) funct (7) funct (8) funct (9) bayes (10) meta (11) meta (12) meta (13) meta (14) meta (15) meta (16) meta (17) meta (18) meta
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
'ipip_buy-weka.filters.un (1) 48.42 | 51.58 51.58 51.58 51.58 51.58 51.65 47.68 51.65 48.42 51.58 51.58 51.58 51.58 51.58 51.61 51.58 51.65
'ipip_sell-weka.filters.s (1) 51.03 | 51.71 51.71 51.71 51.71 51.71 51.71 48.97 51.71 51.03 51.71 51.71 51.71 51.71 51.71 51.71 48.29 51.71
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
(v/ /*) | (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0) (0/2/0)


Key:
(1) rules.ZeroR '' 48055541465867954
(2) trees.ADTree '-B 10 -E -3' -1532264837167690683
(3) trees.J48 '-C 0.25 -M 2' -217733168393644444
(4) trees.RandomForest '-I 10 -K 0 -S 1' 4216839470751428698
(5) trees.SimpleCart '-S 1 -M 2.0 -N 5 -C 1.0' 4154189200352566053
(6) functions.LibSVM '-S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 40.0 -C 1.0 -E 0.0010 -P 0.1 -model \"C:\\\\Program Files\\\\Weka-3-7\"' 14172
(7) functions.VotedPerceptron '-I 1 -E 1.0 -S 1 -M 10000' -1072429260104568698
(8) functions.Winnow '-I 1 -A 2.0 -B 0.5 -H -1.0 -W 2.0 -S 1' 3543770107994321324
(9) bayes.NaiveBayes '' 5995231201785697655
(10) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W rules.ZeroR' -7378107808933117974
(11) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.ADTree -- -B 10 -E -3' -7378107808933117974
(12) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.J48 -- -C 0.25 -M 2' -7378107808933117974
(13) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.RandomForest -- -I 10 -K 0 -S 1' -7378107808933117974
(14) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W trees.SimpleCart -- -S 1 -M 2.0 -N 5 -C 1.0' -7378107808933117974
(15) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.LibSVM -- -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 40.0 -C 1.0 -E 0.0010 -P 0.1 -model \"C:\\\\Program Files\\\\Weka-3-7\"' -7378107808933117974
(16) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.VotedPerceptron -- -I 1 -E 1.0 -S 1 -M 10000' -7378107808933117974
(17) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W functions.Winnow -- -I 1 -A 2.0 -B 0.5 -H -1.0 -W 2.0 -S 1' -7378107808933117974
(18) meta.AdaBoostM1 '-P 100 -S 1 -I 10 -W bayes.NaiveBayes --' -7378107808933117974
Krzysiaczek99 is offline   Reply With Quote
Old Nov 19, 2010, 12:34pm   #86
 
1 Posts
Joined May 2006
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

You can see my work about the C++ tester for MT4:
https://sites.google.com/site/ctesterformt4/
__________________
Better EA Clone
barnix is offline   Reply With Quote
Old Nov 21, 2010, 3:22pm   #87
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread
Quote:
Originally Posted by barnix View Post
You can see my work about the C++ tester for MT4:
https://sites.google.com/site/ctesterformt4/
Welcome to the thread.

Are you using mostly classification or regression methods. Can you post some
results of your predictors in form of confusion matrix or ROC curves ??

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Old Nov 24, 2010, 3:50am   #88
Joined Jan 2009
ROC analysis

Krzysiaczek99 started this thread Here is great info about ROC curves and link to presentation from P. Flach

http://videolectures.net/ecml07_flach_pto/
Attached Files
File Type: pdf ROCanalysis.pdf (299.0 KB, 345 views)
Krzysiaczek99 is offline   Reply With Quote
Old Nov 24, 2010, 10:56am   #89
 
2 Posts
Joined Oct 2010
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Hi, I figured something like that.
Boltzmann-Gibbson Selection

double temp = 0.001;

int gibson(double buy, double hold, double sell) {
double T=temp;
double x_buy=MathExp(buy/T);
double x_hold=MathExp(hold/T);
double x_sell=MathExp(sell/T)+0.01;
double y=x_buy+x_hold+x_sell;
double y_buy=X_buy/y;
double y_hold=X_hold/y;
double y_sell=X_sell/y;
double random=MathRand()/32767.0;

if (random>y_buy+y_hold) return (2);
if (random>y_hold) return (0);
return (1);
}


What do you think of reinforcement learning ?

Last edited by xan023; Nov 24, 2010 at 11:05am.
xan023 is offline Coach/Trainer   Reply With Quote
Old Nov 24, 2010, 12:53pm   #90
Joined Jan 2009
Re: 3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machi

Krzysiaczek99 started this thread
Quote:
Originally Posted by xan023 View Post
Hi, I figured something like that.
Boltzmann-Gibbson Selection

double temp = 0.001;

int gibson(double buy, double hold, double sell) {
double T=temp;
double x_buy=MathExp(buy/T);
double x_hold=MathExp(hold/T);
double x_sell=MathExp(sell/T)+0.01;
double y=x_buy+x_hold+x_sell;
double y_buy=X_buy/y;
double y_hold=X_hold/y;
double y_sell=X_sell/y;
double random=MathRand()/32767.0;

if (random>y_buy+y_hold) return (2);
if (random>y_hold) return (0);
return (1);
}


What do you think of reinforcement learning ?
Hi,

Can you explain in more detail what is a relation of your post to this thread ??

Can you post any paper which would show applicability of Boltzmann-Gibbson Selection to the financial time series ??

Regarding reinforcement learning and online algorithms. It's plenty of them around
and they can be a solution for non stationarity of financial time series. But so far I didn't have time to look more deeper.

Krzysztof
Krzysiaczek99 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
Neural Networks - Genetic Algorithms - Boltzmann Machines - FOREX crystal balls? gtatix Forex 27 Apr 30, 2012 10:27am
How deep is your Level II? insight2 Technical Analysis 6 Apr 29, 2008 11:41pm
In at the deep end! superspur First Steps 12 Dec 24, 2007 11:34am
Deep insight Disqplay Trading Software 1 Feb 7, 2006 12:01am
Deep INsight NKE Trading Software 0 Apr 8, 2005 11:57am

LinkBacks (?)
LinkBack to this Thread: http://www.trade2win.com/boards/trading-software/105880-3rd-generation-nn-deep-learning-deep-belief-nets-restricted-boltzmann-machines.html
Posted By For Type Date
Better NN EA development - Page 82 - Forex-TSD This thread Pingback Oct 17, 2010 1:51pm

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)