3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

[danielfppps]

Hi, looks like you've already done most of the things I'd like to try !! I'm trying to learn Python as a trading aid/analysis tool these days, and I see this name "Asirikuy" quite a bit here and there. I was wondering whether subscribing to his site (which looks a bit old) worth the money. Would you care to share your opinion on this please ?!

Actually the website was changed recently, so the new image is quite new If you're looking to leverage on our developments in NN I believe that subscribing will be completely worth it. However feel free to ask me any questions you may have about our community (by PM here or by using the contact form at the Asirikuy website).

Yesterday I also published a new article about machine learning on my blog (mechanicalforex.com), there are links to some of my other machine learning developments on that article as well.

 

[hamjii]

Actually the website was changed recently, so the new image is quite new If you're looking to leverage on our developments in NN I believe that subscribing will be completely worth it. However feel free to ask me any questions you may have about our community (by PM here or by using the contact form at the Asirikuy website).

Yesterday I also published a new article about machine learning on my blog (mechanicalforex.com), there are links to some of my other machine learning developments on that article as well.

Hey, congratulations on nice site renovation, and welcome to Web 2.0 ! I didn't know your strategies are ML-based and that there is a python-based trading platform directly linkable to MT4, which is a good surprise (did I tell you I'm a newbie by the way !). Maybe it's better if I discuss the matter on your site.

Many thanks

 

[danielfppps]

Hi Hamjii,

Thanks for writing. We do not have a python platform that can interface with MT4. What we have is a programming framework that interacts with different front-ends using a single library (so you code a system once and can trade it within a variety of platforms). One of the front-ends is a python back-tester program (NST) and the other one is a python trading platform that can execute systems in Oanda/JForex. Here is an image that better explains what we have (attached).

Do let me know if you have any questions,

Best Regards,

Daniel

 

[Krzysiaczek99]

I'm back

After few months off I'm back with continuation of my research. Here are the trading results of last 2 days of the system. So far so good

Krzysztof

 

[xyz1453]

Hi ,

How do the Theano based algos effect your over all equity curve?Are they worth the current hype?

 

[Krzysiaczek99]

Hi ,

How do the Theano based algos effect your over all equity curve?Are they worth the current hype?

No Theano algos and no deep learning algos in live system. During MATLAB backtest first i was using theano deep nets and RBMs then 2 different MATLAB implementations of deep nets, RBMs and autoencoders but they turned out not to be so efficient and easy to train like other algos so I gave up with them.

Krzysztof

 

[surfeur]

Krzysiaczek99,

You have backtest your algo or it's only forward test ? what is your timeframe ?

 

[ECNFXTrader]

hi ,

Thanks for this post, it was quite informative.

 

[Krzysiaczek99]

Krzysiaczek99,

You have backtest your algo or it's only forward test ? what is your timeframe ?

By back test i mean forward test so collection of 1 day forward results. In pararell i run live test 4 currency pairs, 4 EAs and 4 MATLABs and results what i posted a few mails back are from this live test. TF is 1min

Krzysztof

 

[Krzysiaczek99]

4 days

And here are results after 4 days of trading. More than 1000 trades done. Maximum number of open lots is 2

Krzysztof

 

[Krzysiaczek99]

5 days

Here are the final results of 1st week of live test and I can say they are very promising. Lets see how it will go next week. Making MonteCarlo analysis of trades predicted monthly return would be like 11.5k USD based on maximum 2 lot trading.

Krzysztof

 

[Krzysiaczek99]

7day

After 7 days still good results. Overall PF 4.25, > 2200 trades done.

Krzysztof

 

[Krzysiaczek99]

10 days

Here are the results after 10 days of trading. Seems after stream of good days, a few bad days come. The position sizing for this system seems to be kind of random and it can cause such big drawdowns.

 

[Krzysiaczek99]

1 month

So here is a situation after 1 month of trading. More than 6700 trades done, system
seems to work. Im still trying to improve EAs to have trading as much close as possible to MATLAB trading. Hopefully corrections which i introduced will improve the performance.

Krzysztof

 

[Krzysiaczek99]

L1 L2 regularizations, dual, bias and bad daughter

Happy New Year !!!

As markets are closed now and my bad daughter refused to play tennis with me i made some investigations about impact of different types of regularization for results of my system. For this I choose algo unfriendly days i.e. 15,16 and 17.12 and EURUSD 1 min when EURUSD broke long term suport level and market behaved very strange. Then I rerun for those days in MATLAB LIBLINEAR classifier with different regularization and bias values

options:
-s type : set type of solver (default 1)
0 -- L2-regularized logistic regression (primal)
1 -- L2-regularized L2-loss support vector classification (dual)
2 -- L2-regularized L2-loss support vector classification (primal)
3 -- L2-regularized L1-loss support vector classification (dual)
4 -- multi-class support vector classification by Crammer and Singer
5 -- L1-regularized L2-loss support vector classification
6 -- L1-regularized logistic regression
7 -- L2-regularized logistic regression (dual)
-c cost : set the parameter C (default 1)
-e epsilon : set tolerance of termination criterion
-s 0 and 2
|f'(w)|_2 <= eps*min(pos,neg)/l*|f'(w0)|_2,
where f is the primal function and pos/neg are # of
positive/negative data (default 0.01)
-s 1, 3, 4 and 7
Dual maximal violation <= eps; similar to libsvm (default 0.1)
-s 5 and 6
|f'(w)|_inf <= eps*min(pos,neg)/l*|f'(w0)|_inf,
where f is the primal function (default 0.01)
-B bias : if bias >= 0, instance x becomes [x; bias]; if < 0, no bias term added (default -1)
-wi weight: weights adjust the parameter C of different classes (see README for details)
-v n: n-fold cross validation mode
-q : quiet mode (no outputs)

and here are the results. BS means BuySell signal i.e. ensembled from2 separate BUY and SELL signals. For each of this signals LIBLINEAR had the same settings

 

[Krzysiaczek99]

so at least from this is possible to conclude

L2 is preferred over L1 means features build by EURUSD1min are not sparse...this can explain why is so many losers between FOREX traders...Then feature extraction may also not be needed.


Dual solutions is preferred over primal. Who knows what to conclude from this...Dual is differentiable for example but I think LIBLINEAR uses different optimization algo from primal and dual so maybe is connected to this ??

Bias should be not used

SVM is preffered over LR

Comments are welcome. In next few days will try to get more results for different regularization's using this package http://people.sabanciuniv.edu/haerdogan/software/qmm library/qmmindex.htm. Hopefully it can give some insights about FOREX data..

Krzysztof

 

[Krzysiaczek99]

Libsvm

and here are the results for LIBSVM for RBF kernel for the same days. Seems to be profitable but very little profit so little trades. So seems linear kernel is not so bad.....

 

[Krzysiaczek99]

RBF kernel

Basic on my suspicions and observations I got curious if popular RBF kernel has a predictive power on financial HF time series. For this I made a MATLAB backtest with popular LIBSVM classifier with RBF kernel for a week before Christmas (15-19 Dec) for 6 currency pairs, S&P500 and gold. As usual results are surprising.

Total result for different c and gamma.
PP - avg precision
RC - avg recall
MC- avg Mathews correlation index
totTP - true positives total (good trades)
totFP - false positives total (missed trades)

>> resultsAll('LIBSVM*-c 0.1 -w')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-701326.00 0.27 0.02 65.96 0.08 953 4462

>> resultsAll('LIBSVM*-c 1 -w')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-701119.00 0.27 0.03 59.47 0.08 980 4485

>> resultsAll('LIBSVM*-c 10 -w')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-696425.00 0.26 0.03 60.55 0.09 991 4485

>> resultsAll('LIBSVM*-c 100 -w')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-696425.00 0.26 0.03 60.55 0.09 991 4485


resultsAll('LIBSVM*-c 1 -w1 1000 -g 0.01')
resultsAll('LIBSVM*-c 1 -w1 1000 -g 0.5')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-460485.00 0.16 0.05 53.42 0.13 1452 5974

NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
873006.00 4.50 -0.11 18.99 0.44 6519 21011

>>

 

[Krzysiaczek99]

so clearly it is poor performance (avMC < 0.1, low PF and loss). However if you look exactly where were the losers seems that only S&P500 was a loser

 

[Krzysiaczek99]

It is confirmed by results from other SVM like algo Pegasos which also lost for S&P.
But for comparison another technique algo CHIRP was very successful.

So it seems that at least for those days SVM algos with RBF kernel were only efficient for currencies but not for S&P500. And CHIRP was effective for both.

So it looks that SVM technique is widely used for stock market and lost its efficiency.

Krzysztof

>> resultsAll('CHIRP')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
258424.00 8.10 0.12 72.41 0.04 593 89

resultsAll('Peg')
NORMAL DATA AVERAGE RESULTS
Profit PF avMC avPP avRC totTP totFP
-131829.00 0.81 0.11 55.69 0.08 1262 555

>>