3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

Krzysiaczek99 · Dec 28, 2016

Pat494 said:
In testing, the algo should be profitable on any pair but not necessarily on any time frame. Thus if it has to be optimised or restricted to 1 or so pairs it is imho not good enough.

This is a reason that i run it on 8 symbols to decrease the risk.
Here you have results for 8 months both per algo and per pair and you can see that
variance per algo is not so big as variance per pair

Krzysztof

Code:

>> resultsAll('')
resultsAll('*Peg*')
resultsAll('*CHIRP*')
resultsAll('*J48*')
resultsAll('*RBM*')
resultsAll('*SDAE*')
resultsAll('*ELM*')
   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-294941604.50         0.81        -0.28        50.87         0.46      3126561      3033796          680         1620       -47.88

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-45299854.70         0.83        -0.29        51.60         0.53       587115       562889          125          270       -39.39

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-45721371.30         0.80        -0.13        51.65         0.44       467092       435979          112          270       -50.63

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-54606643.40         0.78        -0.32        50.84         0.42       496707       479771          116          270       -55.92

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-67780911.70         0.76        -0.36        49.89         0.46       529967       529545          105          270       -63.97

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-50482614.70         0.82        -0.31        50.86         0.50       569811       553772          107          270       -44.93

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-31050208.70         0.86        -0.29        50.37         0.41       475869       471840          115          270       -32.76

>> resultsAll('*AUDUSD*')
resultsAll('*EURUSD*')
resultsAll('*GBPUSD*')
resultsAll('*XAUUSD*')
resultsAll('*AUDJPY*')
resultsAll('*GBPJPY*')
resultsAll('*USDJPY*')
resultsAll('*US500*')
   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-15189450.00         0.86        -0.29        51.43         0.46       401667       377302           89          204       -19.50

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-33474123.00         0.75        -0.28        48.91         0.48       394242       415935           82          204       -41.32

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-82445558.00         0.67        -0.28        48.96         0.46       382315       405050           78          204      -104.71

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-26464370.50         0.88        -0.28        52.45         0.47       420708       377585           78          204       -33.15

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-35181391.00         0.77        -0.28        48.07         0.46       374913       410893           73          204       -44.77

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-134415565.00         0.64        -0.32        48.65         0.45       380813       399408           70          204      -172.28

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
  7415998.00         1.05        -0.28        53.55         0.48       424010       365899           94          204         9.39

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 24812855.00         1.21        -0.27        55.20         0.43       347893       281724          116          192        39.41

>>

surfeur · Dec 29, 2016

Krzysztof,

On this 8 monthis it's forward backtest or not ? You re-train often your system ? it's same inputs/algo for all pairs ?

Regards.

Krzysiaczek99 · Dec 29, 2016

surfeur said:
Krzysztof,

On this 8 monthis it's forward backtest or not ? You re-train often your system ? it's same inputs/algo for all pairs ?

Regards.

It is forward test - collection of 1 day trading forward results. Inputs are changing dynamically for BUY and SELL everyday using feature selection algos and its retrained every 24h

darktrade · Jan 22, 2017

Does PF stand for profit factor? I see all but two PF < 1.

Krzysiaczek99 · Jan 22, 2017

darktrade said:
Does PF stand for profit factor? I see all but two PF < 1.

Yes, PF means Profit Factor. Results are either average of all algorithms per pair or
all pairs per algorithm. If you look the best algorithm per pair it looks like this.

Krzysztof

Code:

>> resultsAll('*AUDUSD*SDAE*')
resultsAll('*EURUSD*SDAE*')
resultsAll('*GBPUSD*SDAE*')
resultsAll('*XAUUSD*SDAE*')
resultsAll('*AUDJPY*SDAE*')
resultsAll('*GBPJPY*SDAE*')
resultsAll('*USDJPY*SDAE*')
resultsAll('*US500*SDAE*')
   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 16360413.00         1.78        -0.29        68.07         0.35        58571        33651           20           31       177.40

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 27788340.00         2.32        -0.31        65.39         0.40        66586        35585           20           31       271.98

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 14171537.00         1.23        -0.28        70.39         0.38        67606        27857           20           31       148.45

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 -7595151.50         0.91        -0.43        53.38         0.35        58225        40004           19           31       -77.32

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 -7044204.00         0.83        -0.41        63.64         0.26        44487        35719           20           31       -87.83

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 31020729.00         1.44        -0.37        64.71         0.32        59461        29804           20           31       347.51

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
-31781079.00         0.64        -0.39        59.03         0.34        58941        39845           16           31      -321.72

   NORMAL DATA AVERAGE RESULTS
      Profit           PF         avMC         avPP         avRC        totTP        totFP         PF>1     algosnum     perTrade
 29440865.00         1.99        -0.28        66.79         0.43        66728        24571           19           29       322.47

>>

joseph1986 · Jan 25, 2017

my god 7 years and running.
What's your R square value?

Krzysiaczek99 · Jan 25, 2017

R square its not applicable here - its classification not regression.

Krzysztof

joseph1986 · Jan 25, 2017

Krzysiaczek99 said:
R square its not applicable here - its classification not regression.

Krzysztof

ok, let me rephrase it
what is your mis-classification rate?

Krzysiaczek99 · Jan 25, 2017

All info u have in posts with results(e.g. 265) . Anyway here is a description

measures used were :

Profit - profit
PF - profit factor
avMC - average Mathew corr index
avPP - average % profitable (precision(
avRC - average recall
totTP - successful trades
totFP - missed trades
PF>1 - algos with PF>1
algosnum - number of algos
perTrade - profit per trade

Mislassifications (or % profitable or precision as ML measure) is important but you can see that e.g. some PFs and higher but PP is lower.

Krzysztof

joseph1986 · Jan 25, 2017

Krzysiaczek99 said:
All info u have in posts with results(e.g. 265) . Anyway here is a description

measures used were :

Profit - profit
PF - profit factor
avMC - average Mathew corr index
avPP - average % profitable (precision(
avRC - average recall
totTP - successful trades
totFP - missed trades
PF>1 - algos with PF>1
algosnum - number of algos
perTrade - profit per trade

Mislassifications (or % profitable or precision as ML measure) is important but you can see that e.g. some PFs and higher but PP is lower.

Krzysztof

very interesting you have very similar precision %, unfortunately I ditched neural nets all together. One thing I've noticed is that NN is very infamous in getting things right when it got it wrong last time. If your hidden nodes are constantly changing and it got it wrong yesterday use the same nodes for today and see if it matches the prediction of the new nodes. If it doesn't match don't trade, You will trade significantly less but you can expand the number of instruments......so if you have 100 instruments you essentially have a money printing machine.

Pat494 · Jan 25, 2017

joseph1986 said:
very interesting you have very similar precision %, unfortunately I ditched neural nets all together. One thing I've noticed is that NN is very infamous in getting things right when it got it wrong last time. If your hidden nodes are constantly changing and it got it wrong yesterday use the same nodes for today and see if it matches the prediction of the new nodes. If it doesn't match don't trade, You will trade significantly less but you can expand the number of instruments......so if you have 100 instruments you essentially have a money printing machine.

Perhaps you can recommend a free NN to do the job ?

joseph1986 · Jan 25, 2017

Pat494 said:
Perhaps you can recommend a free NN to do the job ?

you can pm me pat for the info, combined cost would be around $4000
for a 6 month license. Made that money back in 2.5 months as a newbie
beats any course dollar for dollar....only problem is that you'll be addicted to a corporate product....which has updates that you need to adjust to once in a while.
On the subject of learning the software, think it would take someone with no statistical background 3 or 4 months to master.....provided you devote countless hours of testing,

Pat494 · Jan 25, 2017

joseph1986 said:
you can pm me pat for the info, combined cost would be around $4000
for a 6 month license. Made that money back in 2.5 months as a newbie
beats any course dollar for dollar....only problem is that you'll be addicted to a corporate product....which has updates that you need to adjust to once in a while.
On the subject of learning the software, think it would take someone with no statistical background 3 or 4 months to master.....provided you devote countless hours of testing,

Thanks
Think I will save my pennies just now.

joseph1986 · Jan 25, 2017

Pat494 said:
Thanks
Think I will save my pennies just now.

yeah, I understand with the price......few years back when I was in college it costs less than $300 now they jacked up the price since they realized its power to dynamically predict consumer behavior. The most I think is one from north carolina went from $169 in 2008 to $18000 last annum.....yearly license.

surfeur · Jan 27, 2017

Krzysztof,

What is your cluster service you use for this intensive calculation ?
Regards

Krzysiaczek99 · Feb 6, 2017

surfeur said:
Krzysztof,

What is your cluster service you use for this intensive calculation ?
Regards

MATLAB pararell toolbox

Krzysztof

surfeur · Feb 9, 2017

Krzysiaczek99 said:
MATLAB pararell toolbox

Krzysztof

Krzysztof, i speak about of your cloud cluster/server

where you rent your server ? have you happy for the service ?

Regards

Krzysiaczek99 · Feb 9, 2017

surfeur said:
Krzysztof, i speak about of your cloud cluster/server where you rent your server ? have you happy for the service ?

Regards

I don't rent, Its my private. I created it from a few old servers, you can buy them really cheap now. Before I was using a few 8core PCs but it was not enough.

Krzysztof

Krzysiaczek99 · Apr 13, 2017

backtest report

Here is a backtest report for a period of 11 months (Apr 2016 - Jan 2017) for 8 symbols. Clearly period July - Oct was a down period for portfolio, however some symbols were profitable. Live test is ongoing. As per today prediction of the system for next days/weeks is strengthening of USD (sell of EURUSD, GBPUSD, buy USDJPY).
Let's see if it will be right.

Krzysztof

Krzysiaczek99 · Apr 22, 2017

MMI filtering

Hello everybody,

Recently I got inspired by post about Market Meanness Index from Financial Hacker. More info about it is here:

http://www.financial-hacker.com/the-market-meanness-index/
http://www.financial-hacker.com/boosting-systems-by-trade-filtering/

As I have big database from my system (>6 millions of trades for lats year) i decided to filter them using MMI. Here is my implementation

Code:

function [profit_tot, possum, negsum, winner, loser, alltrades] = MMISim(lotsize, spread, freq, tradesize, trades, currency, Smooth, Length)
    % for each pair
    % 1) calculate raw MMI    
    % 2) smooth MMI
    
    [startidx, stopidx] = findidxs(trades, currency);
       
    i=1; % index to trade file
       
    alltrades = repmat(trades,1);
    inew = 1; % index to new trade file
    
    rawMMI = calc_rawMMI(currency(:,6), Length);
    MMI = indicators(rawMMI, 'sma', Smooth);
    
    % pSmoothed = indicators(currency, 'sma', Smooth);
       
    for k=startidx:stopidx       
                         
           if round(currency(k,1)+currency(k,2),4) >= round(datenum(trades(i,2)),4) % currency in sync with trade list
           if i < size(trades,1)
             i=i+1;  
           else
             break;
           end
           if (MMI(k)<MMI(k-1)) % MMI is falling
           alltrades(inew,:)= trades(i,:); % copy trade
           inew = inew +1;
           end
           end
                  
    end
    
     % delete superfluus of trade file
        alltrades(inew+1:end,:) = [];
        
        profit_tot = sum(cell2num(alltrades(:,14)))*tradesize*lotsize/freq;
        
        
possum=0;
negsum=0.0001;
winner = 0;
loser = 0;

for ii=1:size(alltrades,1);
            
        if cell2num(alltrades(ii,14))>0
            possum = possum + cell2num(alltrades(ii,14));
            winner = winner + 1;
        end
        if cell2num(alltrades(ii,14))<0
            negsum = negsum + cell2num(alltrades(ii,14));
            loser = loser + 1;
        end;
end
           
     % PFs = abs(sum(possum)/sum(negsum));
     % PPs = sum(winner)/(sum(winner)+sum(loser))*100;   
                
end

function [rawMMI] = calc_rawMMI(Data, Length)

m = movmedian(Data,Length);

  rawMMI = zeros(size(Data,1),1);
for k =size(Data,1):-1:Length+2
       nh=0; nl=0;
  for i=k:-1:k-Length
    if(Data(i) > m(k) && Data(i) > Data(i-1))
      nl=nl+1;
    else if(Data(i) < m(k) && Data(i) < Data(i-1))
      nh=nh+1;
        end
    end
  end
  
  rawMMI(k) = 100.*(nl+nh)/(Length-1);
  
  end

end

and here are the trade files for 8 symbols for 2 different algos. SDAE and Pegassos SVM

Code:

QAtrades_Peg__PF=1.05_Profit=33282336.4_PP=63
QAtrades_SDAE__PF=1.19_Profit=108194086.9_PP=65.4

so initial PF are 1.05 and 1.19.

Here are the results of filtering for different Smooth and Length

Code:

whatifQAtrades_Peg_PF=1.05_Profit=8529502416.6_PP=63.14_freq=1_tradesize=0.01_Smooth=600_Length=50_TP=350392_FP=204589
whatifQAtrades_SDAE_PF=1.2_Profit=23894165020.55_PP=65.63_freq=1_tradesize=0.01_Smooth=600_Length=50_TP=326980_FP=171238

whatifQAtrades_SDAE_PF=1.2_Profit=22116171802.7_PP=65.61_freq=1_tradesize=0.01_Smooth=100_Length=100_TP=330259_FP=173129
whatifQAtrades_SDAE_PF=1.19_Profit=23293250055.9_PP=65.79_freq=1_tradesize=0.01_Smooth=200_Length=100_TP=329397_FP=171282
whatifQAtrades_SDAE_PF=1.18_Profit=22808654859.55_PP=65.71_freq=1_tradesize=0.01_Smooth=300_Length=100_TP=331200_FP=172836
whatifQAtrades_SDAE_PF=1.2_Profit=26104137089.45_PP=65.93_freq=1_tradesize=0.01_Smooth=400_Length=100_TP=333457_FP=172280
whatifQAtrades_SDAE_PF=1.2_Profit=27012799035_PP=65.8_freq=1_tradesize=0.01_Smooth=500_Length=100_TP=332532_FP=172810
whatifQAtrades_SDAE_PF=1.2_Profit=27447013953.5_PP=65.97_freq=1_tradesize=0.01_Smooth=600_Length=100_TP=331464_FP=170994

whatifQAtrades_SDAE_PF=1.2_Profit=22963529441.75_PP=65.72_freq=1_tradesize=0.01_Smooth=100_Length=200_TP=331773_FP=173076
whatifQAtrades_SDAE_PF=1.17_Profit=23790659466.55_PP=65.5_freq=1_tradesize=0.01_Smooth=200_Length=200_TP=336169_FP=177035
whatifQAtrades_SDAE_PF=1.17_Profit=24303021213.8_PP=65.51_freq=1_tradesize=0.01_Smooth=300_Length=200_TP=334265_FP=175962
whatifQAtrades_SDAE_PF=1.18_Profit=25439141483.15_PP=65.54_freq=1_tradesize=0.01_Smooth=400_Length=200_TP=334190_FP=175750
whatifQAtrades_SDAE_PF=1.19_Profit=27360164418.65_PP=65.75_freq=1_tradesize=0.01_Smooth=500_Length=200_TP=337621_FP=175866
whatifQAtrades_SDAE_PF=1.17_Profit=27445995139.35_PP=65.49_freq=1_tradesize=0.01_Smooth=600_Length=200_TP=338051_FP=178145

whatifQAtrades_Peg_PF=1.05_Profit=7349564747_PP=63.13_freq=1_tradesize=0.01_Smooth=600_Length=200_TP=359971_FP=210249

so it seems no big impact on PF, just number of trades cut in half (TP+FP).

as the price data was 1 min but maybe trends are more visible on higher TF i made small modification to simulate 15M TF. I changed

Data(i-1) to Data(i-16) in MMI caclulation function but also not impact....

Code:

whatifQAtrades_Peg_PF=1.06_Profit=6456268137.3_PP=63.63_freq=1_tradesize=0.01_Smooth=600_Length=200_TP=375900_FP=214844
whatifQAtrades_SDAE_PF=1.17_Profit=27481160172.8_PP=65.71_freq=1_tradesize=0.01_Smooth=600_Length=200_TP=349375_FP=182356

So it looks that this method of filtering does not improve or worsen the performance, the timing of filtering seems to be completely random. In next step I will try to make more detailed filtering to see the impact on different symbols, maybe any of them has more 'trendy' characteristics. However this type of selection will introduce higher variance of final result (less trades) and selection bias so it will be risky to say if any improvement in performance is real.

and here is a link to original trades files. If someone has an idea how to filter them to improve their performance let me know. I know already Hidden Markov Models method to switch between regimes but i think this method is not much better than ordinary MA cross.

https://www.mediafire.com/?2bcrqpb3boh23w9

Krzysztof

3rd generation NN, deep learning, deep belief nets and Restricted Boltzmann Machines

Well-known member

Junior member

Well-known member

Junior member

Well-known member

Experienced member

Well-known member

Experienced member

Well-known member

Experienced member

Legendary member

Experienced member

Legendary member

Experienced member

Junior member

Well-known member

Junior member

Well-known member

Well-known member

Attachments

Well-known member