Howard Bandy can teach us a thing or two about good back-testing practice which also incorporates forward-testing too.

Using Amibroker (although I am sure all other good backtesting software does the same) we can easily incorporate a walk-forward optimization as part of our back-tests.

We can set the parameter for optimization, set the in-sample periods and out-of-sample periods and then tabulate the out-of-sample test runs to give us our expected returns when taking the strategy live.

My brother is in the process of testing candlestick patterns so I am lucky enough to have a pre-written strategy that can be used to show an example;

I have started the first In-sample period as 01/01/95 - 01/01/96.

The first out-of-sample period would then be 01/01/96 - 01/01/97.

The next In-sample period would be 01/01/96 - 01/01/97.

The next out-of-sample period would be 01/01/97 - 01/01/98.

And so on.

The parameter that is optimized in-sample is the lookback period for a Rate of Change filter. I.E, we only take signals if the ROC > 0, but we optimize during the in-sample period how long a lookback we use in the ROC calculation during the following out-of-sample period.

The objective function (the metric that we want our optimization to maximise) will be the CAR/MDD.

In summary....Whatever ROC lookback period > 0 produces the best CAR/MDD from the most recent in-sample run, we use in the next out-of-sample run.

At the end of all test runs, we can tabulate the entire out-of-sample results and in doing so we have achieved a simulated walk-forward test on nearly 20 years of data, in less than a couple of hours.

Depending on what the most recent in-sample optimization shows to be the best loockback for ROC, we use those settings to go live.

After a year of live trading, we optimise again and trade the new settings live, and so on until our live results differ greatly from previous out-of-sample runs.

Obviously the above assumes that the out-of-sample results of the walk-forward optimization are profitable. Otherwise, we wouldn't trade the strategy.

It also assumes that you haven't repeated the process multiple times in a bid to find a good system - because each time you do so, the out-of-sample results become more and more 'in-sample'.

If you are interested, the signal that I tested was an inverted hammer if it made a 3 day low and had a green real body, the entry was a break of the inverted hammer high on the following day.

All positions were held for 10 days. Max open positions was 10 and the amount of equity risked per trade was 10%.

Tests were run on a database that included all currently listed stocks and ETFs on the NASDAQ, NYSE or AMEX.

The variable that was optimized in-sample was the ROC(C,X) where X = the number of days lookback for the ROC indicator.

In Amibroker the entire strategy was written as;

////////////////////////////////////////////////

SetOption ("MaxOpenPositions" , 10 );

SetPositionSize (10, spsPercentOfEquity);

SetTradeDelays(0,1,0,0);

LongUpperShadow = H - Max(O,C) > (H - L)*0.67;

rng = abs((C-O)/O);

SmallRealBody = rng < 0.003 AND rng > 0;

DownTrend2 = L < Ref(LLV(L,3),-1);

rngy = H-L;

lowerShadow = Min(O,C) - L;

shaven = lowerShadow < rngy*0.1;

RealBodyGapDown = Max(O,C) < Min(Ref(O,-1),Ref(C,-1));

InvertedHammer = smallRealBody AND shaven AND realBodyGapDown

AND longuppershadow AND downtrend2 AND O < C;

Buysetup = invertedhammer

AND Close > 5

AND MA (C,20) > 1

AND MA(Volume,20) > 100000

AND ROC(C,Optimize("rocLB",10,10,250,10)) > 0;

Sell = 0;

Buystop = Ref(H,-1) + 0.01;

Buy = Ref(Buysetup,-1) AND Cross(H,Buystop);

BuyPrice = Max(Buystop,Open);

ApplyStop (stopTypeNBar,stopModeBars,9,False,3);

PositionScore = 100 + Volume;

/////////////////////////////////////////////////

While there are still inherent dangers with the above approach, (mainly that we might do multiple walk-forward optimizations until we find a profitable strategy out-of-sample - and in doing so muddy the out-of-sample data), I still feel that this is the best approach to back-testing that your average retail trader can implement.

The attached images show the system performance stats and equity curve...Of note is the 59% win-rate and less than 10% average market exposure.

If you are interested...the current best ROC lookback is 160.