Backtesting issues

[syusuf66]

Hi all

Hoping you could help me (excuse the length)

I plan to manually backtest a trading strategy on a number of different stock charts, doing this one chart at a time. So for example, I'd scroll back to a certain date on an individual stock chart and scroll through each bar, taking signals and making trades as I would usually. I will start with a realistic amount of capital and use the same money management and risk rules I use in live trading/forward testing. At the end, I will take note of whether or not I made or lost money, by what percentage my capital increased/decreased, winning/losing streaks, biggest drawdowns, etc. I will go through this process with as many individual stocks as I can, and each test will be done over a 10 year period with the same start and end dates each time.

However I have a number of concerns/questions about this approach.

Backtesting individual stocks at a time means I can only be in one position in one stock at a time, but in live trading I will be in multiple positions in multiple stocks. Trading in real life therefore increases the number of false signals/losses I can take in a row, as well as my potential max drawdown. I'm having difficulty understanding how testing a strategy on individual stock charts one at a time, can accurately reflect the same results as live trading multiple stocks at at time, as there will be a lot more trades (and possible losing streaks) in live trading.

Or am I viewing it wrong? Should I be looking at each test on an individual stock chart as a scaled down version of what it would be like to run the system with a full portfolio (multiple positions in multiple stocks at a time) and expect a similar distribution of wins/losses, win ratio, drawdowns, profits, etc, over each individual backtest of a stock, only on a smaller scale with less trades?

Another concern is that backtesting a span of 10 years on a separate stock chart isn't an accurate reflection of the money I'd make/lose after 10 years of trading the strategy as its only one stock, and the number of trades I take during the backtest will significantly lower that it would in 10 years of actual trading?

I feel like I may be taking backtesting too literally and expecting exact reflections of my actual trading with it, when it is not possible. Is the point just to prove a strategy has a positive expectancy before you starting trading in it in live markets? Even if this is the idea, I still fail to understand how I could get any accurate reflection of the how the strategy will perform in real life on a portfolio of stocks, by only testing one stock at a time... especially with regard to testing the potential max drawdown encountered.

As you can see I am quite confused about this and can't seem to find any resource that explicitly addresses this issue. If anyone has any insight, or can send me any resources that address this, that would be great.

Thanks again

 

[seekingTruth16]

You should backtest the portfolio as a whole. If you can backtest for one stock, then you have the trades and their results and their dates. Overlay the results from stock 2, then stock 3 etc. Is there any reason why you can't do this? You're using software to do this, right?

"Backtesting individual stocks at a time means I can only be in one position in one stock at a time"

Only if your rules say so. If you are allowed to risk 1% per stock, with a max portfolio risk of 5% at any time, then you can have 5 trades on at once. You need to define your max portfolio risk, not just your individual trade risk. And yes, that might mean you have to pass on some trades when you reach the max risk.

"I feel like I may be taking backtesting too literally and expecting exact reflections of my actual trading with it"

You're not. It should be as close as possible to how you would trade. You can even make allowances for bad fills, transaction costs etc.

 

[malaguti]

+1 for seeking truth
I once (maybe twice until I started automating) did as you did
I went through one chart at a time
06.01.2002 buy lloyds enter X, stop loss Y, exit X
13.01.2002 buy lloyds blah blah

chart 2
08.01.2002 by BT.A enter X, stop Y, exit Z

do the same for all your charts. Then you collate your individual chart trades into one trade sheet sorted by date. now your capital is applied at each new trade that comes along, so you would be simulating your true capital..quite easy to do in excel with a formula. you can then set conditions that if your capital is exceeded its limit, then no trade is taken etc
its time consuming but it can be done, but i would still automate if i were you

 

[syusuf66]

Ok. Thanks very much guys, that is really helpful.

So judging by what you said, let's assume I do the following:



1. Run an automated backtest on 100 stocks from varying sectors and countries from 10 years of data (2006 to 2016)

2. Take the separate data of all the trades made on each stock chart (entry and exit price) and collate it in one big spreadsheet ordered by the date the trades were entered.

3. Manually go through each trade, taking each one as I would in real life, with realistic capital, money management, commission costs, also being sure to diversify as I usually would, and not exceed my pre-defined capital exposure.

4. At the end, I will take note of draw downs, winning/losing streaks, average winners/losers, etc.

5. I will also take note of my capital increase/decrease at the end, and calculate the compounded annual growth rate.

6. Repeat the process with varying start and end dates, so starting/finishing a month later, and seeing if I get similar results.



Would these results give me a more accurate representation of what to expect when forward testing/live trading the strategy?

 

[malaguti]

Ok. Thanks very much guys, that is really helpful.

So judging by what you said, let's assume I do the following:



1. Run an automated backtest on 100 stocks from varying sectors and countries from 10 years of data (2006 to 2016)

2. Take the separate data of all the trades made on each stock chart (entry and exit price) and collate it in one big spreadsheet ordered by the date the trades were entered.

3. Manually go through each trade, taking each one as I would in real life, with realistic capital, money management, commission costs, also being sure to diversify as I usually would, and not exceed my pre-defined capital exposure.

4. At the end, I will take note of draw downs, winning/losing streaks, average winners/losers, etc.

5. I will also take note of my capital increase/decrease at the end, and calculate the compounded annual growth rate.

6. Repeat the process with varying start and end dates, so starting/finishing a month later, and seeing if I get similar results.



Would these results give me a more accurate representation of what to expect when forward testing/live trading the strategy?

all of the advice so far is based on you MANUALLY backtesting.
if you can run an automated backtest, why do you need a big spreadsheet?

I plan to manually backtest a trading strategy on a number of different stock charts, doing this one chart at a time

 

[syusuf66]

because of a couple of reasons.. (some of which I may be wrong about)

All of the backtesting software I have looked at so far only allows you to backtest a strategy on one separate stock chart at a time, not a portfolio of stocks all at once. Is there software that allows you to backtest a portfolio of stocks all at once, which will go through and take signals chronologically as they come up, whilst also implementing money management according to a maximum capital exposure?

The second reason is that I spreadbet, and calculate my £/Pt based on the position of my stop loss. I looked at pro-real-time backtesting and (naturally) it runs back tests as if you were buying/selling actual stocks, not spreadbetting them. I feel manually backtesting would be more accurate as I can set my £/Pt and define the spread for each market. I also reduce position size in line with capital available, and trail my stops to free up more capital. To be honest, I thought that the amount of code/knowledge required to programme this would be too much for me, and it would be easier to do this manually.

 

[malaguti]

because of a couple of reasons.. (some of which I may be wrong about)

All of the backtesting software I have looked at so far only allows you to backtest a strategy on one separate stock chart at a time, not a portfolio of stocks all at once. Is there software that allows you to backtest a portfolio of stocks all at once, which will go through and take signals chronologically as they come up, whilst also implementing money management according to a maximum capital exposure?

The second reason is that I spreadbet, and calculate my £/Pt based on the position of my stop loss. I looked at pro-real-time backtesting and (naturally) it runs back tests as if you were buying/selling actual stocks, not spreadbetting them. I feel manually backtesting would be more accurate as I can set my £/Pt and define the spread for each market. I also reduce position size in line with capital available, and trail my stops to free up more capital. To be honest, I thought that the amount of code/knowledge required to programme this would be too much for me, and it would be easier to do this manually.

that makes sense..then yes, you would do your suggestion, backtest them all and if you can download and sort by date order, you should be fine

there are loads of backtesting software that can cater for a portfolio. amibroker for one. Not that i use it, but its free and has portfolio level backtesting
anyway, good luck with the (semi) manual approach

 

[syusuf66]

Great. Thanks very much for the help mate. Appreciate it

 

[options-george]

Great. Thanks very much for the help mate. Appreciate it

Hi syusuf66,
I think it's a great approach to engage in backtesting - even if there are going to be differences between live trading and backtesting. There will be things you will discover during the backtesting that you have not even anticipated at this point. It willl be better to find some of these out during testing than in live trading.

That said, I would suggest you take a small sample of live trades with small size prior to starting the backtest (or at the same time). I recommend this, because every time I work on testing a new strategy/new refinement I do now start with some live trades. i find even a small sample of live trades helps me to design the backtesting project more effectively, as there will always be some nuances that I had not thought through.

I don't trade stocks - but focus on currencies and indices - however I spent a lot time on testing. I blogged quite a bit on the testing I have done. You can search by the 'testing' category. Although we are talking different asset classes, it might still give you some food for thought.

Good luck.

 

[syusuf66]

Hi options-george

Thanks for the advice. I am actually running forward testing/live paper trading whilst carrying out my historical backtesting too and it has been helpful.

Going back to the backtesting though.. I had a question. Do you know of any backtesting software which will allow me to export a list of of trades to an excel spreadsheet with the following format:

Stock - Date of entry - Entry Price - Date of Exit - Exit Price
(each trade on a separate line on excel)

I had a look at ProRealTime export feature and the Entry and Exit dates for each trade are divided over 2 lines.

Thanks

 

[options-george]

Hi options-george

Thanks for the advice. I am actually running forward testing/live paper trading whilst carrying out my historical backtesting too and it has been helpful.

Going back to the backtesting though.. I had a question. Do you know of any backtesting software which will allow me to export a list of of trades to an excel spreadsheet with the following format:

Stock - Date of entry - Entry Price - Date of Exit - Exit Price
(each trade on a separate line on excel)

I had a look at ProRealTime export feature and the Entry and Exit dates for each trade are divided over 2 lines.

Thanks

I am not sure whether your backtesting is automated or manual.
If manual, and you are using ForexTester2 (I think it's a one-off license fee of U$200) - then you can export the account history to Excel - and that includes all the data you have mentioned above.

Note with ForexTester you can upload historical prices for any instrument (regardless whether its a currency, a government bond, or a company stock) and then test it. The name is bound to make people think it can only be used for fx testing.