Manual backtesting

Mar 23, 2016
14
1
13
#1
Hey guys, I am hoping that some of the members might be able to help me with a bit of a research I am doing.

I am working on a tool for manual backtesting that enables you to observe your trade in the past. What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?
 

dbphoenix

Well-known member
Aug 24, 2003
6,916
1,155
223
#2
Hey guys, I am hoping that some of the members might be able to help me with a bit of a research I am doing.

I am working on a tool for manual backtesting that enables you to observe your trade in the past. What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?
The chief difficulties that newcomers to manual backtesting encounter are (1) not knowing how to observe, (2) not knowing how to formulate a hypothesis that is based on those observations, (3) not knowing how to test the hypothesis. Unless and until one develops these skills, manual backtesting becomes chiefly an exercise in random notation and curve-fitting, and there are all sorts of biases that come into play.

Db
 

options-george

Active member
Jan 1, 2006
485
91
38
London
#4
The chief difficulties that newcomers to manual backtesting encounter are (1) not knowing how to observe, (2) not knowing how to formulate a hypothesis that is based on those observations, (3) not knowing how to test the hypothesis. Unless and until one develops these skills, manual backtesting becomes chiefly an exercise in random notation and curve-fitting, and there are all sorts of biases that come into play.

Db
Hi dbphoenix - thanks for your post - would it be possible for you to elaborate a bit further on your three points? [if you have the time]

I am currently doing a lot of manual backtesting (in ForexTester), and am thinking I could benefit (in terms of better return from my time invested in the testing) from insights you might be willing to share.

Many thanks in advance! :)
 

dbphoenix

Well-known member
Aug 24, 2003
6,916
1,155
223
#5
Hi dbphoenix - thanks for your post - would it be possible for you to elaborate a bit further on your three points? [if you have the time]

I am currently doing a lot of manual backtesting (in ForexTester), and am thinking I could benefit (in terms of better return from my time invested in the testing) from insights you might be willing to share.

Many thanks in advance! :)
The following are from my book, so there are of course references to the book. However, all that can be ignored. The process of observing and formulating & testing hypotheses is the same whether one is interested in price movement, indicators, patterns, whatever. A description of this process begins at the end of page 2 and continues through page 9. If one knows and understands the scientific method, this will be familiar territory. Otherwise, it may provide a means of becoming unstuck.

I'm also attaching something which elaborates on the topic of observation.

There is also Chapter 11 in Trading in the Zone.

Db
 

Attachments

Apr 4, 2016
1,959
96
58
#6
What I would like to know is what are the biggest problems and obstacles that you face regarding manual backtesting right now?
The biggest problem is definitely that it doesn't work. This why there are so many vendors, trainers, and tool sellers. If it works, they wouldn't have to scrape for a living from the beginners. They could just make a living from the market.
 
Jan 1, 2006
485
91
38
London
#8
Thanks a bunch for providing all this info dbphoenix. I'll take a good look at these materials.

:clap:

Hi again dbphoenix. I managed to work my way through your two PDF's as well as Trading in the Zone Chapter 11 during my flight to the US yesterday. Great reading - many thanks again! I think it has helped me to clarify the picture on testing to some degree.
 

options-george

Active member
Jan 1, 2006
485
91
38
London
#9
Bump.

Am in the process of doing a strategy refinement - revolves around the idea of entering trades earlier than I currently do.

The first step in doing that was to come back to this thread and re-read some of the material that dbphoenix posted earlier here. It will help me to stay on the path in terms of testing and developing the refinement.

Thanks again! :)
 

Solas0077

Active member
May 29, 2013
227
11
28
#10
Quantopian has free one minute data where you can program you systems in Python and run backtests.

Also this article touches on the problem of noise in backtests when using different data and timeframes.

They plan to have futures and forex data in future so it's a good idea to start learning the platform.
 
Apr 28, 2018
51
4
8
#12
A good idea is to export 1 minute data from Tradestation and copy into Excel. You now have data going back to 1997 futures ES market with 1 minute open, high, low, close. From this excel sheet you can attempt slot of analysis.

Excel for me works really well especially when volatility modelling.
 
May 30, 2018
6
1
3
Westlake, Louisiana
#13
Home Made Software

Spend 30 year trying to write software like I Did. You can do it in Visual Basic
create a data Type with technicals like RSI And Stochastics associated with each plot
which you can screen at any point. Attach is a screenshot of my option software
which can backtest on about 5 different technical one after another in seconds from 1 to 180 days
 
Apr 28, 2018
51
4
8
#14
Spend 30 year trying to write software like I Did. You can do it in Visual Basic
create a data Type with technicals like RSI And Stochastics associated with each plot
which you can screen at any point. Attach is a screenshot of my option software
which can backtest on about 5 different technical one after another in seconds from 1 to 180 days
Don't do what this guys did, don't waste time developing your own software - there is plenty of retail software which can be used and if you want complex functions they can be created with .NET version of Multicharts or even NinjaTrader.

These guys who want to create their own software do not want to trade and make money, they simply want to feel intellectually validated and create software. Having great software is not going to make you find an edge.

Manual backtesting is not a great idea, the same time could be spent learning to code. However, if you don't want to do that then use excel.