Evaluating strategies

[Belligerent_Drunk]

Hi all,

I am opening this thread because I want to build up a constructive discussion of how people evaluate their trading strategies, wether they are manual or automated strategies.

The reason I bring this up, is that I recently started toying with some automated strategies and want to figure out how I can analyze the results coming out of a backtest better. I am building a spreadsheet for this purpose and I'm looking for improvements. Just curious how you guys tackle this problem.

I've attached snaps of the spreadsheet. It's WIP, since I need to add a whole list of things (including changing winners to green and losers to red in the chart hehe).

 

[malaguti]

Hi all,

I am opening this thread because I want to build up a constructive discussion of how people evaluate their trading strategies, wether they are manual or automated strategies.

The reason I bring this up, is that I recently started toying with some automated strategies and want to figure out how I can analyze the results coming out of a backtest better. I am building a spreadsheet for this purpose and I'm looking for improvements. Just curious how you guys tackle this problem.

I've attached snaps of the spreadsheet. It's WIP, since I need to add a whole list of things (including changing winners to green and losers to red in the chart hehe).

I generally have an overview and all the detail behind it. I dont think there is too much difference in what you are calculating and what I get..
largest win, largest loss, average win/loss etc
Hit rate, length in trade etc are all important, along with largest drawdown

the equity curve itself is often misleading, as you will rarely if ever be able to trade every signal given. Its all dependent on your allocation of capital at the time.
stats of course are also misleading for the same reason above, however it helps at least identify what to look at, and factors affecting these.

not sure if this is what you were after

 

[Belligerent_Drunk]

I generally have an overview and all the detail behind it. I dont think there is too much difference in what you are calculating and what I get..
largest win, largest loss, average win/loss etc
Hit rate, length in trade etc are all important, along with largest drawdown

the equity curve itself is often misleading, as you will rarely if ever be able to trade every signal given. Its all dependent on your allocation of capital at the time.
stats of course are also misleading for the same reason above, however it helps at least identify what to look at, and factors affecting these.

not sure if this is what you were after

Yeah this is kinda what I was looking for. But do you for example look at trading time? To see what times work best for the strategy, or if it perfoms during news releases etc or during low volatility periods.

And do you have certain benchmarks for types of strategies?

 

[malaguti]

Yeah this is kinda what I was looking for. But do you for example look at trading time? To see what times work best for the strategy, or if it perfoms during news releases etc or during low volatility periods.

And do you have certain benchmarks for types of strategies?

actual times, no. with regard to best times for the strategy I would test under up trend only, down only and then build an adaptive trend filter to recognise market and sector state.
also was the relative strength against the broader market

the only benchmark i have is the base strategy without any filters mentioned above
so basically I might start with no trend filter, gradually build components into the strategy
then throw in a neural network to possibly identify other variables.
I rarely use a NN to trade off though, but just identify additional criteria
Also I never optimise

 

[Signalcalc]

What about testing on out of sample and across multiple instruments, if that provides good results then you 'probably' have something.

 

[RicardoRibeiro]

Hi everyone,

Going back to the initial post… it is a very important discussion…. because strategy evaluation is a huge big subject.

Well, here you have my two cents…

I believe that the evaluation should start even before we get the first simulation results. I mean, we need to make sure that the logic behind the strategy is sound and that it has a good chance of holding up in the future. Then we also need to make sure our historical data is complete and clean. You know, the results won’t matter if we start with shaky logic and bad data.

After these initial steps, and still before we even start coding our strategy, we need to focus on real-time data and execution details. Here we need to make sure that it is possible to get the data we need, when we need it, and that it is possible to reasonably execute the trades as soon as we get our signals.

Then we still have to crunch all strategy stats, etc…