What are good sharpe & PF values for a system?

[rafla]

Hi,

What kind of results people are having with ProfitFactor and Sharpe ratio with their system?

Those are current ones from one of mine:

# Profitable 53 (80.3%)
Avg trade duration 2.2 hours
Annual return (compounded) 134.3%
Average win $99
Average loss $275
Profit factor 1.5:1
Correlation w/ S&P -0.226
Sharpe ratio 5.333

(dailysetup.collective2.com)

I think that Profit factor should be more to be succesful?
Back test has shown very high Profit Factor but how much slippage is expected to face at real trades ? Any ideas?

 

[Adamus]

Profit factor and Sharpe Ratio are really one dimensional compared to the info contained in your equity curve, assuming it goes back a decent number of years.

I'm surprised that your Sharpe Ratio is so big considering the % profitable and the ratio of your average win : average loss.

 

[rafla]

Profit factor and Sharpe Ratio are really one dimensional compared to the info contained in your equity curve, assuming it goes back a decent number of years.

I'm surprised that your Sharpe Ratio is so big considering the % profitable and the ratio of your average win : average loss.

As matter of fact I was surprised too. (assumed about level 2-3) And if I check individual trades, quite a lot of those has been really "bad" until positive reached.

But the equity curve is so "easy" to get profitable in backtesting, more like curve fitting but let's see how this evolves in real forward test.

 

[Adamus]

There's another thread on here where we discussed Sharpe Ratio. At the end of the day though you use what works works for you. I don't find Sharpe Ratio works for me.

 

[rafla]

Ok Thanks, so maybe not so much use. More like a theory to compare f. ex. managed funds.

 

[intradaybill]

Hi,

What kind of results people are having with ProfitFactor and Sharpe ratio with their system?

Those are current ones from one of mine:

# Profitable 53 (80.3%)
Avg trade duration 2.2 hours
Annual return (compounded) 134.3%
Average win $99
Average loss $275
Profit factor 1.5:1
Correlation w/ S&P -0.226
Sharpe ratio 5.333

(dailysetup.collective2.com)

I think that Profit factor should be more to be succesful?
Back test has shown very high Profit Factor but how much slippage is expected to face at real trades ? Any ideas?

How many trades? Are those real or simulated results?

It looks to me like a curve-fitted system that has a smooth equity curve so that you get a high Sharpe. How many parameters do you optimize?

I talked a few days ago to Michael Harris, who has developed a proprietary method for testing randomness of results using bootstrap techniques (very interesting guy). He gave me enough information so that I can do my own tests. If you reply with your trade returns, I may be able to calculate the probability that your system is random. One trade return per line would be ok so I can copy and paste to a file.

 

[rafla]

How many trades? Are those real or simulated results?

It looks to me like a curve-fitted system that has a smooth equity curve so that you get a high Sharpe. How many parameters do you optimize?

I talked a few days ago to Michael Harris, who has developed a proprietary method for testing randomness of results using bootstrap techniques (very interesting guy). He gave me enough information so that I can do my own tests. If you reply with your trade returns, I may be able to calculate the probability that your system is random. One trade return per line would be ok so I can copy and paste to a file.

Those trades are real ones and done against coming real time data. The system, which generates those have optimized curve fitting (and of course that timing shows great results) and there are five to six meaningful parameters per system. (some more but those are "solid ones" and not optimized well)
Actually you can download all the trades from the site: dailysetup.collective2.com
Don't be afraid, this is more like of my test system and I can get some statistic from there.

Oops, not directly as I via download but with cut/paste at least.

 

[intradaybill]

I run your numbers through a bootstrap algo to find the distribution of returns and the numbers are amongst the worse I have even seen, I am sorry to say that and I do not mean to disappoint you, .

The p-value is about 0.20. There is high probability your results are random. Good systems show a p-value less than 0.02 and very good systems less than 0.001.

I also sent the file to Michael Harris and he ran his own algorithm to find an even much higher p-value of 0.45. His test indicates that there is high randomness in these returns.

The question is for how long this system will be producing profits before it breaks down. It would be interesting to see. I believe you have been lucky. At least this is what statistical tests indicate.

 

[Adamus]

The p-value test sounds interesting. I assume you know what the test is? I had a look around Harris's website but couldn't find any description of how the bootstrap algo / p-value test work.

 

[rafla]

I run your numbers through a bootstrap algo to find the distribution of returns and the numbers are amongst the worse I have even seen, I am sorry to say that and I do not mean to disappoint you, .

The p-value is about 0.20. There is high probability your results are random. Good systems show a p-value less than 0.02 and very good systems less than 0.001.

I also sent the file to Michael Harris and he ran his own algorithm to find an even much higher p-value of 0.45. His test indicates that there is high randomness in these returns.

The question is for how long this system will be producing profits before it breaks down. It would be interesting to see. I believe you have been lucky. At least this is what statistical tests indicate.

Ok. Thank You. I also had some doubts against ES system as it hasn't produced as good results than DAX system in step forward tests (DAX system has had extremely good real results since september implemented, too short time to judge yet and those trades was not included there).
So there might be a classic problem, too many parameters which optimize back tests nice but no real value... I have to study that p-value as well, if it values distributed systems ok (many instrument, couple of systems, different algo's).
And lot to do again and so little time..

 

[intradaybill]

The p-value test sounds interesting. I assume you know what the test is? I had a look around Harris's website but couldn't find any description of how the bootstrap algo / p-value test work.

HI. Most statistical packages offer this test. Harris probably assumes that trading system developers are familiar with the test (he is probably wrong about that). Here is a tutorial

http://people.revoledu.com/kardi/tutorial/Bootstrap/examples.htm

The wiki entry on this is not good an often confusing:

http://en.wikipedia.org/wiki/Bootstrapping_(statistics)

Here is one for matlab

http://www.mathworks.com/help/toolbox/stats/bootstrp.html

 

[rafla]

Despite of bad p-value the systems is still running OK. Although Sharpe value has gone lower but Profit Factor is higher now:

Trades 79
# Profitable 64 (81.0%)
Avg trade duration 2.4 hours
Annual return (compounded) 143.3%
Average win $103
Average loss $265
Profit factor 1.7:1
Sharpe ratio 3.887

Anyway system will be meshed next week when additional subsytem (DAX) is installed so no more comparative values.

 

[Adamus]

I followed that bootstrapping tutorial which was fairly good but it never mentioned p-values.

Rafla - are those live, simulated or backtest results?

 

[rafla]

I followed that bootstrapping tutorial which was fairly good but it never mentioned p-values.

Rafla - are those live, simulated or backtest results?

Live, no way to fool with those, sometimes 1 or 2 tick slippage (usually against the results) because of the delay between my own broker/collective2.

 

[Adamus]

OK I thought they might be since you only have ~80. How does it compare to your backtest results?

 

[rafla]

OK I thought they might be since you only have ~80. How does it compare to your backtest results?

Backtests shows profit factor over 3 & 3.5 and lower sharpe but that's not realistic to achieve (curve fitting and so on). (ES backtest about 2.3 but good sharpe there.)

 

[Adamus]

I'd comment more but cumulative profit including compounding always stops my brain in its tracks. What compounding do you mean? If you do fixed fractional compounding, you can express the compounding stats with figures like Geometric Average and Geometric profit p.a., and show the true profit without compounding.

 

[rafla]

I'd comment more but cumulative profit including compounding always stops my brain in its tracks. What compounding do you mean? If you do fixed fractional compounding, you can express the compounding stats with figures like Geometric Average and Geometric profit p.a., and show the true profit without compounding.

Me too, newer understand totally the consepts of compounding or how those are estimated, now it was directly from collective2. (not so much use I think as it could be much more if initial account size has been f. ex. 5000)

As mater of fact you can get much more data & analyse directly from dailysetup.collective2.com if you klick the "Analytics" button near Statistic, it will show much more.
That's the one reason to use collective2.

 

[Adamus]

But collective2 doesn't show any backtest results for your system, and who's brave enough to invest in a system with only 6 weeks of results?

 

[rafla]

But collective2 doesn't show any backtest results for your system, and who's brave enough to invest in a system with only 6 weeks of results?

That's not even expected. I suppose that there should be at least 3 months results. But if someone would really be interested about that already I could give him/her some more data directly, although those should be real ones as well. (For me those are available only from september timeframe and this is a an exercise to me to enhance algorithms continuously and current algos can stop working at any time)