my journal 3

This is a discussion on my journal 3 within the Trading Journals forums, part of the Reception category; SNL's second show ever: Watch Saturday Night Live online (TV Show) - on 1Channel | LetMeWatchThis - Season 1 and ...

Reply
 
LinkBack Thread Tools Search this Thread
Old Jan 14, 2012, 8:39pm   #121
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread SNL's second show ever:
Watch Saturday Night Live online (TV Show) - on 1Channel | LetMeWatchThis - Season 1 and Episode 2
SNL Transcripts: Paul Simon: 10/18/75

__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 14, 2012 at 9:17pm.
travis is offline   Reply With Quote
Old Jan 15, 2012, 3:23pm   #122
 
travis's Avatar
Joined Mar 2003
Does it make sense to relativize drawdown?

travis started this thread Today I need to focus on a problem that is easier to solve than I had thought. Since I always keep talking and using the "relativized" version of the trades profits/losses, and therefore relativized drawdown, I need to find out if this makes sense.

First of all, what is the rationale for it and does it make sense?

Rationale for it
The investors argued and convinced me that if CL today is at a value of 100 dollars, but was at 50 dollars a few years ago, a 5 dollar loss when it was at 50 needs to be relativized to be a 10% loss today, which is not 5 dollars but 10 dollars. Otherwise you'd feel safe just because the recent losses were small, and fooled into thinking that a 10% loss can never happen. So by "relativizing drawdown/losses" we mean adjusting, to today's price levels, losses/drawdown that took place at different underlying price levels. I described the process at the end of this post. I used this equation: loss of yesterday is to price of yesterday ("value" of yesterday) what relativized loss (x) is to price of today, therefore relativized loss is equal to loss of yesterday multiplied by the ratio of price of today to price of yesterday:

Click the image to open in full size.

Doubts
This now seems to make sense to me, and, after resisting the idea (because of the amount of work involved) for a while, I accepted and implemented it. But I have always had one doubt: is CL (and the other futures) really likely to fall/rise when it's at 50 as it is likely to rise/fall when it's at 100? I keep wondering if maybe when price is at 50, being so low, it is not as likely to fall, but more likely to rise, and viceversa. Or I wonder if volatility doesn't decrease when price is high. And this doubt is not good. I plan to keep my analyses low, because i've done enough of them, and to clarify all doubts, within the work i've already done.

Another important implication of this is that we are not just risking the overestimation of losses/drawdown. If today's prices are lower than yesterday, the ensuing drawdown will be proportionally decreased. So it really has to be right, because otherwise, we could be causing damage to our drawdown estimates.



Solution
Well, I just woke up and as usual I had the solution for a problem. I found out that it can be easily verified. This is what I'll do. I already have the daily prices of all these futures for all the years of backtesting. Now I just need to find out 1) if the average % range varies as price varies and 2) if the maximum rise/fall varies as % range varies. If everything roughly matches, then I'm ok, otherwise there's a risk of overestimating/underestimating drawdown.

[...]

Here's some results. I've run the test on AUD, and here it is:
aud.jpg

Ok, I was fearing that we might move less at a high price, whereas here it would seem the opposite, that the % change is higher with a higher price. Let's talk about volatility, which is a clearer concept. Volatility seems to be slightly higher at a high price than at a low one. This could be a total coincidence, also because whereas the green-highlighted line seems to agree with this hypothesis, the yellow ones disprove it. Unfortunately we do not have a case to show the opposite situation: great volatility with low prices.

I will verify it on the other futures. Otherwise this would mean that relativizing is not even enough anymore, because a 2% move of CL at 50 would be matched by a 4% move by CL at 100. This would mean that if we move 1 at 50, we don't just move 2 at 100, but 4. So if it were a tendency, then relativization is not only necessary, but it underestimates the real impact of drawdown. What if we were at 50 today and were to relativize the drawdown we saw at 100? In that case, we'd be overestimating it, IF and only IF, prices do move more in % when they're high than when they're low. But it's too early to draw such conclusion.

Let's now go and check CL:
cl.jpg

Once again, some disproving, a lot more than for AUD, but also the confirmation that the really big moves happen at high values (high prices of the underlying asset). Yellow highlighting shows that there were big moves in years when price was low. Green shows that there were small moves in years when price was high, but magenta shows that the highest % moves happened when prices were high, and viceversa.

So this would confirm not only that relativization is good, but that it's not even enough to fully compensate for what happens at higher prices. Instead, in those rare cases when today prices are lower than yesterday, it actually underestimates the drawdown... no, wait: that concern is gone, because if, that's the case, a lower price is accompanied by a relative lower volatility, and therefore it's ok. The problem is only when we're relativizing the other way.

Now I'll check YM, because it could still be a coincidence. If it isn't it could mean that people panic (with a consequence on prices) when prices are at their highest. Of course it could mean a lot of other things. It's all a guesstimate here.

Here's YM:
ym.jpg

This is good. Finally something showing the lowest (relative) volatility at the highest prices (magenta highlighting), the opposite of what I've seen so far. And also high volatility at low prices (yellow highlighting). The only problem remaining is that the highest volatility is shown at the highest prices. And this has been the case so far. Everything else varies and might be due to chance.

Now I'll check ZN:
zn.jpg

Once again, high prices have average volatility (cyan highlighting) and everything can be compared to the low prices, except the usual thing: the highest volatility is found at the highest prices (yellow highlighting). This variety of outcomes doesn't bother my method of relativizing, but it makes me wonder about the nature of this. It's as if, for every future, at the highest prices, there were some sort of acceleration, at the top of that mountain of prices. I think I know when it happens: it's when people get euphoric because prices are high, and then people panic because they think it's falling... there are really big swings at the peak of that mountain. However, I remember this happens at the trough, and there it's easier to lose big % (we're always talking about relative volatility, relative to price). So I don't understand why this doesn't show in my statistic. I would think that when we're at the peak of 100, getting to 110 is just as easy as it is getting to 55 from 50 (same for the falls). But this doesn't show. Maybe the falls are not as steep when we're at the bottom?

Let's check the GC and then I'll stop.

gc.jpg

Yellow highlighting shows that once again tops and bottoms are comparable. But the magenta highlighting shows that, once again, the highest volatility takes place near the highest values of any given period.

However this does not invalidate my relativization method, because, as a rule, we can say that the daily % changes are similar throughout the sample, regardless of where price is. Yes, the highest volatility always happens at the top, but this doesn't invalidate my relativization, because the difference is not so noticeable. When I'll be at the top, I'll be slightly underestimating the losses that took place at the bottom (if they happen at the top, they're likely to be bigger) and at the bottom I'll be slightly overestimating the losses that took place at the top (despite the fact that I am decreasing their value, with the equation shown above).

Summary
Relativization is not just useful but it is necessary, in order to assess drawdown and losses correctly. Indeed, since prices more or less move the same (same relative volatility) at the top as they do at the bottom, if we're at the top we need to increase the value of the losses experienced at the bottom, and if we're at the bottom, we need to decrease the value of the losses experienced at the top.


__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 15, 2012 at 9:17pm.
travis is offline   Reply With Quote
Old Jan 15, 2012, 7:35pm   #123
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread
__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.
travis is offline   Reply With Quote
Old Jan 15, 2012, 10:34pm   #124
 
travis's Avatar
Joined Mar 2003
resampling previous portfolio

travis started this thread After writing this post, I kept thinking about the concept of "resampling", and I want to take it yet one step further.

If resampling makes sense, and I think it does, and if it can help discover an overoptimized portfolio (where coincidentally the wins by one system compensate for the losses by another), then the process should be able to detect the huge mistakes in our previous portfolio (investors' and mine), which was intentionally optimized, because we thought that was the way to find the optimal portfolio (we even used a genetic optimizer, Palisade's RiskOptimizer).

Given that the risk of blowing out (by starting trading at any given date) doubled for my present non-optimized portfolio on both my forward-tested and back-tested sample, I would expect that hyper-optimized portfolio to at least triple (relative to the regular chronological order of trades) after a resampling of the back-tested data. My present non-optimized portfolio went from 5% to 10% (chance of blowing out) for forward-testing and from 11.5% to 23% for back-testing. Which tells me that one way or another I did optimize it, even though not scientifically and not consciously.

Let's see what happens to this one, and if, in fact, resampling can help detect an overoptimized portfolio.

Ok, first of all, step #1, the non-relativized performance we assessed was on a daily basis, and here it is:
Code:
max dd $        max dd days    total profit $    sharpe
-17,879                    28    1,650,783    4.92
Step #2: while we were already trading the 160k portfolio, in August, I was asked to calculate the relativized drawdown, and the relativized performance, always day by day, and I came up with this information:
Code:
max dd $        max dd days    total profit $    sharpe
-26,574                    45    2,220,009    4.42
11k increase in max dd in dollars, 17 days increase in days, increase in absolute profit, and decrease in sharpe ratio.

This relativization already brings us closer to reality, with a 50% increase in max drawdown (both depth and duration) and yes an increased profit, but not enough to compensate for the increased drawdown, so the sharpe ratio gets worse, even though by not much. But even if it stayed the same, this tells us that we need a bigger capital.

Now, after relativizing drawdown, let's see step #3: what happens if we go from a daily timeframe to a trade-by-trade timeframe? This should bring performance a bit down (the higher the timeframe, the more losses tend to be hidden by the profit of profitable systems, and viceversa), and it should bring us yet another step closer to reality.

Code:
max dd $        max dd days    total profit $    sharpe
-30,253                    N/A    2,220,009    N/A
Since, as expected the difference is small, I will now focus on the statistical data that I was gathering yesterday, telling us the probability of blowing out by starting at any given day with x capital (I'll analyze that differently, because, unlike my capital, the investors had a much bigger capital).

So here is step #3 again (relativized drawdown on a trade by trade basis) but with different information:
160k.jpg

The investors were willing to risk all the profit made to date (37k) plus some more. According to this table above, the risk of blowing out, with that available buffer capital, "profit cushion", "uncle point", whatever you want to call it, is zero %. So that portfolio, which failed (or simply got unlucky: you don't have to be wrong to blow out an account) seemed on paper much much safer than what I am trading right now (relatively to the capital available). With the portfolio I am trading right now, according to backtesting, I have an 11.5% chance of blowing out. That one says zero. Go figure. I mean I already knew this, but I hadn't fully realized it. After taking that beating I went and traded what appears to be an even riskier portfolio. Even though I have a feeling that the systems are better and I am certain that my portfolio is less optimized, in that I chose the best systems, and not (via genetic and brute-force optimization) those that fitted together the best. But the statistics aren't showing yet. At all.

[...]

IMPORTANT:
Wait a minute: if the systems in the long run are correlated and by overoptimizing the portfolio I chose a combination that limits but doesn't entirely eliminate this correlation, by resampling I might even get a better result, because if there's anything my resampling does is guaranteeing there's no correlation between any of the systems. So I guess resampling is only good at detecting if I put together a bunch of systems that compensate each other, but it might yield even better results than reality when I could not manage to fit together the systems. The reality is that my optimization is better than reality, but also resampling is better than reality, because in reality systems are correlated. Let's just say that resampling is there to tell us that the systems are at least as bad as resampling, in case we got better results from the datasample available.

Let's move to step #4, and see what resampling (randomizing the trades) does to that "160k" portfolio. If it will give us worse results, it will mean that the portfolio is less curve-fitted than random trades, thereby representing and suffering from the correlation of trades, but still more curve-fitted than what the future is like, and underestimating the correlation of trades and systems. (Or, once again, we might have gotten very unlucky).

Given that for my present portfolio the results of resampling were these:
1) back-testing: % of blowing out from 11.5 to 23%, max drawdown from 10k to 20k.
2) forward-testing: % of blowing out from 6% to 12%, max drawdown from 5k to 10k.

And, given that this 160k portfolio was extensively overoptimized, I would expect the 30k drawdown to more than double, at least to triple. And I would expect the % of blowing out to go from zero % to 20%. Plus of course, as mentioned and explained several times there are many reasons (soccer championship example) why the future is worse than the past (according to my estimates, from previous trading, the future is half as good), and this might explain why we stopped trading exactly at the lowest point of the drawdown, which was 48k, at the end of September (it kept going up ever since).

Here it is, step #4:
160k_resampled.jpg

The max drawdown did not triple but increased only by 30%. And the % probability of blowing out is still zero, given that we waited until 48k (pretty amazing that we stopped trading exactly on the day of the max drawdown), and that according to the resampling summary table, there are no situations where that drawdown would ever be reached, even had the trades be random.

Now the problem is two things: by optimizing the portfolio I managed to make the trades slightly better than random (just 30%), but we know that all futures are correlated, and that creating, on those correlated futures, systems that are not correlated is practically impossible. So, even this resampling, as i said before, only tells us: "hey, you can't expect your systems to do better than me" (i.e. "random trades"). But what I am saying is that even this extra step ("step #4", after #2, relativizing losses, and #3, switching the timeframe) of resampling would not have warned us as to the risks of the portfolio.

The only thing missing is that there's an underperformance relative to the past.

So it's not only every portfolio that does better than random sampling of its back-tested trades is overoptimized (by choosing the systems that fit well together).

We also have to expect an underperformance due to:
1) the systems losing their edge because others start using them
2) the markets changing
3) the survivorship bias or a similar concept (i don't know what it's called), the championship bias. Some systems may be successful just because they got lucky and you found them by trying and trying (despite the out-of-sample methodology, some lucky ones could get through). Other systems may have won the soccer championship but that doesn't mean they'll keep winning: just because you pick the previous winner, it's not automatically the next winner. Things keep changing (I guess this is close to point #2).

For one reason or for another, that I can't explain nor understand fully, my estimate is that my systems, even without optimizing the portfolio perform 50% worse. You expect 50k of profit, and you only get 25k. You expect a drawdown of 37k and you get more, maybe 50k.

Basically, we did overoptimize the portfolio, and we did get very unlucky, but even after taking that into account, to explain the bigger drawdown, we have to recur to this concept of underperformance.

So, what happened to the famous 160k portfolio that we started trading on the 16 of August and stopped trading on the 26 of September? Here it is (this below is the forward-tested sample, which covers a different period from the back-tested sample, on which i based the study and tables until here):
160k_its_story.jpg

And, in light of this, what are my chances of not blowing out?

It depends how we interpret that 50% underperformance. Given that a portfolio cannot expect to have better trades than its trades randomized, we have already brought the relativized max drawdown from 10k to 20k, and the probability of blowing out (with a capital of 4k) from 11.5% to 23%.

Now we should add to that, the certainty of 50% (my estimate) underperformance. This will bring me to about 33% chance of blowing out and 66% of surviving. Also, I am counting on the fact that this time I did not choose the systems based on how well they fit together but on how good they are: I practically chose all the best systems and found out that they fit well together (I probably got rid of one or two that did not fit well, but nothing compared to the previous portfolio, where I used the palisade's RiskOptimizer software).

So, I am risking 2k, and I am going ahead with the experiment, knowing that I have on my side about 66% of probability.



I said before, but it's important so I am just going to repeat this to myself to clarify it. Writing just as I think out loud.

A portfolio's trades cannot be uncorrelated, no matter how many futures you're trading, it's not going to be like rolling dice or flipping coins. So, if resampling the trades turns out to be worse than your back-tested results it means that your sample was lucky or that you made it lucky by choosing only specific systems. Then what counts as correct is the performance on the resampled sample.

If instead the systems turn out to do worse in the back-tested sample than in the resampled one, it means you didn't overoptimize anything, and you should trust the worse result, which the ones on your sample.

On top of all this crap (relativization of profits/losses, measuring trade by trade, resampling), you have to add an underperformance of your systems by about 50%. Only then will you have fair assessment of your future risk with that portfolio and capital.
__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 16, 2012 at 1:33am.
travis is offline   Reply With Quote
Old Jan 16, 2012, 3:27am   #125
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread

__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.
travis is offline   Reply With Quote
Old Jan 16, 2012, 3:41am   #126
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread



__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.
travis is offline   Reply With Quote
Old Jan 16, 2012, 12:06pm   #127
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread Damn, a discretionary trade:
snap1.jpg

One of mine. First one in a year. I couldn't hold it. Today the systems are not trading (US holiday), but some markets are half open, I hadn't slept, I looked at it, and the NG was losing 4% on an early Monday morning, with gap down.

It looked like it was coming back up (going up and down, forming a horizontal support line), so I just went long on it.

This is what happens when I can't sleep and I skip work. Once by staying home for 3 weeks, I more than tripled my account, from 8k to 26k (then one month later I blew out the account). But it's definitely something unhealthy that I want to avoid.

It was stronger than me. It happened. I saw the opportunity, actually I was looking for it. I found it, and it happened: I placed a discretionary trade.

That's why I'll stop doing math for a while, a week or more, stop staying so much at the computer and try to sleep. That's the number one problem I have right now, of those that I can solve.

Now I am up, waiting for price to take off (the famous "it can't fall any lower" trades).

[...]

Forget about it. I closed it (profit of one tick). Let's pretend it never happened.
__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 17, 2012 at 7:37am.
travis is offline   Reply With Quote
Old Jan 17, 2012, 6:58am   #128
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread

MADtv - Wikipedia, the free encyclopedia
Watch MADtv online (TV Show) - download MADtv - on 1Channel | LetMeWatchThis
__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 17, 2012 at 7:32am.
travis is offline   Reply With Quote
Old Jan 17, 2012, 4:04pm   #129
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread

__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.
travis is offline   Reply With Quote
Old Jan 18, 2012, 2:01pm   #130
 
travis's Avatar
Joined Mar 2003
Re: my journal 3

travis started this thread Very related to automated trading:
Watch Moneyball online free 2011 - download Moneyball - LetMeWatchThis



Peter Brand (Character) - Biography

__________________
Read: E.P. Chan, Cogneau - Hubner, Sewell, Tverberg. Search: expected shortfall, Monte Carlo VaR, extreme value theory. Trade.

Last edited by travis; Jan 18, 2012 at 4:13pm.
travis is offline   Reply With Quote
Reply

« bloggin the ES | - »
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Similar Threads
Thread Thread Starter Forum Replies Last Post
my journal Fuzzybid Trading Journals 16 Jul 17, 2011 6:49pm
My Journal for all to see herronryan Trading Journals 44 Aug 26, 2010 5:32pm
A journal arabianights Trading Journals 66 Jan 10, 2010 2:09pm
My little journal. HaloTrader Trading Journals 16 Jun 26, 2009 1:02am

Currently Active Users Viewing This Thread: 4 (0 members and 4 guests)