Probabilistic Distribution

TheBramble

Legendary member
Messages
8,394
Likes
1,170
First posted on a Commercial Systems Forum thread and responded to here in the hope of further development.

frugi said:
An example of the gambler's fallacy would be the notion that, given a fair coin: After a long run of heads, it is wise to bet on tails because the law of large numbers says that tails are due to "catch up". Nein! What the law of large numbers actually says is that as the number of flips increase, the proportion of heads and tails will become closer to 50:50. However while the proportion draws closer, the numerical difference between heads and tails flipped tends to increase.
{my emphasis} Frugi, I really don't get that. If the proportion draws closer (to 50:50) how can the difference 'tend to increase', surely it must decrease?


frugi said:
I mention this because I was wondering how this concept could be applied to a trend following (could be a breakout) trading system, mainly in regard to position sizing.

After a number of consecutive losing trades (perhaps approaching the maximum recorded in backtesting) it is tempting to increase the number of contracts until a winner or two appear.
Which would be a surefire way to failure. Position size must remain constant. The possibility of increasing size on losing trades makes it by the same factor of the increase, that much harder to make it up with winners. Reduction in size has exactly the same impact with winning trades, they are by the same proportion that much less useful in covering the normal sized losing trades.


frugi said:
Afer prolonged consolidation the chance of a trend surely grows higher, [...]

Thus I almost suspect that, if following a trend following system, to increase contract size after prolonged drawdown (assuming the added size does not break basic money management principles) can be a wise move, despite seeming to fly in the face of conventional gamblers' wisdom. The obvious difficulty, and this cannot be overstated, is in knowing exactly when to increase, then when to step down again.
While I agree the probability of a trend increases in direct proportion to the length of a congestion (not a drawdown; you wouldn't be trading a trend-based system during a non-trending period in the market) as per my comments above, I wouldn't be risking anything other than standard position size - on any position.


frugi said:
Yet another variable to add to the nocturnal backtests, a dangerous and wrong interpretation of probability, or a tool to be used very occasionally, with utmost discretion?
A really good thought. We sometimes tend to get locked into certain viewpoints (as I am in mine on keeping position/risk size constant), but that one for me just makes sense from a probabilistic perspective and based on the basic math involved. While that make 'make sense' to me and be a 'safe' way to trade, it'll probably mean I'm never going to hit the home runs either.
 
A good post by frugi.

I think a system that increases position size when loosing will make the system more unstable, ie increase it's likelihood of failure and put huge pressure on its creator at very vulnerable times. A break out system is trying to overcome the sideways moves by design anyway.
 
Hi Bramble,

Re proportion v difference, with thanks to Peter Webb (Scripophilist) whose explanation I have shamlessly lifted.

'Imagine that a coin has been spun 100 times, and has landed 60 times head uppermost and 40 times tails, many gamblers will state that tails are now due for a run to get even. There are fancy names for this belief. The theory is called the maturity of chances, and the expected run of tails is known as a 'corrective', which will bring the total of tails eventually equal to the total of heads. The belief is that the 'law' of averages really is a law which states that in the longest of long runs the totals of both heads and tails will eventually become equal.

In fact, the opposite is really the case. As the number of tosses gets larger, the probability is that the percentage of heads or tails thrown gets nearer to 50%, but that the difference between the actual number of heads or tails thrown and the number representing 50% gets larger.

Let us return to our example of 60 heads and 40 tails in 100 spins, and imagine that the next 100 spins result in 56 heads and 44 tails. The 'corrective' has set in, as the percentage of heads has now dropped from 60 per cent to 58 per cent. But there are now 32 more heads than tails, where there were only 20 before. The 'law of averages' follower who backed tails is 12 more tosses to the bad. If the third hundred tosses result in 50 heads and 50 tails, the 'corrective' is still proceeding, as there are now 166 heads in 300 tosses, down to 55-33 per cent, but the tails backer is still 32 tosses behind.

Put another way, we would not be too surprised if after 100 tosses there were 60 per cent heads. We would be astonished if after a million tosses there were still 60 per cent heads, as we would expect the deviation from 50 per cent to be much smaller. Similarly, after 100 tosses, we are not too surprised that the difference between heads and tails is 20. After a million tosses we would be very surprised to find that the difference was not very much larger than 20. '

HTH.

While I agree the probability of a trend increases in direct proportion to the length of a congestion (not a drawdown; you wouldn't be trading a trend-based system during a non-trending period in the market) as per my comments above, I wouldn't be risking anything other than standard position size - on any position.

It is usually the congestion periods that cause drawdown for trend following systems, surely? I trade a TF system every day, because I have no idea in advance whether it will be a trend or a range day. The breakout filter removes some of the most pernicious range days, but obviously bigger ranges will fool it and cause drawdown.

If I was able to restrict my use of TF systems to trend days/periods only I would be very happy! If you are able to do this I would be very interested in purchasing your system. :cheesy:

If you agree that 'the probability of a trend increases in direct proportion of the length of a congestion' (unlike a coin), and you'd had an unusually large number of consecutive losers thanks to said congestion, would probability not favour a small increase in stakes, perhaps adding an extra few % per trade until a winner occurs?

Sure you might back 2 or 3 more losers, but if you keep conservatively increasing till you hit the big winner then immediately (or perhaps incrementally) drop back to original stake they should not matter. The stake increase would have to be calculated so as not to make the system too unstable, as Tuffty says.

That said, I'm not going to do it because my math is not strong enough to work out if, when and by how much to meddle with the staking.

My only point really was that though a system may have a 50% chance of winning on paper, and a series of wins and losses might look exactly like a run of coin flips, it does not actually truly behave like a coin (with unconditional probability of 50%) because it trades in a market (which in some complex fashion must involve conditional probabilties).

As soon as we have conditional probabilties there is surely some way to take advantage of them, much as there are in a hand of poker. But at least in poker you have some idea what all the cards are and the chance of drawing a particular one, or winning a hand given x; whereas in the market these parameters are probably impossible to find and define.

Thanks for your replies anyway. Back to Pascal's triangular drawing board for me :cheesy:
 
Last edited:
I've never understood why it is that the probability of outcomes when tossing a coin seem to have found relevance to the world of investing/trading...I don't see the connection because tossing a coin brings none of the influences to bear that are to be found when a person trades or invests..ha not true what about mechanical systems?..yes, well what about them..who identifies and sets the parameters and inherently introduces theirown bias? ...who can' t help tweaking the parameters from time to time for whatever reason ?...who can't bring themselves to ahere strictly to the parameters set for whatever reason?

To run tests based on tossing a coin is meaningless unless you can also bring to bear the investors/traders bias in the test ,because in reality that is certainly what will be present and influencing his/her outcome....


Cheers
 
To run tests based on tossing a coin is meaningless unless you can also bring to bear the investors/traders bias in the test ,because in reality that is certainly what will be present and influencing his/her outcome....

Assume bias is eliminated, tweaking earns you a beating from Rainier Wolfcastle and you are following a mechanical system to the letter. Further assume that in back and live testing the system wins a constant x% of the time.

Likewise does a coin, but I perceive that there is a subtle difference in their behaviour despite both methods leading to, superficially, the same result (a % of winners that approach a known constant, as number of flips/trades increase). It is this difference I am trying to quantify and exploit, badly I might say. :)
 
Frugi, I have a thought. If you could consider your idea as a completely new system designed to take advantage of long sideways areas of congestion, i.e. it doens't trade any break outs unless it the market has been going sideways for x. In effect it is like lengthenning the breakout parameter of your current system anyway. Any break out system will be fitted to the market(s) its trying to capture, so there will always be drawdowns when the consolidation period is longer than average.

What I think I'm saying is that increasing bet size when the market is going sideways is exactly the same as lengthening the break out parameter.
 
The chance of the next coin flip of coming up tails remains at 50/50 regardless of whether or not tails are in a drawdown or not.

This implies that there are 2 perspectives here:
1/ Each trade is unique and independent from the last
2/ The importance of having a proven edge which you can rely upon in the long term (tails coming back?).

I may be wrong as I only glanced through the thread, but some seem to be confusing short and long term outcomes.

Stats are important in trading. The scary thing is, is that some people on this BB (but not this thread - well not yet anyway) who set them selves up a cosy warm gurus publicly state that the use of stats in trading is useless and there is no place for it. Such people are dangerous and should be ignored at all costs. Especially your own!
 
A filter such as "Only take a BO signal if range of last x hours is <y " could well help eliminate losers and pinpoint the explosive breakouts that follow prolonged tight coiling. Less trades, higher accuracy, no messing with the stakes. An interesting thought, thanks Tuffty.

BBB for the purposes of my system you are spot on: I assume that each trade is unique and independent of the past while trying to maintain an edge by ensuring a long term positive expectancy. Stats are vital I would agree.

However a bit of me has recently come to suspect that not every trade is truly independent of the past as it has been shown time and time again that, for instance, price action following prolonged consolidation tends to be typified by a sharp, strong trend. Thus perhaps a little bit of conditional probability creeps in.
 
Last edited:
To continue my post above,

A break out system is set or optimised to the average length of consolidation period which has a probability distribution. Whether you optimise to this by altering the bet size on long consoidation periods or change the break out parameter has the same affect i.e. moves you up or down that probability distribution.
 
Frugi said 'My only point really was that though a system may have a 50% chance of winning on paper, and a series of wins and losses might look exactly like a run of coin flips, it does not actually truly behave like a coin (with unconditional probability of 50%) because it trades in a market (which in some complex fashion must involve conditional probabilties).'

I think I understand what you mean regarding conditional probabilities. My interpretation of this is if the market has been going sideways for x time, then there is a higher chance of a trend happenning sooner rather than later. Although I've not researched this my guess is that the probability distribution of length of sideways moves will have a fat tail i.e. a sideways move can happen more regularly and longer than what would be predicted if a normal distribution curve is assumed.

How to profit from that .... I'm not sure.
 
frugi said:
If I was able to restrict my use of TF systems to trend days/periods only I would be very happy! If you are able to do this I would be very interested in purchasing your system. :cheesy:
I could send you details on my Trend-O-Find system, but I think I'll keep away from that topic for a while. :rolleyes:


frugi said:
If you agree that 'the probability of a trend increases in direct proportion of the length of a congestion' (unlike a coin), and you'd had an unusually large number of consecutive losers thanks to said congestion, would probability not favour a small increase in stakes, perhaps adding an extra few % per trade until a winner occurs?

Sure you might back 2 or 3 more losers, but if you keep conservatively increasing till you hit the big winner then immediately (or perhaps incrementally) drop back to original stake they should not matter. The stake increase would have to be calculated so as not to make the system too unstable, as Tuffty says.
Isn't this the same as the roulette system (forget the name - Martingale???) where you need an infinite amount of trading capital, a no limit table and an infinite amount of time to play?

Increasing your stake on a losing streak makes no sense. You just make it harder to recoup.

But to get back to your example (shamelessly lifted from Scripto) which I read a number of times - are you saying that with a large enough run of coin tosses they will approximate very closely to 50/50, but not necessarily exactly (in fact, probably never...)? And, at a time in that sequence, especially in the beginning of the series, there could well be a significant imbalance which might not 'rectify' until relatively late in the series?

Because if you are, this is exactly the same as a topic a while back where I provided a calculation of maximum expected consecutive losses for any system - given the W:L probability. The thing is, it is just a probability calculation, it might never occur or a string of double that size could occur right off the bat.

And in relation to the instruments you trade Frugi, do you have an analysis of the %-age trading to non-trading periods over the last X months? And is there any pattern? I tend to use recent market action to help me decide if I'm more likely to have a trend day in any given instrument than by using range techniques. (Damn! There's Trend-O-Find accidentally let out of the bag :cool: )
 
Thanks for the responses Tony and Tuffty. I'm afraid I have little original or useful to add; in fact all I've managed is to confuse myself. :confused:

But to get back to your example (shamelessly lifted from Scripho) which I read a number of times - are you saying that with a large enough run of coin tosses they will approximate very closely to 50/50, but not necessarily exactly (in fact, probably never...)?

Yes. The more tosses, the closer the proportion of heads to tails will draw to 50:50. There is no requirement for the numbers to be exactly equal at any point in the series, and the likelyhood that they will be decreases as the series increases.

And, at a time in that sequence, especially in the beginning of the series, there could well be a significant imbalance which might not 'rectify' until relatively late in the series?

Yes, sort of, but beware of exactly what you mean by "rectify". While the proportion will slowly rectify toward 50:50 (though not in strictly linear fashion), the absolute difference between the number of individual heads and tails need not, indeed almost certainly will not, gravitate toward 0.

Although what happens at infinity? You could either assume the difference in throws would be 0 as the percentages would be infinitely close to 50:50 or that because the number of throws is infinitely large, the infinitely small difference in percentage that necessarily exists between them would still be enough to give a wildly different number of throws. I go with the latter. Does anyone know the answer? Ta.

To restate in the real world, even more patronisingly :)

Say heads take a surprising early lead of 20 after 50 throws (i.e. 35 heads to 15 tails) it is tempting to think tails are due a run ("the corrective") because you know the series tends as a whole towards a 50:50 distribution. But after another couple of million throws, though the proportion will now be much closer to 50:50, the number of total throws will be so big that the difference between heads and tails thrown will probably now be in the hundreds, e.g 999,900:1000100 =1:1.02 = 49.5%:50.5%

Thus backing one side after the other has enjoyed a long run is no better a strategy than backing that side at any point in the series. This was, at least to me, rather counterintuitive, though I wholly accept it as truth.

---

My notional trading strategy is not quite a Martingale because I am not proposing to double the stake with every loser to eventually win my initial stake back (you are right about the infinites required to do this with certainty of success), I am proposing to add to a stake after a selected number of losers, for a selected number of trades, then reduce to original size. The stake is "dynamic",if you like, depending on previous market action, and the idea is to encapsulate an unusually juicy trend in the occasional raised stake period.

Why will this be likely? Because not only have we been consolidating for bl**dy ages but also the stakes will be raised incrementally until there is either a big winner and/or the average pip per trade within the raised stake period exceeds the system average.

The stake increase is designed to happen rarely, pehaps only each time the number of max consecutive losers exceeds the number as derived from mathematical formula, or perhaps the dreaded backtests. This may of course never happen.

Of course if the winner is 4 pips and it occurs after two 30 pip losses I look stupid :cheesy:

Increasing the stake at this point might be a good strategy because, unlike the coin, which knows nothing of its past performance, the chance of a win soon WILL be larger than usual. It will be larger than the average 35% success rate whereas a coin will always be 50%. A consolidation is about to burst, and the win is likely to be bigger than usual too, a factor unconsidered in the coin example.

:eek:


Sorry. The rest of me is shouting "Well if the system wins 35% on average then even after 10 losers there is still only a 35% chance of the next trade being a winner, and the next one, and the....etc. you idiot! Nobody gives a toss if it's a coin or a dartboard, it wins 35% of the time, full stop, and you have just contradicted yourself most ignominiously!"

And as I must certainly concede to you that it is by no means impossible for the expected max losing string of losses to double in practice, albeit a carpaccio-rare "black swan" event, I will not be implementing any form of this stake meddling strategy. Well, perhaps I'll add 1% to my stake after 10 losers until a winner crops up again :) for fun.

For now, a positive expectancy is more than enough to elicit my gratitude.

And in relation to the instruments you trade Frugi, do you have an analysis of the %-age trading to non-trading periods over the last X months? And is there any pattern? I tend to use recent market action to help me decide if I'm more likely to have a trend day in any given instrument than by using range techniques. (Damn! There's Trend-O-Find accidentally let out of the bag

Do you mean "%-age trading to non-trading" or "%-age trending to non-trending"? If it is the latter there is no pattern I can truly discern, despite the best attempts of my brain to find them lurking in the noise. Trends, as a factor of time, probably last about as long as the frequency with which my system wins and that is all I can tell you. What patterns there are tell me weakly when a trend has started and how likely it is to continue, but not when this may happen. I used to think I could predict the timing of moves based on past action - indeed I probably could 50% of the time - but now I don't bother to try.

That is not to say that if I see ES has consolidated in a 3 point range for 74 straight hours that I won't try and buy an explosive upside breakout, but it is to say that since I don't know when or if this will happen until it happens, there is no point in trying to tweak my mechanical systems to take account of it. To do so one must be discretionary at the moment of birth and I don't believe a mechanical system can do this with logic alone.

Anyway Bramble my brain is now all a-scramble and the cats are idly circling, just like my thoughts, so I'd best leave these notions to someone with a better grip on stats and stop trying to tinker with a perfectly adequate brace of systems!

PS I remember that calculation for max expected losses. In fact I think I robbed a formula off poor old Scripophilist again - it enjoyed a similar result to yours and I still use it!
 
Last edited:
Incidentally you may be amused by the Monty Hall problem Bramble. I'm sorry if you are already familiar with it.

You are a contestant on a game show in which you are shown 3 closed doors. Behind one is a car, behind the other two, goats. Assume, for the purposes of this example at least, that you want to win the car. You shout out a door number and it remains closed. The host, who knows what is behind each door, then opens one of the other two doors to reveal a goat. You are then offered the opportunity to change your original choice to the other door if you want to. Should you do it?














Yes you should change. And I'll bet that isn't the intuitive answer :)
 
I would always choose the other door.

Why...? Because I'm incredibly intelligent...well actually, no...it's because I am aware of the MH Problem logic.

It is initially counter-intuitive and indeed the person who first proposed this solution had an awfully hard time from some top mathematicians.

Someone else recently posted on t2w I think that it was easier to conceptualise the solution if you had 100 doors rather than just 3.
 
Last edited:
There are LOADS of links, but this one seems the most succinct.

http://www.comedia.com/hot/monty-answer.html

The pivotal point in this is that Monty Hall (it was a "game show" and Monty was the host) KNOWS what is behind each door and he selects one with the goat behind it after the contestant has chosen.
 
There is an addition to this problem that I heard from a physicist friend where the game show host rather than opening one of the doors simply says as you go to open the original door of your choice,
" Stop do not choose that door, there is a goat behind it, but I may not be telling the truth".
Should you then choose the other door?
 
Not sure I understand.

If the host opens none of the doors then the contestant has a simple 1/3 choice like he did to start with, i.e. he can change to either of the two doors not originally chosen. What the host may say is irrelevant if no doors are opened by him. ?
 
But the host knows the answer so this implies a probablility of whether he is telling the truth or not. Also you have 3 doors 2 with goat 1 with car so if you work it out this probability is not the same for each door and hence across the 3 doors.
 
The last time I remember this being brought up it caused a huge argument from the quants. I will try to get hold of a proof, it came from some research student in my friends group at Cambridge. I do not any longer remember why exactly it is supposed to bias the chance but it was not straight forward.
 
Forgive me if this is a crass comment because I haven't laboured painstakingly over the whole thread, but surely this is what we bridge-players call the "law of restricted choice". Clearly it _must_ be right to change your mind and go for the other one!
 
Top