Deadline June

[Adamus]

1) Deadline (of a few weeks) for everything to be ready to be traded with real money?

It's due to personal problems getting my **** together. Basically I have been trying to do this since 1991. So this New Year just gone, I decided to "just ****ing do it" to use the colloquial. I wrote a project plan, made some estimates, looked at my requirements and set the deadline. Basically my girlfriend is holding a gun to my head

 

[Adamus]

2) ... start testing with intraday data ...

I have thought about it so much, but essentially I decided that getting intra-day data and starting with intra-day systems and leaving EOD stuff behind is just another excuse for me not to get down to it and start trading. As far as I can tell, EOD data is not a hinderance to trading successfully. Plus it's not in my project plan, so I'm not going to do it.

OK if this current stuff I'm working on fails to provide what I'm looking for, then maybe I will look at it. But the data you are using - disktrading I think - I've heard it's not good enough quality to test properly. Do a search on this forum. Maybe I'm wrong, perhaps I'd only need hourly bars and they'd be fine.

Elsewhere, tickdata is massively more expensive but maybe worth it. I could probably afford 2 or 3 markets.

We'll see how it goes these next few days with my latest idea - and even that isn't definitively my last ditch attempt with EOD data.

 

[Adamus]

By the way, if the same system works on different securities, then it's probably good. Personally I feel it's better to focus on quantity than quality, because that search for quality lead me to overoptimizing, producing worthless systems with a lot of parameters, and wasting a few years.

Trying to find something that works in all the markets I'm looking at is something I really wanted to do. I read it somewhere. In Market Wizards or similar. But it cost me months, perhaps years of fruitless searches. The only thing I ever found was really long term trend following that brings big drawdowns that would wipe out an account like mine.

So I've been down the same road of curve-fitting and unsuccessful walk-forwards. But Hope springs eternal in a young man's heart...... :jester: There should be an emoticon for the Holy Grail.

 

[Adamus]

I've read the whole journal and I feel entitled to write one short post with my advice (advice from an unprofitable trader so far).

Almost forgot: thanks for the comments and you're welcome to comment. Anyone is.

 

[Yamato]

I will now reply to your replies, if you don't mind, because I find it stimulating since we're doing very similar things right now. I will only refer to the 2 points you referred to in your replies:

1) I see, the girlfriend... Personally, I'd leave the girlfriend, if she's interfering with my trading. Actually, I told her she could leave me, since she was asking me for money which i needed for trading. And she did (but also for other reasons). I don't know if I could do both things at once: worry about the girlfriend and trading. If she's like... a regular woman, she might leave you one day because you don't have money, and so she might as well leave you immediately, so she won't interfere with your trading. Anyway, this is personal stuff for you, so of course feel free to ignore this point.

2) The longer the timeframe, the lower the transaction costs. But the longer the timeframe, the bigger the drawdown. And since you complained about the drawdown, I do not recommend a daily timeframe, because one needs to find a balance between the two, according to their priorities.

For example, my systems make trades that last from 6 to 48 hours, and are back-tested on a 15-minute timeframe. They have these advantages: they're still backtestable ten years at a time (in terms of computer memory and speed), they don't cause me concerns in terms of transaction costs nor drawdown. WIthin these past 8 months, and a few dozens trades, almost all of them have turned profitable, as expected, since their maximum drawdown lasts a few months. With a daily timeframe, I would have had bigger drawdowns (and different strategies), and you wrote about your drawdown concerns, so that is why I wrote about this.

By the way, very important point: all strategies that are implementable/back-testable on a daily timeframe are also implementable on a 15-minute timeframe, but the opposite is not true. So by using daily data you're renouncing to a big share of possible edges / trading opportunities / trading systems.

Disktrading.com data may not be as perfect as other sources (sure, I've also found problems with it: holes, for example), but why would they create bugs or problems with it on purpose? Whatever problems cancel each other out. Also, there's many ways to take care of the holes in data. Also, whatever problems don't make it better to have no data at all (it only costs 150 dollars for the whole data set).

Systems working on different securities. Yes, there was this quote in market wizards, by Richard Dennis:

How much common behavior is there between markets? Are the patterns of beans similar to the patterns of bonds, or do markets have their own personalities?
I could trade without knowing the name of the market.
So, what you are saying is that patterns in different markets are very similar.
Yes. In our research, if a system doesn't work for both bonds and beans, we don't care about it.

I do just the same.

You said you wasted years trying to do the same and now gave up. Well, the answer is out there. It worked for me, as well as for Richard Dennis. If it doesn't work for you, it probably means you're overoptimzing. Trust me about this: I wasted several years overoptimizing systems.

Also, as I already said, by only testing on a daily timeframe, you're ruling out a big chunk of potential strategies, because whatever is tested on a daily timeframe can also be tested on an hourly timeframe, but hourly strategies cannot be tested on a daily timeframe.

Lastly, sorry for repeating myself over and over again, but this is important: the daily timeframe decreases your trades to an unnecessary extent, since transaction costs stop being an issue at an hourly level, and increases your drawdown, which encourages you to improperly reduce it by overoptimizing (adding parameters, curve-fitting, etc.). And if you overoptimize, the system will not work across securities. Use these principles: accept a low edge, few parameters, decrease your timeframe. The fact that it doesn't work across bonds and beans, as richard dennis says, it means there's something wrong with it.

In my case of course I never tested my systems on beans, and you shouldn't either. But what I did find to be true is that good systems - without any changes - were profitable with EUR, GBP, JPY. You can then add some slight variations to make it perfect for each currency (e.g.: different volatility sometimes, different times of the day on the JPY, due to the Japanese timezone). Also, good systems worked for CL, ES, YM and even GC. And the same system should work for GBL and ZN. And finally, ideally, the same system should be profitable on all these futures I mentioned. This objective and knowledge (that it can be done) necessarily forces you to use a few good parameters, it forces you to simplify, and to find common denominators.

I am done with hijacking your thread. Obviously do as you prefer. I won't bother you again with these same concepts.

 

[Adamus]

Food for thought. Thanks.

 

[Yamato]

You're welcome.

 

[Adamus]

Travis,

I just re-read your post because you edited it after I had first read it - I don''t think changed anything, did you?

I agree with the points you make, about trading the system on different markets, about the timeframe correlation to the drawdown, about parameters.

I don't get what you mean about the time frame and transaction costs - this bit:

Lastly, sorry for repeating myself over and over again, but this is important: the daily timeframe decreases your trades to an unnecessary extent, since transaction costs stop being an issue at an hourly level, and increases your drawdown, which encourages you to improperly reduce it by overoptimizing (adding parameters, curve-fitting, etc.).

I can see as you said before that the drawdown for a day TF is greater than for hour TF, but what do you mean about transaction costs?

After pondering the options I have again, I decided I'll carry on developing a trading system using daily data for the moment, since you say I can transfer it up to the hourly TF, I won't be missing out if I decide to go that route. I have to make a project plan together to see how much work it would be.

 

[Yamato]

Yeah, I might have changed some spelling mistake, or added a line or two. Sorry, but I can't remember now. Certainly I never do anything unfair like deleting something people are referring to.

Regarding your question. Transaction costs depend on the timeframe in the following sense. Because if your trades last a few weeks, most likely they will have a profit (or loss) of say 5000 dollars on a typical future, and that totally takes any transaction costs out of the picture, since what is 10 dollars compared to 5000 dollars? Of course I am always referring to futures, because that's what I trade.

Now, if such a long-term system is profitable, it will be profitable regardless of those little expenses. But then what I added is that you can pretty much ignore transaction costs also at lower timeframes like 6 hour long trades, which is close to my kind of trading. And so I concluded that I'd understand if you did daily trades (as you said you're doing) to reduce transaction costs, but that, beyond a certain point, you would be losing the advantages you get from a more frequent trading, such as a lower drawdown. And this works like this: if you have two identical systems, that win 2 times out of three, and one is on a daily level and one on a 1-minute level, you will agree that the drawdown is lower on the 1-minute level, and - except for transaction costs - the lower timeframe guarantees a smaller drawdown and is therefore more convenient. (By time, I simply mean the time price has to move around: of course if price didn't move, all the time in the world wouldn't make a difference). Unfortunately transaction costs keep us from making any money on the 15 seconds trades, or else we'd all be trading those timeframes, and not only we wouldn't have an unprofitable day, but not even an unprofitable hour (given we have a profitable strategy and enough speed to seize trading opportunities, and everything else, you get my point).

As far as what I mean by "transaction costs", I gave the exact definition on this post:
http://www.trade2win.com/boards/trading-journals/85510-my-journal-2-a-53.html#post1083370

Regarding the "transfering" ("since you say I can transfer it up to the hourly TF"), maybe I meant to say the opposite of what you understood, or maybe I am not understanding correctly your sentence. If you want to test any type of daily system, you'll be able to test it on hourly data as well. But if you want to test hourly systems or 15-minute systems (like mine), you can't test them on daily data. So I would definitely move to the 15-minute timeframe, and, as i said before, I would advise to buy the data in case you still don't have it. I wasted years and thousands before I decided to spend 100 dollars to get the right data. Not lower than 15-minute because then your computer can't test things quickly enough (at least on tradestation). Besides, as I said, lower timeframes imply "bigger" transaction costs (in percentage terms with regards to your trades: you can't have an average win of 20 dollars because transaction costs are close to half of it), so I would simply rule them out, and stick to the 15-minute timeframe.

Having lectured like a professor for so long on this post (I hate myself for doing that), I will stress out once again that I am down 50 thousands dollars since I started trading in 1997.

 

[Adamus]

OK I'm with you again regarding transaction costs. Must have had temporary cerebral aphasia when I wrote that.

My transaction costs were a big deal up until last week because I was working on an end-of-day data system which traded break-outs and closed out the positions market-on-close, generating a lot of trades. I ditched the whole system though because the returns were so wildly random, always bunched up with long drawdowns. I might look at it again but I found a bug which meant all the testing I'd done was junk - plus curve-fitted. Great concept - exceptionally poor execution :jester:

For more of that kind of crap with software bugs, see my next post too.

 

[Adamus]

Back on the straight and narrow

I held my own self-enforced marathon bug-fixing / bug-work-around Great Leap Forward.

CRB Trader / Barchart.com let me down pretty badly on the historical data front. I upgraded their continuous contracts software, and they had introduced 2 bugs which took ages to figure out. With the help or some might say hinderance of their support, I spent about a day all in all before coming to the conclusion that upgrading their software opened a can of worms. Work-arounds now in place but I need a new supplier.

There is strong possibility I'll just chuck out the whole EOD data approach, buy some tick data and move onto hourly bars. This has to fit into the June deadline though and in terms of the set-up to get hourly data working with my software, there will be blood

Second problem - my TradeStation code. I discovered the high frequency systems were losing some trades when I output the data into my Java app, which meant I had totally overexaggerated the profitability of those systems. I worked out what the Easy Language solution was and sorted it out, but it should be called "Easy ****-up" instead.

Third problem - my own risk analysis software is riddled with untested classes and methods which I bashed out in a hurry, and just this last week while all this other **** was flying, 2 bugs came out of the woodwork to haunt me. Managed to isolate them and now have got back to business.

I'm testing a trend following system now, using limit entries on pull-backs and trailing stops, with filters to stop trading at bad times. So far so good.

 

[Adamus]

9 weeks to go - ouch: single figure

Ploughed my way through about 30 different trading systems on 25 markets on 10 years of data up to 2008-06-30 and came across a good one. This is the result showing the short-only system on the left, the long system on the right. The data is # trades and profit per trade. The colour scheme is just to make it pretty - red means the system didn't even turn a profit in that market before transaction costs, orange means was there was a gross profit but not enough to cover even half transaction costs, yellow means the profits covered 50% to 100% of transaction costs, green means it was net profitable.

Should anyone think the returns across the different markets compare strangely, it's a weighted portfolio. 1 DAX contract, 10 Eurodollar contracts etc.

The data at the bottom of the table after the colour is the aggregate. First line: # markets brutto profitable - half profitable - net profitable. Second line: total profit. 3rd line: std dev of returns. 4th line: avg trades per market / avg profit per market.

Funny how on all my trading systems so far today (all the same rough basis) the short side of the markets are massively worse than the long side.

So now I have a profitable optimisation / trading system, I stopped searching for anything better so that I can test all the latest changes to my software set-up including the continuous contracts, the TradeStation file output, the Java software.

Lucky I decided to test.

Here's the equity curve for the system on all markets using the same dates as for the table above:

And the stats:

and the drawdowns

Slight problem: it makes a loss. But the mini-analysis I did in the first table above shows it's profitable - making 1.15M long and losing 866K short. So why is the full analysis here showing a loss?

Nice - not another bug somewhere. But anyway, not to be put off, I ran the walk-forward, fully expecting it to show a loss and hint at curve fitting again, but no - a positive sign:

And all together up to 2010:

Better get that bug fixed and then maybe see if I can come up with some more profitable systems that also walk-forwards.

 

[Adamus]

interesting but massive account required

Optimisation:

Max drawdown: 0.5 million
Max time in drawdown: 932 days

The optimisation process seemed to run quite smoothly, in terms of changing parameters and seeing returns increase and increase to a peak and then decrease.

The system is still essentially entering on a pullback in a trend defined by a simple MA, using limit orders set about 0.5 of the 50 day ATR from the close, and exiting on a trailing stop.

I optimised the length of the MA and the ATR, and the factors of the ATR for limits and for stops. Plus I tried out lots of variations of ATR e.g. true range, or high to low, or Max(High - Close, Close - Low) etc etc

This is the walkforward - look at it go in 2008. Like a rocket. Completely blows the whole idea that it might be representative or normal.

The market breakdown is quite interesting - optimisation on the right, walkforward on the left:

Quite a few markets were hopeless in the optimisation but made huge profits in the walkforward. Leads me to think the whole walkforward result is purely random and doesn't lend any credibility to the longevity or future of the system.

 

[Adamus]

To make sure that the walk-forwards are not just profitable by chance, I have to do a certain significant number of optimisations and walk-forwards

If I just did one optimisation and one walk-forward, just because the walk-forward is positive, this lends no guarantee that it's not curve-fit. So what's a significant number that I should do, to prove to myself that I'm not curve-fitting?

Secondly, any trading system based around the one I just tried is quite likely to have a fantastic couple of months in 2008 which make the walk-forward ultra profitable. But if I take the rest of the walk-forward, the system doesn't look good.

 

[Adamus]

thinking out loud

One idea is to backtest over the year 2009-03-01 to 2010-02-28 and then see how the best performing system performs over the whole 1996 - present. That backtest is just too short though and is bound to be curve-fit.

My backtesting and system building process probably needs adapting.

Currently I write the system in Easy Language in TradeStation and then run it in the 'ChartScanner' across my basket of 25 futures. This was designed by the TradeStation people to identify orders that need to be placed after the last bar. I put some code in my system to write the system performance to a batch file which I import into my own java software and look at the results for the basket.

I've got one batch file format which just outputs the number of trades and the total profit per market, which is quick and dirty, which I use most of the time. I've got another output format which outputs all the trades. That takes longer to run and longer to import, but it gives me detailed stats for the whole basket of markets.

So using the quick and dirty scan, I can backtest for total profit and this gets me in the situation I am in now with massive profits for 2008 and pretty average performance for the rest of the 14 years.

 

[Yamato]

market wizards: larry hite

Regarding some things I told you (build them simple, build many of them), here's a great excerpt from market wizards. A question asked to Larry Hite. I will highlight in red the parts I agree the most with.

There are many money managers who use trend-following systems—and quite a few of those don't second-guess their own systems. What makes Mint different? How have you been able to achieve return/risk ratios far above the industry average?

Because we know that we don't know. No matter what information you have, no matter what you are doing, you can be wrong. I have a friend who has amassed a fortune in excess of $100 million. He taught me two basic lessons. First, if you never bet your lifestyle, from a trading standpoint, nothing bad will ever happen to you. Second, if you know what the worst possible outcome is, it gives you tremendous freedom. The truth is that, while you can't quantify reward, you can quantify risk. I will give you an example how important this advice is. One of the world's largest coffee traders invited me to his house in London. When I walked into his library, I noticed he had just about every book ever written on power. He took me to one of the finest restaurants I have ever been at. At dinner, he asked me, "Larry, how can you know more about coffee than me? I am the largest trader in the world. I know where the boats are; I know the ministers." "You are right," I answered, "I don't know anything about coffee. In fact, I don't even drink it." "How do you trade it then?" he asked. I told him, " I just look at the risk." Well this great meal lasted for several hours. Five times he asked me what I did, and five times I told him that I managed the risk.
Three months later I heard that he had blown $100 million in the coffee market. He obviously didn't get the message. And you want to know something? He does know more about coffee than I do. But the point is, he didn't look at the risk.
So the very first rule we live by at Mint is: Never risk more than 1 percent of total equity on any trade. By only risking 1 percent, I am indifferent to any individual trade. Keeping your risk small and constant is absolutely critical. For example, one manager I know had a large account that withdrew half the money he was trading. Instead of cutting his position size in half, this manager kept trading the same number of contracts. Eventually, that half of the original money became 10 percent of the money. Risk is a no-fooling-around game; it does not allow for mistakes. If you do not manage the risk, eventually they will carry you out.
The second thing we do at Mint is that we always follow the trends and we never deviate from our methods. In fact, we have a written agreement that none of us can ever countermand our system. The trades are all the same. That is the reason why we have never had a bad trade at Mint. There are really four kinds of trades or bets: good bets, bad bets, winning bets, and losing bets. Most people think that a losing trade was a bad bet. That is absolutely wrong. You can lose money even on a good bet. If the odds on a bet are 50/50 and the payoff is $2 versus a $1 risk, that is a good bet even if you lose. The important point is that if you do enough of those trades or bets, eventually you have to come out ahead.
The third thing we do to reduce risk is diversify. We diversify in two ways. First, we probably trade more markets worldwide than any other money manager. Second, we don't just use a single best system. To provide balance, we use lots of different systems ranging from short term to long term. Some of these systems may not be that good by themselves, but we really don't care; that is not what they are there for.
The fourth thing Mint does to manage risk is track volatility. When the volatility of a market becomes so great that it adversely skews the expected return/risk ratio, we will stop trading that market.
Essentially, our approach has three lights in determining the acceptance of trading signals. When the light is green, we take all signals. When the light is yellow, we will liquidate an existing position on a signal, but we will not put on a new position. Finally, when the light is red, we liquidate existing positions automatically, and we do not take any new positions.
For example, in 1986, when coffee went from $1.30 to $2.80 and back to $1.00, we got out of our long positions on the way up at $1.70 and didn't trade the market for the rest of the price climb and subsequent collapse. Now, while we may have lost some additional profits, being out of markets like that is one of the ways we are able to achieve such rigid risk control.

Here's an interesting interview by his partner, Peter Matthews:
http://www.opalesque.com/OFI133/Founding_Father_Peter_Matthews_the_developer_of791.html

 

[Adamus]

That's an interesting article. I haven't come across him before. Wouldn't mind having a chat with him myself.

 

[Adamus]

One day I will get down to writing the Sharpe Ratio for this to see whether it is really as good as an equity curve for showing risk.

The following are the equity curves from all the individual markets that make up that equity curve above

Aussie $

German Bund

Canadian $

Cotton

Dax

Euro future

Euro / Yen cross

Eurodollar

So which of those would I trade on its own, as a theoretical question?

The big question now for me is - can I come up with something profitable with each individual marktet, without curve fitting? 25 tweaks on the basic system, and then 25 walkfowrds.

Something wrong with coffee. Must have added way too much commission.

 

[Adamus]

Put that last message up late when I was tired, hence the mislabelling of the charts.

Now I'm looking to optimise the system for individual markets. This is the best optimisation I could come up with for the Australian Dollar:

I spent an hour trying out lots of different things. In the end, I found this, and although the equity curve looks horrible, at least it conforms to my optimisation rules.

Since I already know that the core system walks forward with massive profits thro 2008 - 2009, I felt it was highly likely that this would walk-forward with profit too.

And so it did.

This whole optimisation / walk-forward thing is a bloody art form.

This system walks forward profitably on average for the basket of 25 futures. That's 250 years of data. I'm pretty sure I'd need a fairly complicated system to curve-fit on that much data.

My system is simple, but is it simple enough? It has these parameters or code snippets (where choice of code snippet is more or less a kind of parameter, a system tweak):

(1) I define a price range, which could be based on the true range, the overlap of one bar to the next, the distance of one close to the next etc.

(2) the range is the rolling average of whichever measure I use above, normally between 3 and 55. This is the second parameter - number of bars in the rolling average.

(3) third parameter is a factor that I multiply the range by to set the distance from yesterday's close of the upper and lower limit orders

(4) fourth parameter is the factor that I multiply the range by to set the stop. e.g. 2 * the range from the entry, and then it trails.

(5) fifth parameter: a filter, e.g. 13 day MA must be rising to allow new long positions

That's it - 5 degrees of freedom gone. Probably enough to curve-fit 10 years of data, which is a shame because I need to optimise the system on each market.

I've been racking my brains trying to work out how to simplify it even more, but anything I think of to make it simpler just seems 'random'. The simplest a system could be is to have a single-degree-of-freedom entry method and another for the exit method.

So I could reduce my system here. I could get rid of the filter (5) - that would generate lots of trades, in fact it would probably be 'always in', just reversing.

Or I could ditch the stop (4) - and just exit the trade market-on-close.

Maybe I could replace (1), (2) and (3) with a simpler mechanism.

So the goal is to avoid curve-fitting, but the room for manoevre is tight and I'll have to stick rigidly to the straight and narrow to avoid a repeat of my earlier curve-fitting episode.

Thanks go to travis for reminding me of this. It's so easy to add another little tweak, and waste another day....

 

[Adamus]

My goal for today is to try to walkforward optimisations for all these markets with the same trading system, hopefully controlling the tendency to curve-fit by 'freezing' some of those parameters, just pretending that they aren't there.

I think I'll freeze the moving avg filter which controls direction of entry. I could have tried all sorts of different things - the length of the MA, simple, exponential, adaptive etc, basing it on the close of the bar, or the true median price, etc.

I'll freeze the length of the rolling average of the price range - just use the length that I used on the whole basket of markets.

That leaves me with 3. I'd like to freeze another, but I'm finding it incredibly difficult to work out which one.