Can mechanical forex systems work?

[rnicoll]

Turns out my system stopped working at 10pm when the markets closed, and didn't restart at 10:15. *sigh*

Meanwhile, just to confuse people, I've attached the results if it paper-trades after each winning trade until after the next losing trade. In short, 89 pips "rescued" (for a profit of 67 pips compared to a loss of 22), taking around half the trades in total.

Now, that's probably not a fantastic plan itself, but strongly suggests that reducing the amount traded by, for example, 10% for each winning trade, and increasing it 15% for each losing trade might significantly improve draw-downs.

 

[Vrothdar]

Sounds like you've got a bit of a martingale going on there *runs scared*

 

[rnicoll]

Sounds like you've got a bit of a martingale going on there *runs scared*

Fairly much, yeah. Haven't had time to properly modify the report generator to do this, but here's where my first stab at a stake-modifying simulation performs. In short, 119 pips at the same time that the regular version was showing a loss last night, and 256 pips vs 45 profit (before commission) when IB's servers went down for reboot last night.

 

[Vrothdar]

Not sure what it is, and I've not got the time to look into it properly at the moment, but something just doesn't look quite right to me with that.

 

[rnicoll]

Not sure what it is, and I've not got the time to look into it properly at the moment, but something just doesn't look quite right to me with that.

Hence "Haven't had time to properly modify the report generator to do this,"

Haven't had time to verify the results yet, but I think these are right, now. Two sheets attached, one was a "bugged" run where the stake was allowed to go negative... shows a massive profit, but keep in mind the stake manages to hit about double what it should be (but negative). The other shows a stabler run with a decent (227 pip) profit at the end of the sheet, compare to 80 from the regular version.

I _think_ I might try something funky with exponentials tonight...

 

[rnicoll]

Quick highlights before I take a break... As of 20:12, showing a 73 pip profit (47.3 after commission), with a day peak of 192.5. With stake management, that shows as 356 pips with a peak of 476.4.

I've finally got the connection recovery code implemented, so hopefully this will now make it through the 5am reboot of IB's systems. That will patch a 3-5 hour gap in the data.

Next priorities:

1. Add code to work out stake based on previous trades and account size.
2. Modify contract manager to recover state from database on startup.
3. Make system startup less user intensive (currently each instrument must be enabled manually)
4. Sleep

I'll do a wrap up post tomorrow evening, or morning if there's any particularly exciting news.

 

[north5]

Just read the article on the turtles posted elsewhere. I was interested in their money management idea basically based on ATR - suggest you give it a quick read as it sounds very sensible and may be of use.

I think I'm going to code up their system in MT4 - looks very suitable for Forex and even if it is no longer relevant I think there are a few things to be learnt doing so. Also gives me a break from my system which is still just a collection of pieces again looking for the glue to bind it together. Gone back a step by mking it an indicator of sorts so I can analyse more easily visully than running it through the market as an EA.

 

[dcraig1]

Next priorities:

1. Modify contract manager to recover state from database on startup.

Is that a good idea ? Is it not safer to recover state via the broker API which is the only authoritive source.

 

[rnicoll]

Is that a good idea ? Is it not safer to recover state via the broker API which is the only authoritive source.

Problem is that with currencies, because trades are in pairs rather than in single things, it's very hard to determine actual position. Even more so if you don't trade directly back in the same pairs. Other places may implement this better, admittedly, but with Interactive Brokers I'm thinking I'm better off reading it from the database and letting the user override in the UI if necessary...

 

[rnicoll]

Just an update

Spreadsheets and start of a PDF report attached. Basically, showing an 81 pip profit, about 40 pips after commission, on the regular version, 400 pips with stake management. There's a PDF showing my attempts to break the results down and see if I can extract anything useful, so far hour of day is most promising looking but I need more data and I need it to have less holes in it.

Connection auto-restart didn't work as expected. More logging, and I suspect being awake at 5am Monday, may be in my future. JOY.

At this rate not going to have the new stake handling code in live trades for next week, so it'll be another week after that before I can get decent results from it. Still, getting there, getting there...

 

[north5]

Yes every step must be seen as a step forward (even if it does feel like a step backwards) or we'd just end up going mad.:cheesy:


Had a quick code up of some of the turtle stuff today. Quick look across a few markets has perked some interest. I'm finding the 55 bar high/lows interesting. They do look like very good points for reversals and breakouts. They look good for most markets too, and on multiple timeframes. So how to use them I guess is the next question. Look good as entries(provided you pick the right direction) and especially exits.

The ATR stop limit also looks interesting - but needs some more looking at.

All in all very glad to have looked into this system - obviously won't be using it as designed, but then I feel this thread is now about finding our own ways and sharing so that is fine here. Will run it as an EA exactly as designed just out of interest though at some point next week.

 

[rnicoll]

Had a quick code up of some of the turtle stuff today. Quick look across a few markets has perked some interest. I'm finding the 55 bar high/lows interesting. They do look like very good points for reversals and breakouts. They look good for most markets too, and on multiple timeframes. So how to use them I guess is the next question. Look good as entries(provided you pick the right direction) and especially exits.

From what I've heard, the turtle stuff has some fantastic theory in it. Haven't had time to read it, but it's on my to-read list.

Couple of things to wrap up the week. Firstly, looking over the charts, I think this definitely wasn't the easiest week to start testing a trend reversal system on, as the GBP/USD for the last 3 months helps demonstrate:

Secondly, I've been mis-calculating profits... and it's a good thing! I've been counting pips, but realised that there's no way I'm going to trading this live with fixed quantities, so percentage profit makes a lot more sense. I've also put some formula into the Excel report generator, so it's easier if others want to play with it afterwards. The result is up at http://jrn.me.uk/trade2win/cmfsw/trades_1218834650.xls

In summary, 0.69% down for the regular version, 2.11% up for the stake modified version, both numbers excluding margin and commission. Commission should be about 0.42%, so that's -1.11% and 1.69% respectively before margin.

My greatest concern at this point is that NinjaTrader shows massive losses across USD/JPY and NZD/JPY which aren't included this week due to problems with the fact the prices are quite different to anything else my strategy deals with, and would normally be included. Having thought about this, what I'm going to do is leave most of the underlying strategy and tested currencies the same for next week, and focus on realiability, logging and backtesting code, which will make development faster later, rather than moving me closer to going live.


Edit: On a completely unrelated note, it's going to be a tough weekend. I've got absolutely nothing at all to do except sleep tomorrow, and Sunday afternoon a friend's asked me to take photos of her pole dancing, then in the evening I'm spending time with my significant complication. :-D

 

[north5]

Just a quick post as it has occured to me that my original scalper used to play at the 6 hour high/low levels - that would be 72 bars on the 5 minute chart which is pretty close to the 55 bar I was looking at yesterday. Definately some playing to do there - and I think it would be worth looking again at my orginal ideas and trying again to find a way to implement them in metatrader and seeing if they combine well with the stuff I have been playing with since.

One other thing from turtles that I think may be of use to you in managing your trades is the correlation - essentially mainatining sensible levels with heavily correlated markets.If you are getting runs of losses at the same time then it suggets to me that you are over exposed in this way.

Hve a good weekend - some of that definately sounds like fun.

 

[rnicoll]

One other thing from turtles that I think may be of use to you in managing your trades is the correlation - essentially mainatining sensible levels with heavily correlated markets.If you are getting runs of losses at the same time then it suggets to me that you are over exposed in this way.

Currency markets are basically all correlation, one moves and it ripples outwards through the markets. This is why reacting to a profit/loss directly is too late, it's already entered half a dozen trades by the time the first exits, instead it needs to react to the previous trades by expecting change-around. It might prove that this isn't sustainable, and that limiting to 1-2 trades at a time in markets with no common currency is a better answer, but we'll see. Oh, and this I think shows the correlation rather well:

Still working on the charts, which is why I don't show them often, but I will start producing the reports with them in more often over time.

 

[rnicoll]

Spent this morning trying to pick out the spread on each trade I make from the data I have. Mostly, discovered that a lot of the data I have cannot be directly matched up, due to offets in the times and sampling issues. So, I'm going to need to rework a lot of the data collection parts of my platform to get this working correctly.

Did get the slippage though, and discovered it's mostly negative, which is nice. Admittadely, this is because I'm grabbing the market as it's moving fast the wrong way, which is also why trending markets wreck my system, but nevermind eh.

Anyway, back to spread; thing is, when I've watched my system I've noticed it can sometimes end up making trades over some insane spreads. 11 pips was one that stuck in my mind, for example. Filtering these out screws with the stats, as a lot of them turn into profitable trades anyway, but it would be nice to get that spread down. A lot of the worst spreads are on lightly traded pairs; EUR/AUD, EUR/CAD, etc. I'm going to be looking at having my platform do a search for lowest cost path for what it's trading, so if it's cheaper to go EUR/USD -> USD/CAD for a EUR/CAD trade, it will. With a little bit of luck, this will save me a good number of pips in spread, which should significantly improve profitability. It also as a side-effect increases my number of trades, which will get me closer to the reduced high-trade-volume commissions IB offers.

On a related note, I'm also looking at whether it would make sense to have it try reducing the number of trades by looking for things it's likely to trade soon. For example, if it wants to trade EUR/AUD and AUD/USD at the same time, it's going to make much more sense to trade EUR/USD only, cancelling out the common currency. It gets harder though if it sees EUR/AUD as trade now, and AUD/USD as trade soon. Does it make sense to have it wait until AUD/USD hits a signal (I think this is most promising)? Or maybe push the AUD/USD trade through early? Or does trying this overcomplicate things and make everything too brittle?

 

[Vrothdar]

For example, if it wants to trade EUR/AUD and AUD/USD at the same time, it's going to make much more sense to trade EUR/USD only, cancelling out the common currency. It gets harder though if it sees EUR/AUD as trade now, and AUD/USD as trade soon. Does it make sense to have it wait until AUD/USD hits a signal (I think this is most promising)? Or maybe push the AUD/USD trade through early? Or does trying this overcomplicate things and make everything too brittle?

I think it'll probably make things too brittle. I take it if it wanted to go long EUR/AUD and long AUD/USD it'd trade them both as you're not going to be able to cancel out the AUD there. Maybe you can get it to work but it's a bit more than I'd be comfortable with myself.

On my own notes. The couple of days away has helped me a fair bit - bit of r&r has helped bring my thoughts back from lightspeed and now I can follow them Going to try running my system on minute bars calculated over a daily time frame (ie 1440 minute bars) to see if that will be enough for it to fine tune it's entries/exits. Will have to wait until later in the week when I've got a copy of excel 07 to try that. In the mean time I'm going to change it to 2 EMAs and 2 others, perhaps RSI and stochs to see if using a variety of types of indicator will help combat the lag.

 

[rnicoll]

Significant complication has fallen asleep on sofa, as she's working too hard, so I'm back on the forums...

I think it'll probably make things too brittle. I take it if it wanted to go long EUR/AUD and long AUD/USD it'd trade them both as you're not going to be able to cancel out the AUD there. Maybe you can get it to work but it's a bit more than I'd be comfortable with myself.

For both long, or both short, it would double the stake... I mean, you are:

Buying EUR with AUD, so you have, say, 100,000 EUR and -100,000 AUD
Buying AUD with USD, so you have 100,000 and -100,000 USD

Mix them together, you get 100,000 EUR/USD equivalent. Now, the exchange rates means the maths is a little trickier, but I hope you can see the basic plan?

On my own notes. The couple of days away has helped me a fair bit - bit of r&r has helped bring my thoughts back from lightspeed and now I can follow them Going to try running my system on minute bars calculated over a daily time frame (ie 1440 minute bars) to see if that will be enough for it to fine tune it's entries/exits. Will have to wait until later in the week when I've got a copy of excel 07 to try that. In the mean time I'm going to change it to 2 EMAs and 2 others, perhaps RSI and stochs to see if using a variety of types of indicator will help combat the lag.

Sounds good, in theory at least more accurate measurements should increase the profitability if it's a good system. If it makes you feel any better, I'm hitting problems too; holding a year's worth of minute bars in memory isn't actually trivial (mostly I'm too used to memory not being an issue, and then suddenly I'm trying to hold a few million numbers in memory and it's suddenly an issue again).

 

[Vrothdar]

Significant complication has fallen asleep on sofa, as she's working too hard, so I'm back on the forums...

For both long, or both short, it would double the stake... I mean, you are:

Buying EUR with AUD, so you have, say, 100,000 EUR and -100,000 AUD
Buying AUD with USD, so you have 100,000 and -100,000 USD

Mix them together, you get 100,000 EUR/USD equivalent. Now, the exchange rates means the maths is a little trickier, but I hope you can see the basic plan?

Oh aye. Crossrates can be fun. Just wanted to make sure you weren't trying to do it with opposite directions! Not sure why I doubted you really.

Sounds good, in theory at least more accurate measurements should increase the profitability if it's a good system. If it makes you feel any better, I'm hitting problems too; holding a year's worth of minute bars in memory isn't actually trivial (mostly I'm too used to memory not being an issue, and then suddenly I'm trying to hold a few million numbers in memory and it's suddenly an issue again).

Sounds painful. Decided last night that I'll have to move my system from formulae in the spreadsheet to VBA if I've any hope of it being usable. If not I'll have to learn to use ninjatrader or something

 

[rnicoll]

Oh aye. Crossrates can be fun. Just wanted to make sure you weren't trying to do it with opposite directions! Not sure why I doubted you really.

I think you can do it in odd directions too, sort of. Okay, say short AUD/USD, long EUR/AUD, that means you want to end up with 100,000 EUR, -200,000 AUD, 100,000 USD. EUR/AUD tends to have a high spread, so shorting AUD/USD 200,000 and then going long 100,000 EUR/USD might actually be cheaper. Going to write the code so the computer knows it all and it can figure it out.

Sounds painful. Decided last night that I'll have to move my system from formulae in the spreadsheet to VBA if I've any hope of it being usable. If not I'll have to learn to use ninjatrader or something

It'll be okay. NinjaTrader isn't that scary, really.

Right now, my system's fairly badly losing money. This leads to one of three possibilities, which are not mutually exclusive:

• The market is out of its normal flow enough to screw it up.
• My code doesn't implement the strategy correctly.
• My strategy doesn't work.

I _think_ it's primarily the first two. Certainly, the results aren't matching NinjaTrader's results for the week, which means that something's up... somewhere. So, what am I doing about it:

• Got a datafeed subscription from IQ Feed to I can get per-tick data as long as I collect it frequently.
• Reworking my trading platform to source from per-tick data, which will allow it to calculate routing costs around different currency pairs.
• Completing backtest & optimisation code in my own platform so I can test my code and see if it's a data problem or a code problem that it differs from NinjaTrader.
• Getting the recovery code to work so I'm not always losing the trades from 5-8am-ish.
• Continuing forward testing to see if the market stabilises out at all, or my system is just screwed up...

So probably going to be a while before I have anything particularly new, now...

 

[rnicoll]

Just a quick update; after the bumpy end-week/Monday, it's beginning to pick itself back up again. The stake modified version is pulling itself slowly upwards, with a small but not insignificant profit on the week, while the original isn't doing so well. Trade list attached.

Have a data feed from IQ Feed... generally good, but NinjaTrader appears to be losing ticks between midnight and 5am. Need to do more tests, see if I can track down exactly why.

My flatmate has been working on backtesting, and has it mostly working, so we'll be able to better see where things differ from the backtests I was doing in NinjaTrader, are, soon. Improved reports and optimisation tools are then next on the list. Hoping to do a source code release this weekend so people can see what I'm working with.