Best Thread Algorithmic trading experiences

[blackcab]

After wasting years developing with wealthlab, tradestation, metatrader, and a bunch of other stuff I have to agree writing your own solution (for system design purposes at least) is infinately more sensible in every concievable way (other than time and/or money required !)

Commercial products are great for checking out simple ideas quickly, unfortunately their development is driven by the requirements of their users, and most of the users of these tools dont really know what they want, which pretty much results in tools that dont really do very much.

If you need database support, or AI, or specialist statistical analysis techniques, its simple enough in most development environments, try doing it with a commercial trading application and you invariably end up with a clunky fudged solution.

If I where starting from scratch, I'd probably go down the open source Java route (rather than .NET which was the easiest and quickest solution based on experience), anything that involves money is not to be trusted to a microsoft operating system (and unfortunately I have the scars to prove it )

That was my hunch as I was starting out with this - I'm glad I followed my intuition and didn't waste time with Tradestation etc.

'Waste' may be too strong a word, I'm sure they're good for prototyping ideas as you say, but to me it seemed like writing a video game with a 'game creator' cut & paste type tool versus programming down to the metal in assembler or C. (You can tell my age ;-)

I agree Java would be a great option for today. I never made it into the OO age - completely repulsive to me - so all my stuff's in C. It's fast, if nothing else. AFAIK the pros all use C++.

 

[blackcab]

So - what is it exactly that you can't do that makes you want to write all that other stuff ?

For me, developing/finding systems using whatever technique I want - simple or complex - as opposed to being limited to a macro language. One example of many is genetic programming.

 

[blackcab]

A quick question for those people who say you should only forward test, not back test. Why? It comes up regularly and I can't see the logic of it.

 

[pedro01]

For me, developing/finding systems using whatever technique I want - simple or complex - as opposed to being limited to a macro language. One example of many is genetic programming.

So how many systems have you developed using genetic programming ? How are they working for you ?

I think the issue here is we have a lot of grail seekers who don't have a trading methocology to follow. Genetic programming isn't going to give you an execution platform, you'd use it to 'discover' new trading theories, although I really think the whole thing would be a waste of time. Most likely you would end up with the ultimate curve fitted system doing this.

The trading system I am implementing with my 3 partners is complex yet utterly implementable with Tradestation script (basically Fortran) plus a DLL plug in to handle a few things Tradestation doesn't do well. We still have a few features to put in to phase 1 and we have a load of features slated for phase 2 but you need to have a cut off point somewhere.

 

[pedro01]

A quick question for those people who say you should only forward test, not back test. Why? It comes up regularly and I can't see the logic of it.

There is no reason to do only one but the danger with backtesting is that you end up curve fitting the parameters of your system to fit prior data. Also - market conditions change over time and a system that worked last year may not work this year.

Some systems are too complex to backtest but a very simple system (where all the decisions are made based on the action of a single symbol) will be very easy to backtest.

You can get books on system design and some I have read did a good job of telling you what to avoid in backtesting. Buggered if I can remember which book it was now but you could have a thread just covering those issues.

One thing I would say is that any system should not be left to blindly run regardless of fundamentals. Let's say that you have a system that is trading US equities based on technical signals. On Friday, the US employment figures will be released. At the time before that happens & some time after, you would be better off to have your system switched off (or not making new trades at least) as there will be some price movement that won't make sense to your system. Of course you may have a system specifically to take advantage of such times but generally speaking a system based purely on technical analysis may fail on days where the fundamentals are driving the market as opposed to the technicals.

If you subscribe to that opinion then it follows that your backtesting will not take into account non-technical days.

 

[robster970]

So. My little ATS has been busily pretending to buy and sell this week using FTSE. It's working out profit factor, average win, average loss, win ratio.

Over what number of trades do people tend to run new strategies to get statistically meaningful results?

I think this might have got lost in the noise. Can somebody guide the newbie?

 

[pedro01]

I think this might have got lost in the noise. Can somebody guide the newbie?

It's hard to say without knowing a little more about the system itself.

Can you explain a bit about the system ? speak generally, you don't have to give us the rules - just give us a flavor of what it does.

Also - do you have historical data on which to test this ?

For our system, I'll probably leave it for 3 months of live testing before I commit funds to it but it depends on the performance to some extent. I may 'babysit' the system before the 3 months is up.

Also - give us some idea of the results of testing so far - how long tested, how many trades, % profitable, drawdowns etc.

 

[robster970]

It's hard to say without knowing a little more about the system itself.

Can you explain a bit about the system ? speak generally, you don't have to give us the rules - just give us a flavor of what it does.

Also - do you have historical data on which to test this ?

For our system, I'll probably leave it for 3 months of live testing before I commit funds to it but it depends on the performance to some extent. I may 'babysit' the system before the 3 months is up.

Also - give us some idea of the results of testing so far - how long tested, how many trades, % profitable, drawdowns etc.

It's been running with basic strategies for 1 week now. It is essentially intraday trading on FTSE liquid stocks. There are 4 strategies which are combinations on trend breakout, moving average crosses/direction and price movement. Nothing complicated - just sticking to price, trend and average right now. Works on a tick being 5 mins.

Stats so far

Your current position is £ 20,428.00. This is growth of 2.14% and down on yesterdays trading.

My R is 42.42% with 14 out of 33 trades profitable. PF is 3.48.
The average loss is £ -9.49. The average gain is £ 44.87.
The worst loss is £ -22.84. The biggest profit is £ 155.1.


I can see that my PF ranges between 3-4 already. Losses are small because stops are tight in relation to the price paid. Also because it's intraday/day I can't afford to be sloppy with capital.

 

[pedro01]

Sounds like a simple system that could be backtested then. Obviously a weeks data is a very small sample size.

Are all of these results based on a single stock or do you have a number of stocks the system is watching ?

First thing that comes to mind is the trend in higher timeframes, you may find that your system worked well in that week as it was trading in the direction of the overall trend. Also note that it was a very thin week.

Do you have access to historical data to backtest the system ?

 

[robster970]

Sounds like a simple system that could be backtested then. Obviously a weeks data is a very small sample size.

Are all of these results based on a single stock or do you have a number of stocks the system is watching ?

First thing that comes to mind is the trend in higher timeframes, you may find that your system worked well in that week as it was trading in the direction of the overall trend. Also note that it was a very thin week.

Do you have access to historical data to backtest the system ?

No - there are 22 stocks being monitored currently. It switches in and out of them when buy/sell signals are generated based upon the strategy that stock is using.

This week in FTSE has been fairly 'normal' at the moment. Fri -Tues up 4%, Wed down 3%, today down another 0.5%. The quite violent swings are exercising how it behaves in these volatile times. The basic entry for a buy is if the 200SMA is trending up. If it is then it will look at other stuff. If it isn't then it won't. I'm not looking at shorting right now btw. Would rather get competent in long positions before turning my attention to shorting.

I don't have access to historical data at the moment. I feel that 1 month might be my minimum period but am not sure, hence the question in the first place.

 

[pedro01]

hmmmmm....

Lets face it - the last week was not typical - some of the last week was NYE. Don't just look at the week in terms of movement, look at in in terms of volume & hangovers.

I'd backtest your strategy over 12 months initially. This is easy for me to say as I have Tradestation and have access to years of data.

If I were you, I'd now be looking at one of the following:

1 - get some historical data - this is a serious business, you need the data, it's gonna mean paying some $$$ but you could lose a lot more $$$ if you just go live with it
2 - get a trial of a product like Tradestation, downsides are the learning curve & the fact you'll still have to pay for the data. Product will be free for 30+ days though. Some products like Ninjatrader are free for demo accounts
3 - find someone that already has the data & a product and ask then to backtest your system, downside is there's now 2 people that know the system.

Your choice. Given the path you are following, it seems that option 1 would suit you more.

 

[robster970]

hmmmmm....

Lets face it - the last week was not typical - some of the last week was NYE. Don't just look at the week in terms of movement, look at in in terms of volume & hangovers.

I'd backtest your strategy over 12 months initially. This is easy for me to say as I have Tradestation and have access to years of data.

If I were you, I'd now be looking at one of the following:

1 - get some historical data - this is a serious business, you need the data, it's gonna mean paying some $$$ but you could lose a lot more $$$ if you just go live with it
2 - get a trial of a product like Tradestation, downsides are the learning curve & the fact you'll still have to pay for the data. Product will be free for 30+ days though. Some products like Ninjatrader are free for demo accounts
3 - find someone that already has the data & a product and ask then to backtest your system, downside is there's now 2 people that know the system.

Your choice. Given the path you are following, it seems that option 1 would suit you more.

Option 1 for me. Where from?

 

[pedro01]

Option 1 for me. Where from?

absolutely no idea - I get my data with the product.

Someone else help here ?

 

[Adamus]

http://www.trade2win.com/boards/intraday-data/47568-fast-turn-around-data-provider.html
for EOD at least.

For intra-day, you might want disktrading, pinnacle, genesis, tickdata....

Alternatively get some charting / scripting software that can be hooked up to a live feed.

 

[adamscj]

Option 1 for me. Where from?

There are loads of places on the net you can get historical data for free - just be careful of the quality. I run my own strategies on FX, FTSE and Metals.

FX I get from Currency Forex Trading, Interbank Forex Broker, Low Spreads

FTSE I get from Yahoo (OHLC data back to 1984).

Metals I get from London Metal Exchange (sign up for their free data service).

In each case I've taken a historical dunp of all the close data into my database, then capture it live on a daily basis for tick data.

On you previous point, you need to back test your strategies over as much past data as you can. The important thing you are looking for is Stationarity - i.e. that your strategy has traction over different time frames. This means testing it for different time periods over different points in history - e.g. what would you return have been if you have run it from Mar-1986 to Dec 1989? What was your volatility and VaR during that time? Could you have sustained the maximum drawdawn in that period with the capital you have invested today?

Also for some strategies stationarity applied over different sample times. e.g. if you're long term trend following, this works whether you use daily data, weekly or monthly. This gives you a feeling that the strategy is a good one and not subject to overfitting your data.

 

[robster970]

I've been collecting data since the 8th December but have only been running the strategies this week. Anyway, I've written some code and created a test system that I can back test from. I'm running the strategies now from 8th December and can see already that they have problems that primarily affect the PF.

I've got to say, the help I've had off people on this thread has been enormous. You really have saved me weeks with your snippets of information. Many thanks guys.

 

[adamscj]

I've been collecting data since the 8th December but have only been running the strategies this week. Anyway, I've written some code and created a test system that I can back test from. I'm running the strategies now from 8th December and can see already that they have problems that primarily affect the PF.

I've got to say, the help I've had off people on this thread has been enormous. You really have saved me weeks with your snippets of information. Many thanks guys.

I run my database on a VPS so you could connect to it if you want read-only access to historic data. PM me if you're interested.

 

[blackcab]

So how many systems have you developed using genetic programming ? How are they working for you ?

I think the issue here is we have a lot of grail seekers who don't have a trading methocology to follow. Genetic programming isn't going to give you an execution platform, you'd use it to 'discover' new trading theories, although I really think the whole thing would be a waste of time. Most likely you would end up with the ultimate curve fitted system doing this.

The trading system I am implementing with my 3 partners is complex yet utterly implementable with Tradestation script (basically Fortran) plus a DLL plug in to handle a few things Tradestation doesn't do well. We still have a few features to put in to phase 1 and we have a load of features slated for phase 2 but you need to have a cut off point somewhere.

I'm working on the framework code and educating myself on various techniques like GP, and am not at the stage of developing complete systems.

I agree on the dangers of blindly finding systems with no trading rationale behind them - it's one of the top things to bear in mind and avoid. "Correlation doesn't imply causation". I think of a graph with 2 axes, level of correlation & level of rationale, where you can position trading systems. High correlation, low rationale = curve-fitted system (butter production in Bangladesh predicts the SP500). Low correlation, high rationale = 'proper' trading idea that 'should' work but doesn't ("this market's oversold and P/Es are on the floor, it *must* rise").

Like anything, there are intelligent and unintelligent ways to do this; because many/most people do system development unintelligently, it gets a bad rep.

 

[blackcab]

There is no reason to do only one but the danger with backtesting is that you end up curve fitting the parameters of your system to fit prior data. Also - market conditions change over time and a system that worked last year may not work this year.

Some systems are too complex to backtest but a very simple system (where all the decisions are made based on the action of a single symbol) will be very easy to backtest.

You can get books on system design and some I have read did a good job of telling you what to avoid in backtesting. Buggered if I can remember which book it was now but you could have a thread just covering those issues.

One thing I would say is that any system should not be left to blindly run regardless of fundamentals. Let's say that you have a system that is trading US equities based on technical signals. On Friday, the US employment figures will be released. At the time before that happens & some time after, you would be better off to have your system switched off (or not making new trades at least) as there will be some price movement that won't make sense to your system. Of course you may have a system specifically to take advantage of such times but generally speaking a system based purely on technical analysis may fail on days where the fundamentals are driving the market as opposed to the technicals.

If you subscribe to that opinion then it follows that your backtesting will not take into account non-technical days.

Agree completely, it goes with developing intelligently. It also means a lot more work & time which is probably why most people don't do it.

I still don't see the forward vs back test argument - forward testing over 3 months is the same as waiting 3 months then backtesting, so you may as well just backtest 3 months immediately, but it's not an important question and I'll let it go.

 

[blackcab]

So. My little ATS has been busily pretending to buy and sell this week using FTSE. It's working out profit factor, average win, average loss, win ratio.

Over what number of trades do people tend to run new strategies to get statistically meaningful results?

Not such a simple question. You could do:

Run a backtest over the same length of time or no. of trades as you want to run it forward before revising it - X months, or X trades regardless of time, etc.

Split historical data into categories that you could put any current market into - trending up/down/flat, high/low volatility, etc. - and treat each as a separate case. Backtest one system for 'long enough' in each category, or design a system for each category.

Loads of possibilities but you need more than a week or two, unless you're doing something very specific to now, which - in these times, who knows?