Automatic Pattern Search

[jimbotrader]

It seems that JIm decided to turn legit...

JIm always legit. I was just looking for a better deal. I'm a trader. Traders always look for good deals.

APS has a lot of potential. I don't understand why the company does not develop it further to add charting and other nice features. I am seriously thinking of raising some money in the form of venture capital to buy this company and their software rights and develop it to run real-time scans using multiple processor architecture. I think it can do statistical arbitrage with a few changes which is the big thing now. Maybe you will be my customers in the futures. So be nice people

 

[ZEPPO]

APS has a lot of potential. I don't understand why the company does not develop it further to add charting and other nice features. I am seriously thinking of raising some money in the form of venture capital to buy this company and their software rights and develop it to run real-time scans using multiple processor architecture. I think it can do statistical arbitrage with a few changes which is the big thing now.

What makes you think M.Harris hasn't already done so? He offers services for institutions, too; some people in the thread have got a customised version of APS, if they can do that, so do the institutions.
But good luck in your business venture, anyway.

Eduardo.

 

[alexander]

What makes you think M.Harris hasn't already done so? He offers services for institutions, too; some people in the thread have got a customised version of APS, if they can do that, so do the institutions.
But good luck in your business venture, anyway.

Eduardo.

He has probably already developed features not available in the commercial version. If you read his letter to traders maybe you will get the same idea as I did, there is not enough money to be made selling to retail traders:

MHarrisMsg

I think I will switch to the new version 8. It is not compatible with previous versions of the program but It has the p Indicator which sounds very interesting. I will need to redo my workspaces and update the results I guess. (provided they offer to me the new version at a discount)

 

[ZEPPO]

He has probably already developed features not available in the commercial version. If you read his letter to traders maybe you will get the same idea as I did, there is not enough money to be made selling to retail traders:

MHarrisMsg

As I suspected; thanks for the link.

Eduardo.

 

[intradaybill]

I upgraded to the p-Indicator version guys. Actually I scrapped my previous version since the new one is not compatible with older versions. APS is by far the best program out there for finding non-optimized/high probability setups. The p-Indicator is probably some kind of a serious edge but I have to play with it more. The price of a license is high enough to sustain any edge for a while and if you take a look at the website they are very selective even with whom they allow access to the demo.

JImbo stay away

 

[bashatrader]

I'm looking for a second-hand license of this program, preferably v8 with p_indicator, at a reasonable price. Pls send pm.

 

[intradaybill]

Update: I screwed up upgrading my APS to the latest version last time I tried but I was able to install again yesterday and I got everything up and running. I will post a few updates from time to time about the performance of the p_Indicator. My first reaction is very positive. It is something completely new it appears.

@bashatrader,

I think you should forget about it. What you are trying to do is called software piracy.

 

[markies21]

I've just got myself a demo copy of this program and am thinking of the following as a trading strategy.

a) Run software on 500 ASX listed shares
b) Get 10x Long trade & 10x Short trades by "anding" the pattern matches as suggested in this thread to get a stronger statistical inference
c) Of these twenty, synthetically construct a portfolio with a Beta of 0 (market neutral) by entering into Long / Short trades as required
d) If (a) + (b) +(c) turn out to be successful, possibly run the portfolio at 2x leverage

My questions are:

1) Does anyone already do this?
2) What's the most important thing in knowing that would help when using this software?

This software looks really exciting and I wish I discovered it years ago when I was back looking for patterns by hand!

Okay any help much appreciated

 

[markies21]

Are we all curve fitting?

I have doubts on the way walk-forward is being used to validate patterns in APS.

Say you find 30 patterns on an instrument and then you walk froward those patterns to find that:

- 10 work well
- 10 don't work at all
- 10 break even

You go and then trade the 10 profitable patterns. Isn't this curve fitting? The walk-forward result is what would have happened with any pseudo-random data if trades follow some sort of normal distribution.

I think the problem is that the patterns APS finds do not have strong statistical inference on future price movements, so the only real benefit gained would be from not walking-forward testing individual patterns, but "anding" individual patterns together to increase their probability into something marginally useful.

Therefore, only walk-forward testing of combined patterns is of any use.

Still, a great piece of software IMO :smart:.

I've just posted a similar post in http://www.trade2win.com/boards/mechanical-systems-trading/26419-backtesting-results-vs-forwardtesting-results-new-post.html.

 

[intradaybill]

You go and then trade the 10 profitable patterns. Isn't this curve fitting? [/URL].

No because you have changed no parameters of a system to fit it to the data. Instead, you have introduced a selection bias based on the assumption that patterns that pass forward testing are better than the ones that do not. This is an assumption that is considered valid by many trading system developers.

Curve fitting - Wikipedia, the free encyclopedia

 

[markies21]

Hi Bill, this has nothing to do with selection bias.

Say you had a system for playing Roulette at a casino. Your system rules were - bet in this sequence - red, black, red, black, red, black. You backtested this on 1000 roulette tables, and found that you actually made money on 30 on them.

Then, you go and "walk forward" your results and you find that, unsurprisingly,
- 10 tables you broke even
- 10 tables you lost money
- 10 tables you made some money

This walk-forward result does not validate a trading system, because in a Null Hypothesis test, you would find that you get same result.

See Null hypothesis - Wikipedia, the free encyclopedia.

I would like to have APS ask the question - when it walks forward - "what would happen if I ran this simulation on a completely different set of data" - conduct a Null Hypothesis test. If from the Null hypothesis test:
- the net outcome is the same, trading system has no edge, doesn't work
- the net outcome is considerably worse, only then you can say your trading system works

 

[jimbotrader]

Say you had a system for playing Roulette at a casino. Your system rules were - bet in this sequence - red, black, red, black, red, black. You backtested this on 1000 roulette tables, and found that you actually made money on 30 on them.

Hey jimbo --- oh sorry I am jimbo...

Market is not roulette or casino. If you think like that it means you are a loser rookie. Market price moves are not always random. People often buy or sell because price moves above or below certain levels. Roulette outcomes are completely independent, no outcome depends on previous outcome.

I think intradaybill tried to explain to you something you do not seem to understand. Let us say you look at price data (forget about APS for a moment) and you decide that a 3-bar key reversal and an inside break-out (also 3-bar pattern) seem very promising. You backtest and then forward test the two pattens and you decide to trade the key reversal. This is not optimization. You have selected a pattern based on historical testing. Your bias is based on historical performance. Maybe the other pattern will turn out to be better. You don't know that. This is the nature of trading. If you think selecting a system based on testing is curve-fitting then all systems ever thought or designed are curve-fitted because you could have chosen a different one.

 

[dcraig1]

I'll admit that I haven't read all of this thread, but it seems to me that data mining price time series for patterns of <= 10 bars that look like they might be profitable is really barking up the wrong tree for stock trading.

You need to see what other traders see and it's probably not these patterns. Furthermore, the market has a lot longer memory than 10 bars.

As an alternative, there are oldies but still goodies that are very simple:

Only long "strong" stocks. Only short "weak" stocks.

Strong stocks have high relative strength and are probably in a sector/industry of high relative strength. Preferably they should have strong fundamentals. The inverse for weak stocks. Buy strong stocks at support or oversold and the inverse for weak.

Intelligently apply this according to general market conditions with an eye on both technicals and fundamentals.

A good screener is essential and effort applied to developing good screens is likely to be more rewarding than data mining time series patterns. I wrote my own screener, and that taught me far more about stock trading than looking at price patterns ever could. But it is in a different class to the free screeners on the net.

 

[bashatrader]

I'm looking to purchase a second hand APS license. Please PM.

 

[markies21]

Hello!

Hey jimbo --- oh sorry I am jimbo...

Market is not roulette or casino. If you think like that it means you are a loser rookie. Market price moves are not always random. People often buy or sell because price moves above or below certain levels. Roulette outcomes are completely independent, no outcome depends on previous outcome.

I think intradaybill tried to explain to you something you do not seem to understand. Let us say you look at price data (forget about APS for a moment) and you decide that a 3-bar key reversal and an inside break-out (also 3-bar pattern) seem very promising. You backtest and then forward test the two pattens and you decide to trade the key reversal. This is not optimization. You have selected a pattern based on historical testing. Your bias is based on historical performance. Maybe the other pattern will turn out to be better. You don't know that. This is the nature of trading. If you think selecting a system based on testing is curve-fitting then all systems ever thought or designed are curve-fitted because you could have chosen a different one.

Hi Jimbo,

I tested the product myself this weekend, back-tested 10 randomly selected FTSE stocks, 1:1 risk reward, 5% P/SL, 66% profitability, >2 profit factor, >20 trades per pattern, 500 days. When I walked forward the results a year, the average profitability was 48.5% - below break-even. There was no one stock more or less suited to the patterns APS found.

I then thought about the previous post on Selection Bias, and thought that maybe I should be discarding the "patterns that went bad" during walk-forward. But here's why I think that could make no difference. Imagine this scenario:

1) You buy a very fast computer, and pattern search 10,000 stocks
2) You find patterns on 1,000 of those stocks
3) You walk forward those patterns one year, and 100 are still profitable

Does that mean the 100 will remain profitable next year after that? From the previous distribution of trades in this example, my guess would be only 10 would stay profitable.

This is the reason why I have a problem with silently discarding bad patterns during walk-forward (Selection Bias). I mean, I can prove myself wrong, the test would be:

-> pattern search a large enough pool to get (a) - backtest patterns
-> walk forward (b)
-> discard loser patterns
-> walk forward again (c)

If (c) is as profitable as (a) - then Selection Bias works.

 

[intradaybill]

Hi Jimbo,

I tested the product myself this weekend, back-tested 10 randomly selected FTSE stocks, 1:1 risk reward, 5% P/SL, 66% profitability, >2 profit factor, >20 trades per pattern, 500 days. When I walked forward the results a year, the average profitability was 48.5% - below break-even. There was no one stock more or less suited to the patterns APS found.

IMO >20 trades is not a good criterion. The number should depend on the price history. If you have 1000 bars in your history something like 30 trades may be enough but if you have 5000 bars you should have many more.

I think you are starting from wrong selection criteria. I demand more than 50 trades for each pattern before I combine them and if I cannot get enough patterns I do not trade the particular market.

Also the choice of P/SL is important. If you choose something like 5% for P in a stock that barely moves then you are not going to get a sufficient number of trades. If you use a low SL in a very volatile stock you will not get enough winning trades.

I think maybe you got the impression that APS can be used with closed eyes but that it not the case. It is just a tool to find patterns with performance that agrees with the selected criteria. The choice of performance criteria is very important. It took me one year of intensive use of this program to get a handle of this.

I think statistical significance is a very bogus concept and many statisticians agree. Bayesian probability is more appropriate with data mining and trading. All you want is to have a measure of probability of a trade to succeed and make profit. You should care less if the sample is significant or not in itself because that is not useful. If you see a plane (pattern) losing altitude close in the horizon you can assign probability .7 (prior success rate) that there is a close by airport (landing) and .3 that it is losing altitude for some other reason. Then, if you see another one doing the same in a few minutes, depending on your method of revaluing probabilities, you can say for example that the probability of a close by airport (landing) is now .9. Of course, the two planes may be flying low for other reasons (open positions) and you do not know what will happen. It may turn out there were both having problems (bad trade).

 

[markies21]

Big pattern search

Hello Bill,

I've run the pattern search again using these parameters:
-> 112 pattern search
-> 4% profit target, 6% stop loss
-> Min 30 patterns
-> PF >1.5
-> Percentage Profitable >75%
-> 10 stocks listed on the London Stock Exchange
-> 1000 bars

The test took 10 hours in total.

There are three files attached:
- ftse_10_1000_bars_4_6_insample.csv [backtest, 1000 bars back from 31/12/2007]
- ftse_10_1000_bars_4_6_outsample1.csv [walkforward 1, 01/01/2008 - 31/12/2008]
- ftse_10_1000_bars_4_6_outsample2.csv [walkforward 2, 01/01/2009 - 09/07/2009]

What I found was that the patterns didn't walk forward profitably. In both walkforward groups, the percentage profitable is around 50%, resulting in a loss.

I don't think the problem is with APS not finding any patterns, it's that they don't remain profitable in walk forward. Also, the patterns with 50+ occurances were not any more or less likely to be profitable in the future than those with only 30 occurances.

 

[fobos]

This is why I gave it up a few months ago. It is not that they are totally useless, just they would not give me any edge above other good approaches. Good luck all of you, who wants to become really rich with APS. Besides I was not happy about some designing issues and restrictions with APS, so unhappy that partially implemented my own analogy of APS, which, at least, is not worse.

Hello Bill,

I've run the pattern search again using these parameters:
-> 112 pattern search
-> 4% profit target, 6% stop loss
-> Min 30 patterns
-> PF >1.5
-> Percentage Profitable >75%
-> 10 stocks listed on the London Stock Exchange
-> 1000 bars

The test took 10 hours in total.

There are three files attached:
- ftse_10_1000_bars_4_6_insample.csv [backtest, 1000 bars back from 31/12/2007]
- ftse_10_1000_bars_4_6_outsample1.csv [walkforward 1, 01/01/2008 - 31/12/2008]
- ftse_10_1000_bars_4_6_outsample2.csv [walkforward 2, 01/01/2009 - 09/07/2009]

What I found was that the patterns didn't walk forward profitably. In both walkforward groups, the percentage profitable is around 50%, resulting in a loss.

I don't think the problem is with APS not finding any patterns, it's that they don't remain profitable in walk forward. Also, the patterns with 50+ occurances were not any more or less likely to be profitable in the future than those with only 30 occurances.

 

[markies21]

Thanks Fobos... I've not given up on APS yet - but I think I'm 80% there... May try it out in some less sophisticated markets like Skippy stocks on the ASX200.

I think most of the patterns that used to be profitable to trade have been ironed out ever since computing power became cheap - but this is probably more a philosophical than academic argument!

 

[ronblack]

Hello Bill,

I've run the pattern search again using these parameters:

-> 1000 bars

hello,

What is the 1000 bars stand for? Is this the number of bars in each file? In other words, what is the length of the history in each file?

I sorted the results you posted and I notice the following

-In 2008 forward, 108 out of the 232 patterns remained profitable (PF >1). Given that this was a disaster year for stocks, I find the results very good. As a matter of fact, extraordinary good.

- In 2009 forward, during March of this years FTSE index touched 3500, its lowest level in many years. Yet, 102 out of the 232 patterns remained profitable. This is remarkable given that most of the patterns are "long".

Most funds, trading systems, discretionary traders, etc. lost money in 2008 and in the first quarter of 2009. Except if you are doing these tests to find out if APS would have made you rich in 2008.

Another thing: what would be the combined P/L had you traded all these patterns, (although that not make any sense)?

Ron