Automatic Pattern Search

[ronblack]

Elaborate scheme. The indicator works, no need to backtest it. Backtesting results will not be any more significant than the value of the indicator.

My success rate fo far has been close to 90%. Michael Harris is light years ahead from the rest of the pack. He shouldn't be selling his p-Indicator and I wonder why he is doing this.

 

[ZEPPO]

He shouldn't be selling his p-Indicator and I wonder why he is doing this.

.
Maybe because he doesn't want somebody else to do it for him?
After all, the P-indicator mechanics are disclosed in two of his books.

Eduardo.

 

[trader3cnd]

.
Maybe because he doesn't want somebody else to do it for him?
After all, the P-indicator mechanics are disclosed in two of his books.

Eduardo.

This is true with most things. If you do not do it someone else will, sooner than later

I bought the books and I tried to program the p-Indicator. I spent about six months working on it but it got me no place. I understand that besides the indicator values, there is another value S that denotes their significance. I have no idea how that is calculated and it is not explained in the software manual.

One thing is for sure, this indicator is very interesting. Nothing like it out there.

 

[saratur]

I am evaluating APS now.
I understand from posts in this thread that in order to build very effective trading systems, one should combine patterns for higher probability trades.
Still, my expectation is that if the APS is effective in discovering robust and profitable patterns, those should show positive profit expectancy in out of sample test.
Am I wrong with this expectation?
So far, in all the evaluation tests I made the patterns found in search did not perform well in out of sample test. There were more losers than winners, and for each price series, averaging across all patterns and all trades, the profit factor was in almost all cases less than 1.00 .
- What am I missing?
- Suggestions?

Here is some data on some price series I tried:

All are in regular search (not expanded), no delay.
EMD (S&P 400 futures), daily session, 15 min bars.
Training (in-sample) over appx 48,000 bars .
T/S = 5/5 points , PF> 1.5 ; found 46 patterns.
OSS on three 2,200 bar sections most patterns not profitable; PF on all patterns, all trades 1.02, 0.95 and 0.94 .
T/S = 10/10 points, PF>1.5 ; found 116 patterns.
OSS on three 2,200 bar sections most patterns not profitable; PF on all patterns, all trades 0.99, 0.91 and 0.9 .

QQQQ 60 min. bars.
Training (in sample) 10years, appx 16,000 bars.
T/S 4% / 4% ; PF > 1.5 ; found 92 patterns.
OSS two one year sections, PF over all trades 0.99 and 0.84 .

 

[intradaybill]

I am evaluating APS now.

Demo or full version?

Still, my expectation is that if the APS is effective in discovering robust and profitable patterns, those should show positive profit expectancy in out of sample test.

It depends what you mean by out of sample.

So far, in all the evaluation tests I made the patterns found in search did not perform well in out of sample test. There were more losers than winners, and for each price series, averaging across all patterns and all trades, the profit factor was in almost all cases less than 1.00 .
- What am I missing?
- Suggestions?

How many runs have you done? It is normal to have more losers than winners in out of sample testing. Otherwise, markets would be totally screwed up.

Why are you averaging profit factors by the way? This does not make sense to me.

All are in regular search (not expanded), no delay.

Regular search is someting like 1/6 of the program capability (20 vs. 112). I do not count delay, that maybe lowers that to 1/18 or 1/24 of program capability.

I think you should do extended searches for many different T/S entries and also increase the size of the data files.

QQQQ 60 min. bars.
Training (in sample) 10years, appx 16,000 bars.
T/S 4% / 4% ; PF > 1.5 ; found 92 patterns.
OSS two one year sections, PF over all trades 0.99 and 0.84 .

Are you sure that in 10 years of hourly data you only have 16,000 bars? Something wrong with your data? Also, are you looking for 4% profit from entry price in hourly data? That is more like position trading. Better use daily data.

 

[ronblack]

Out of curiosity, yesterday I had APS run a few searches with 10,000 bars worth of 60-minute QQQQ data for several different sets of target/stop. Out_of_sample was 1,800 bars long.

Accross the board, about half of the patterns showed pf > 1 in out_of_sample testing. Some showed very high pf and several had a success rate of 100%, all in out_of_sample testing.

I think @saratur is either a newbie and his tests are all messed up or he is simply trying to smear APS for reasons he only knows.

 

[saratur]

Intradeaybill, thank you for the response.

Demo or full version?

Demo.

It depends what you mean by out of sample.

Price bar series which is out of the training ("pattern search") data series. For example, I used 10 years of QQQQ 60 min data till mid April 2008 for pattern search, and then tested the found patterns on the two following years, until mid April 2010, as OSS (Out of Sample) . I did split that OSS into two separate 1 year periods so as to see the different results on a the very bullish last year, and the bearish year before that.

How many runs have you done?

So far only ~ 15 runs; I got timed out of several expanded searches.

It is normal to have more losers than winners in out of sample testing. Otherwise, markets would be totally screwed up.

I am not sure I understand your comment - whether you relate to win %, or to overall profit.
I am trying to get some quantitative confirmation to the power of APS, hopefully for both daily and intraday data. My thinking is that if APS is useful in identifying robust patterns, Out of Sample tests should show, on the average, positive profit - that is overall PF>1 ( the win % as standalone is meaningless when not tied to the win/loss ratio of trades) . They should be significantly better than random trades. Probably for good trading systems one will need more than the raw patterns - (e.g. use pattern combinations) if the numbers are not strong enough.
Nevertheless, I do expect the result to have positive expectancy with reasonable statistical confidence and robustness; and while I should probably see some degradation relative to the training period - it should not show a total collapse . Similar to what Michael Harris is reporting in an article on QQQQ daily data - PF in one out of sample period kept roughly same as in the 'training' period, and only small degradation in the next period, still being at around 1.5 .
Am I wrong with this expectation?
Am I missing something important here?

Why are you averaging profit factors by the way? This does not make sense to me.

I was probably not clear. I am not averaging profit factors.
The demo version does not allow me to test systems or export the trades one by one. So I look at the results as if I were trading all the found patterns in parallel, with equal $ amount for each. I export the results to CSV, sum up the profit from all winning trades, the loss from all losing trades, and the ratio of these sums is what I reported in my post as the overall PF.
If one were to attempt try trading this in real life there will be "doubling up" on trades that were identified by different patterns, so the PF might lead to an "optimistic" view.
This is the best I could come up with for evaluation; any better idea?

Regular search is someting like 1/6 of the program capability (20 vs. 112). I do not count delay, that maybe lowers that to 1/18 or 1/24 of program capability.

With the demo I had several expanded searches timed out. With a couple I managed to complete, I got more patterns; the resulting PF for the calculation I so is in the same order of magnitude as with the regular search (as I would expect; the benefit of more patterns will be, I presume, when I try to build a real system and combine patterns for higher probabilities).

I think you should do extended searches for many different T/S entries and also increase the size of the data files.

Are you sure that in 10 years of hourly data you only have 16,000 bars? Something wrong with your data? Also, are you looking for 4% profit from entry price in hourly data? That is more like position trading. Better use daily data.

For QQQQ, being a stock, I used hourly bars only for the day, regular session.
I do hope to get good results for intraday trading; I started with 5 min data and got nowhere. So I am trying the hourly data, figure out what works, and then try again lower time frames. For the hourly data I thought of using a small multiple of ATRs, so as to get trades typically lasting from few hours to few days. Maybe it is too high. Following your suggestion here I tried lower T/S (e.g. 1%, 2%, 3%) .
I am getting better results, and several of the tests show positive profit factor. Still, the PF is relatively low (max 1.2, most are lower though), much lower than the search PF (> 1.5 or 2.0 ), and the profit/trade is quite low.
I continue exploring; If you would not mind giving me some more pointers on where to look (e.g. time frames for intraday, T/S, instruments) I would be thankful.

 

[saratur]

I think @saratur is either a newbie and his tests are all messed up or he is simply trying to smear APS for reasons he only knows.

ronblack - I like the concept of APS. Following several very positive posts from happy users, including yourself, I decided to evaluate it. I am trying to get some quantitative results that will make me feel comfortable I could use it effectively. I would not be putting the effort unless I would hope it will come out golden, not smeared.
I simply showed the results of my tests, looking for help on how to get better results. If my tests are messed up - I will be thankful if you could help me to find out where.

Accross the board, about half of the patterns showed pf > 1 in out_of_sample testing. Some showed very high pf and several had a success rate of 100%, all in out_of_sample testing.

Yes, in my tests as well. For example, in the out of sample period of last year, I get 48 out of 92 patterns with PF>1, and 15 of those have 100% success rate. However, this is only part of the statistics. There are also 18 patterns with 0% success rate; and as I look at all the trades as a group, there are more losing trades than winning trades. If there was a fixed dollar amount per trade, the ratio between gross profit to gross loss is a PF = 0.83.
For the tests that you ran, what is the total profit vs. total loss across all patterns?

My thinking is that looking only at those patterns that got high PF results in OOS, and discarding those that failed - is not a good idea. Do you agree?

Testing with 2%/2% I got better results, OOS PF=1.20 and 1.07 ; with 1%/1% , 1.16 and 0.72 . WITH 2% / 3%, pf=1.17 and 1.13 .
While some T/S are positive, this is a very high degradation from the PF> 1.5 of the search I used.
I probably need to look at other time frames/instruments/parameters. If you could help it would be great.

 

[ronblack]

For example, in the out of sample period of last year, I get 48 out of 92 patterns with PF>1, and 15 of those have 100% success rate. However, this is only part of the statistics. There are also 18 patterns with 0% success rate; and as I look at all the trades as a group, there are more losing trades than winning trades. If there was a fixed dollar amount per trade, the ratio between gross profit to gross loss is a PF = 0.83.

Why do you care about the losing patterns in the out_of_sample? It is like someone developing 10 different systems based on a few indicators, then testing them out_of_ sample but still considering their performance as a group.

My thinking is that looking only at those patterns that got high PF results in OOS, and discarding those that failed - is not a good idea. Do you agree?

No because what you say makes no sense at all. It is like you believe that all patterns APS finds must form a system of some sort, they are inseparable and what you are actually doing is evaluating the performance of a system not of each pattern. Besides the fact that this is an absurd notion, you can never know how different trades interplay, like for example how many long signals close out short signals or vice versa and turn out what was an original loser to a winner. What you get from APS is a list of patterns you can use to build a system after you evaluate each one of them. If you read the manual carefully, there is a number of groups and only within each group patterns have common characteristics. Patterns from different groups have different characteristics. If you are summing them, it is like summing any two arbitrary system together, like one with moving averages and one with CCI. More specifically, what is common between two patterns when one looks like a double bottom and the other like an inside day breakout other then the fact that they are made of price bars? The answer is that the only commonality is the concept of a price bar but otherwise they are totally different animals. Summing their trades is like summing apples and potatoes. This is why I thought you are a newbie and I was correct. I think before you get involved with software like APS you should try to understand basic concepts.Otherwise you are wasting your time with APS. It won't help you. You will throw your money away. Start with something simpler you can understand. This is friendly advice. You appear to me like someone who has never driven a vehicle and wants to become a Formula One driver right away.

 

[saratur]

ronblack - with the demo version I cannot evaluate how the different patterns interplay, nor try combining patterns into a system.

The tests (intentionally summing up results from different patterns) are intended to answer a simple question: "are the different patterns identified by APS have better expectancy for profit than a group of randomly chosen six bar patterns ?".

I agree that the tool can be useful regardless. I also think the answer is important.

 

[intradaybill]

The tests (intentionally summing up results from different patterns) are intended to answer a simple question: "are the different patterns identified by APS have better expectancy for profit than a group of randomly chosen six bar patterns ?".

IMO

(1) You are summing the results the wrong way. I think it was explained to you already why.

(2) You expect 100 patterns generated by APS, or something like that, to average to positive expectancy. This is the wrong expectation from the program. APS will find a positive expectancy only if it exists. It will not get one for you just because you are looking for it. If you do not find positive expectancy APS saves you money as a pattern trader. It basically tells you not to trade in this particular market and timeframe. This information is as valuable as getting a system with positive expectancy because it can save you a lot of money.

(3) I also think that pattern selection based on statistical tests is a viable technique and turns the random generator argument into a red herring. The real issue here is what are the criteria and how the selected patterns must be combined to increase the expectancy.

By the way, in an article by Michael Harris it was shown that all QQQQ patterns generated by APS in 2002 stayed overall profitable until 2008 or something. Thus, by a counter-example, he showed that APS has the potential of finding even overall positive expectancy patterns. By no means he proved that all groups of patterns found by APS will result in overall positive expectancy. He specifically states in the beginning of the article that he cannot answer this question.

article link

 

[saratur]

Intradaybill - thanks, this makes sense.

 

[Albert Carey]

The tests (intentionally summing up results from different patterns) are intended to answer a simple question: "are the different patterns identified by APS have better expectancy for profit than a group of randomly chosen six bar patterns ?".

I agree that the tool can be useful regardless. I also think the answer is important.

Very interesting. You download a demo version, you do a few tests and you want to get better expectancy for profit than random picks. Right away. The program will do that for you. This in turn means you will get filthy rich. This what positive expectancy better than random picks means.

So how much APS goes for these days? 2k? Here is a guy who thinks he will pay that much and get rich trading the markets after choosing any market at random and having the program to identify patterns. Even more stunning is the fact that he wants to make sure before paying the 2k that he will become rich. In other words, he wants to get rich without risking anything at all. What we do call this type of persona?

 

[intradaybill]

So how much APS goes for these days? 2k?

It doesn't matter. Saratur made some good points. APS is not going to make anyone rich without putting a lot of work in working with its output. I do not recommend this program to anyone who does not have experience testing systems and probability and statistics knowledge. It just won't help. APS can find many systems but determinign which ones have a chance to stay robust is a very challenging task.

 

[saratur]

Albert Carey - It appears you did not understand my post. I am evaluating a tool, trying to understand what exactly its capabilities are, and how it compares with other tools that support system synthesis. Using it will mean a large investment of time and effort, and it does not make sense deciding to do so without understanding what it does, and what should I expect from it.
Initially looking at APS I could not fully understand what it does or fully reconcile statements I heard about it (in spite of engineering background and trading systems development experience). So - in my post, presenting some data, I was asking for suggestions, advice, and feedback from those who are experienced with the tool. Some of the feedback I received is definitely helpful. Yours, sadly, is not.

 

[alexander]

Initially looking at APS I could not fully understand what it does or fully reconcile statements I heard about it (in spite of engineering background and trading systems development experience).

What APS does is pretty straightforward in a sense but only in principle. The difficulty comes when trying to interpret the program results. I now understand there are two kinds of APS users. (A) The naive user is one who tries to find good patterns and use them to generate signals and (B) the sophisticated user is one who tries to use the program results to calculate the probability of win at any given time for a specific position.

 

[ronblack]

Hi guys, I hope you are all doing well.

I guess the following is to be taken as an indication of the predictive ability of the p-Indicator:

http://www.tradingpatterns.com/Support/pindicator/pindicator.html

The high significance S and several values with a spread > 20 at the close of Friday, May 7, 2010, were an indication of a strong move for next Monday. I also think they may indicate that this correction to the upside will continue for a few days.

 

[alexander]

There is an APS user here who has analyzed several search results out_of_sample using Amibroker. Good work.

 

[NIandP]

If anyone has a used APS copy for sale please pm

 

[NIandP]

Does anyone know what happen to this company? Website appears to be under construction.