Automatic Pattern Search

[Jack01]

Alexander, a roulette wheel may be "completely deterministic" but I do not believe that the roulette dealer is. The dealer can spin the wheel in any whimsical way that he (or she) wishes. He can do a slow spin, then a fast one and then a medium one. At our local casino the dealres are told to vary their spins. The dealer may be in a bad mood and he may take it out on the wheel. As well, he can choose to use the small white ball instead of the regular size ball. He can also choose to release the ball a bit away from where it landed in the previous spin. That happens quite often. Furthermore, he can flick the white ball as slowly or as quickly as he wishes, whatever takes his fancy! So we have four factors which are not at all predictable: the speed of the wheel itself, the speed that the ball is flicked, the choice of ball, and the release point of the ball. All four factors are determined by the dealer (a human being, who is not predictable) and therefore not mechanistic. The dealer's brain does not conform to Newton's Law! Anyway, aside from the dealer, I do not believe that all of the "initial conditions" of the wheel itself are knowable. Will the casino allow anyone to measure all of the dimensions of a particular roulette wheel, including each baffle separating every number on the wheel? Are we allowed to test the wheel to see whether it is perfectly balanced? Are the players allowed to examine the roulette balls in order to weigh them and measure their dimensions? Common sense would say that roulette is not a "completely deterministic system". From a practical perspective, the outcomes can not be predicted, and the casinos bank on that. I do not believe that the use of a computer (super or otherwise) would make any difference! The East Europeans may have had a luck streak or several but I don't believe that their success was due to a crafty computer program. As they say "Do not confuse **** (US = "ass") with class".

Regarding your comment that a "truly random number generator" does not exist. That may be true, in the sense that a perfect sphere or vacuum does not exist. However, casinos don't need to have a perfect RNG. One which is 99.99% random (or whatever the case may be) is adequate for their purposes, just as a person with a gold ingot is satisfied if it is 99.9% pure.

I am very well aware that APS does not attempt to predict the next outcome. I have never said that it does. I can live with probabilities. Life itself confronts us with uncertainty, all the time.

I would be very happy if APS fails (to use your word) when subject to random data. That is, if it does not find any patterns I would be very favourably impressed. That is precisely the test that interested me from the beginning of my imput to this thread. By the way, I believe that you contradicted yourself. Earlier in your post you seem to believe that you did not think there was a "truly random number generator". Later on, you took for granted that random numbers could be generated, for testing by APS. If a "truly" RNG does not exist then how would you be able to get your hands on random numbers?

 

[trader3cnd]

jack01, I tested APS demo with random data, no patterns found. You can test yourself --- it's easy --- no need to have full program if you don't have.

 

[Jack01]

APS tested with random data

jack01, I tested APS demo with random data, no patterns found. You can test yourself --- it's easy --- no need to have full program if you don't have.

Trader3cd:

That is brilliant! My estimation of the program has now greatly increased, provided your test was valid. What I wanted verified was that APS does not generate any (or very few) "false positives". If APS finds no tradeable patterns in random data I am reassured on that point. Perhaps others do not regard this point as important.

You are right. I do not yet have the program, but I understand what it does. I have read some of the APS author's articles and they impress me.

May I ask you how did you get the random data? Did you make your own random data? If so, how did you do it?

Jack

 

[ronblack]

What I wanted verified was that APS does not generate any (or very few) "false positives". If APS finds no tradeable patterns in random data I am reassured on that point. Perhaps others do not regard this point as important.

Jack

jack01, be aware that what you call "false positives" will show up even if APS cannot find patterns in random data. This is one reason I do not trade individual patterns but instead I trade probabilities that I calculate based on the patterns APS generates.

I have found the use of system tracking in APS very useful for that purpose combined with the function to track open positions of models saved there.

Ron

 

[Hotch]

This looks really interesting. Need to look into it a bit more, but I think this is very similar to part of what i have been developing with my farther. Might try to incorporate it as it would save a lot of time, just think ours might be better :O

Thanks for the heads up.

 

[equtrader]

This is one reason I do not trade individual patterns but instead I trade probabilities

Ron

Excellent point.

 

[Jack01]

jack01, be aware that what you call "false positives" will show up even if APS cannot find patterns in random data. This is one reason I do not trade individual patterns but instead I trade probabilities that I calculate based on the patterns APS generates.

I have found the use of system tracking in APS very useful for that purpose combined with the function to track open positions of models saved there.

Ron

Ron, I do not actually have APS yet. So I am unaware of the "system tracking" capability of APS. When you say "system tracking" are you referring to particular patterns that APS has found, based on the criteria you specified? If so, that sounds a good feature to me. I suppose some profitable patterns may change over time (becoming less and less profitable), while others patterns may remain stable, while others may grow in profitability. I can easily imagine that. Am I right?

When you say that you trade probabilities rather than individual patterns, would you be able to elaborate on that? I thought that APS itself gives you patterns based on the probability criteria (and other criteria) that you set. I think that probability is the essence of successful trading, along with self-discipline etc.

I agree with you when you say that, just because APS does not find any patterns in random data, the problem of false positives may still apply. As I said in a previous paragraph, the characteristics of patterns may change over time, some changing from "good" to "bad" while others doing the opposite. I think that patterns represent the collective sentiment/psychology of the market (i.e. all of the individuals trading). If I am correct, a profitable pattern would mean that the market participants (as a whole) act/react in a predictable way (for the most part), given a particular set of price bars. Over time, the sentiment of the mob, in relation to a particular set of price bars, may change, just as societies change in many other ways.

Jack

 

[Jack01]

This looks really interesting. Need to look into it a bit more, but I think this is very similar to part of what i have been developing with my farther. Might try to incorporate it as it would save a lot of time, just think ours might be better :O

Thanks for the heads up.

Hotch, would you care to tell us something of what you have been "developing with your father", without giving away all of your secrets? When you said that you "think ours might be better", what were you referring to?

Jack

 

[Hotch]

Well, it's a mechanical trading system, which is, AFAWK, thinking outside the box (he's been in the ciy for...20+ years i guess, so hopefully he'd know).

As to "ours might be better". I was not refferring to pattern search, i decided (as i typed), that what we have so far as an equivilent to the APS, is probably better for our system. Still deffo something to look into though.

I'll update when I'm retired (lol)

 

[alexander]

When you say "system tracking" are you referring to particular patterns that APS has found, based on the criteria you specified?

Very neat feature. I use it and cannot live without it. Basically, any patterns you select from a search or scan you can add to a model in System Tracking, either as single patterns or in any groupings you want. If you update your data files, APS will tell you whether you have signals for the open of next day and which patterns generate them.

I especially like the System Tracking feature that calculates the conditions that must be met during the next day bar for a pattern to generate a signal. If you use the close to place orders that is. I use MOC orders frequently and this feature is quite useful to me. For example, it will tell you that if Close > 48.15 and High > 48.78 then this is a condition to have a signal at the close. So the only thing you have to watch is whether the High is above 48.78 and place the MOC order. Sometimes it will be a close call, but you can postpone the order for later during the night session or place it at the open of next day.

Also, the VIew Open Positions feature in System Tracking is very useful to me. You can see the open positions of every model you have stored there and whether they are making money or not.

I believe the System Tracking function is disabled in their demo but I'm not sure.


Alex

 

[jorune]

I have been reading through Michael Harris's latest book Profitability and Systematic Trading. I can recommend it as an interesting read. The book is a useful primer for anyone looking to create a trading system. He covers the key questions any system developer must face, which market, what time frames and the importance of risk management. He resolves the question of entry and exit through use of his software program, APS Automatic Pattern search.

At the back of the book there is a demo copy of the software on a CD with a selection of generic price patterns for four markets, QQQQ, DAX, SPY and FTSE. These are in the Appendix of the book and on the CD as well.

I would like to ask a question about these entries. Random Example:
{File:FTSE.txt Index:3 Index Date:20041007 PL:83.67% PS:16.33% Trades:49 CL:2}
{LONG, %, TARGET: 2, STOP: 4, ENTRY PRICE: OPEN, DELAY: 4}
( Followed by APS code line for Metastock or Tradestation or Wealth Lab)

I use Metastock 7.22 and I would like confirmation that I am getting the right inputs.

Please can you correct me if I am missing something.

I create a new system test and copy the APS code line into the Enter Long tab of the System Editor, I do not add anything else to Close Long, Enter Short or Close Short.
I click on the stops button which has five options, Breakeven, Inactivity, Max Loss, Profit Target and Trailing.
I presume that the Target figure goes in Profit Target tab and the Stop goes in the Max Loss Tab.
The entry price on open and delay are set in the System Testing options button.

Anything else obviously missing? I hope to compare and contrast the results in APS with the output of the Metastock system tester.

 

[ronblack]

Anything else obviously missing? I hope to compare and contrast the results in APS with the output of the Metastock system tester.

Sounds fine to me. Also look at this:

Using APS FAQ

Yep, I agree about the book. Down to earth writing, no hype like other authors, sensible advice. Michael Harris is a real trader.

Ron

 

[ZEPPO]

I have been reading through Michael Harris's latest book Profitability and Systematic Trading. I can recommend it as an interesting read. The book is a useful primer for anyone looking to create a trading system. He covers the key questions any system developer must face, which market, what time frames and the importance of risk management. He resolves the question of entry and exit through use of his software program, APS Automatic Pattern search.

At the back of the book there is a demo copy of the software on a CD with a selection of generic price patterns for four markets, QQQQ, DAX, SPY and FTSE. These are in the Appendix of the book and on the CD as well.

I would like to ask a question about these entries. Random Example:
{File:FTSE.txt Index:3 Index Date:20041007 PL:83.67% PS:16.33% Trades:49 CL:2}
{LONG, %, TARGET: 2, STOP: 4, ENTRY PRICE: OPEN, DELAY: 4}
( Followed by APS code line for Metastock or Tradestation or Wealth Lab)

I use Metastock 7.22 and I would like confirmation that I am getting the right inputs.

Please can you correct me if I am missing something.

I create a new system test and copy the APS code line into the Enter Long tab of the System Editor, I do not add anything else to Close Long, Enter Short or Close Short.
I click on the stops button which has five options, Breakeven, Inactivity, Max Loss, Profit Target and Trailing.
I presume that the Target figure goes in Profit Target tab and the Stop goes in the Max Loss Tab.
The entry price on open and delay are set in the System Testing options button.

Anything else obviously missing? I hope to compare and contrast the results in APS with the output of the Metastock system tester.

Hi Jorune,

I haven't read M. Harris's last book, but have got the other three he wrote; the whole point of designing a backtesting engine as part of APS was because he was not happy with what the commercial softwares provided him with - in fact when he contacted them, he realised that, though very good programmers, they simply didn't know what he was talking about! They didn't understand the mechanics of running historical simulations.
You will find differences between MS and APS - I did; however, this si not all.
In my experince, once you factor in the trading costs in the simulation, some "profitable" patterns become losers; also, Michael Harris is very clear about backtests simulations: take them with a large bucket of salt. I do: they only show the best outcome, there is no guarantee that such performance will happen in future.
The area where M. Harris is constantly banging up is money management - a topic from which I have heard nothing in these forums in reference to APS, yet the most important one in M. Harris books and methodology.
APS is a very good software, but M. Harris is clear: it is a data mining engine; over time, the profitability of those patterns return to mean,ie., stop working and become losers; some other profitable patterns just stop showing up in the data.
It would be nice if there was a way of ascertaining the trend (sorry guys, moving averages won't do for me) and trade in the direction of the trend only, this would increase the effectiveness and longevity of the patterns.
Oddly enough, in his first two books M. Harris went to some lenght in the presentation of his "P indicator" but not in his third volume; does he cover this topic in his fourth book? Anyone knows?

Eduardo.

 

[intradaybill]

Oddly enough, in his first two books M. Harris went to some lenght in the presentation of his "P indicator" but not in his third volume; does he cover this topic in his fourth book? Anyone knows?

Eduardo.

I am interested in this indicator too. I think the fourth book is just a revised edition of his third book published by Wiley.

Moving averages are bad indicators, I agree. If not for losers, they are for people who can sustain huge drops in equity. Not for me.

I know people who relied on systems based on moving averages and have gone out of business since last year.

Markets have gotten too smart lately to trade them with moving averages.

 

[ZEPPO]

Intermarket analysis

There is another topic which was presented in Michael Harris first book and for some reason has not been explored any further: intermarket patterns.
In it, the analysis between two markets, such as the S&P500/T-Bonds and Swiss Frank/Silver is presented.
I think this is an important topic today as seen by the correlation between the Dollar and Gold, and such markets as oil, metals, and others.
Unfortunately there is no way (that I know of) to use APS to find profitable patterns in one market to be applied in another, that is, a pattern which would have no effect in, say, the S&P500, but which would have an effect in, say, the T-Bonds, appears in the S&P500 data.
This is just one possibility; in the UK market, for example, such patterns found in the FTSE100 could be a Buy/Sell signal on some of its components, say, BP or Aztrazeneca.
Also, this could be applied to the currency markets vs. the major indices, etc...
It would even be possible to find pair trades using this strategy.
I wonder if this topic could be explored more in depth in future, as I believe there are profitable opportunities in such trading style. Perhaps in another APS update?

Eduardo.

 

[ronblack]

In my experince, once you factor in the trading costs in the simulation, some "profitable" patterns become losers; Eduardo.

Hi Eduardo,

In his recent book Mr. Harris makes explicit references to this fact and derives a formula about the impact of trading costs on pattern performance, are you aware of that?

Basically, when you set APS to search for patterns you must make sure that the success rate is above a certain value Pmin:

Pmin = pf/(pf+aT/S)

Pf is the profit factor you are shooting for, T and S are the profit target and stop-loss and "a" is a "safety factor" to account for the trading costs. He also proposes values of the factor "a" depending on the different styles of trading.

I agree pattern performance returns to mean value in the longer term but the question is how long that takes and whether there is first an increase in performance.

In other words, if you can find patterns that will maintain good performance for up to 3 years you can make huge profits. Then, at the end of the third year you search again for new patterns.

Also, a better way of using pattern signals is to trade combined probabilities rather than individual signals.

My regards to you and all the best,

Ron

 

[ZEPPO]

Hi Eduardo,

In his recent book Mr. Harris makes explicit references to this fact and derives a formula about the impact of trading costs on pattern performance, are you aware of that?

Basically, when you set APS to search for patterns you must make sure that the success rate is above a certain value Pmin:

Pmin = pf/(pf+aT/S)

Pf is the profit factor you are shooting for, T and S are the profit target and stop-loss and "a" is a "safety factor" to account for the trading costs. He also proposes values of the factor "a" depending on the different styles of trading.

I agree pattern performance returns to mean value in the longer term but the question is how long that takes and whether there is first an increase in performance.

In other words, if you can find patterns that will maintain good performance for up to 3 years you can make huge profits. Then, at the end of the third year you search again for new patterns.

Also, a better way of using pattern signals is to trade combined probabilities rather than individual signals.

My regards to you and all the best,

Ron

Hi Ron,

Thank you for your best wishes - and likewise.
Yes, I am aware of this as it is covered in his first three books.
This is the kind of posts I'd like to see, more on the side of risk and trade management than on the trading signals themselves, traders ideas on these topics, how they go about it.
I haven't got M.Harris last book; what's new in it that wasn't in the other three? "Profitability and Systematic Trading" was quite deep, wouldn't this last one be a re-print of some sort? Just asking before I commit.

Eduardo.

 

[trader3cnd]

Eduardo, time to use a deep discount broker like IB.

Trading costs not an issue nowadays. Good signals is the issue.

I also have all books authored by Mike Harris. The last book is Profitability and Systematic Trading just the publishing house changed along with cover, etc.

 

[ZEPPO]

Eduardo, time to use a deep discount broker like IB.

Trading costs not an issue nowadays. Good signals is the issue.

I also have all books authored by Mike Harris. The last book is Profitability and Systematic Trading just the publishing house changed along with cover, etc.

Thanks for the tip about the book.

Eduardo.

 

[alexander]

There is another topic which was presented in Michael Harris first book and for some reason has not been explored any further: intermarket patterns.
In it, the analysis between two markets, such as the S&P500/T-Bonds and Swiss Frank/Silver is presented.

Eduardo.

Hello Eduardo,

Let's ask tradingpatterns.com to add this feature. Good idea! Maybe if several users ask they will add it.

By the way, some patterns I got from APS back in 2005 when I first played with the program still make money. The problem with me is that I tend to be discretionary at times and I have ignored many good signals.

Alex