Changing parameters every week, is that curve fitting?

[SuperDriveGuy]

Hi All,
I am a newbie, so please bear with me. Just wanted some opinions from the veterans, whether I am progressing in the right direction!!!

I have created a simple system based on Stochactics I will try to illustrate as best as possible..

if Stoch(14) (crosses below) 80
SELL

if Stoch(14) (crosses above) 20
BUY

The parameter, I optimize are Stoch(14), 14 being the period, I run that through an optimizer and that comes with the best value for a say a week, same for the Stoch HIGH and LOW(80 and 20) respectively. So for say 01/06-05/06 I may have Stoch 14,HIGH 75 and LOW 25 as the best parameter that wll give me good return, low drawdown etc. But if I forward test that with say 08/06 - 12/06, I will get abysmal returns, but a different set of values say Stoch 20, HIGH 85 and LOW 15 will give best results.

I am not looking for shortcuts here, just the methodology, thought process for strategy optimization. I have also run the same optimization against 1 months data then back and forward tested 1 month each, but similar results i.e. what works in 1 week/month does not work in the next week/month. The principle as such looks sound(to me), but the tweaking of the parameter values, will that be considered curve fitting?

Appreciate your thoughts on this.

Regards

 

[Hoggums]

Hi All,
I am a newbie, so please bear with me. Just wanted some opinions from the veterans, whether I am progressing in the right direction!!!

I have created a simple system based on Stochactics I will try to illustrate as best as possible..

if Stoch(14) (crosses below) 80
SELL

if Stoch(14) (crosses above) 20
BUY

The parameter, I optimize are Stoch(14), 14 being the period, I run that through an optimizer and that comes with the best value for a say a week, same for the Stoch HIGH and LOW(80 and 20) respectively. So for say 01/06-05/06 I may have Stoch 14,HIGH 75 and LOW 25 as the best parameter that wll give me good return, low drawdown etc. But if I forward test that with say 08/06 - 12/06, I will get abysmal returns, but a different set of values say Stoch 20, HIGH 85 and LOW 15 will give best results.

I am not looking for shortcuts here, just the methodology, thought process for strategy optimization. I have also run the same optimization against 1 months data then back and forward tested 1 month each, but similar results i.e. what works in 1 week/month does not work in the next week/month. The principle as such looks sound(to me), but the tweaking of the parameter values, will that be considered curve fitting?

Appreciate your thoughts on this.

Regards

Your examples show that you are curve fitting and why your strategy will lose you lots of money.

Let's say 01-05 Jun you trade 20,85/15 levels - you lose loads of money that week

So you optimise based on that weeks data and you trade 08-12 Jun using 14,80/20

You lose again - you optimise again and the following week you trade 20,85/15

It is important with any auto strategy that you test it on unseen data. If it fails then your strategy is no good, and most likely curve fitted.

 

[RedGreenBen]

The principle as such looks sound(to me), but the tweaking of the parameter values, will that be considered curve fitting?

Appreciate your thoughts on this.

Regards

I wouldn't worry about what it is called. You need to see if this approach has 'value'. Test the tweaked parameters against the preceeding ones and see if it improves your results.

I think I know the answer, but this approach should give you useful information.

Ben

 

[SuperDriveGuy]

Thanks Hoggum and RedGreenBen,

So, if I understand correctly, I should optimize and only use those values that stand the test of "unseen" data i.e. forward testing etc.

I believe this approach has "value", but I being newbie and all I am not sure because of the optimization results and the results from forward testing(unseen data).

Maybe I am being naive, but I believe this is part of my learning curve, that simple systems work and this is very simple.

Thanks again for your comments.

 

[pedro01]

Here's my take.

If you use any sort of oscillator, there will be no setting that fits the market for the past 10 years. As the market becomes more/less volatile, settings for oscillators have to change. There is nothing wrong at all with this.

When you backtest & have to change settings each week/month - this is fine - as long as you can change those settings each week IN ADVANCE.

If you can only come up with your oscillator/MA/Whatever settings after the event, you effectively have something as useful as tits on a bull.

 

[SuperDriveGuy]

When you backtest & have to change settings each week/month - this is fine - as long as you can change those settings each week IN ADVANCE.

If you can only come up with your oscillator/MA/Whatever settings after the event, you effectively have something as useful as tits on a bull.

I like this

I guess the trick now is to find those settings in advance

 

[Adamus]

It is important with any auto strategy that you test it on unseen data. If it fails then your strategy is no good, and most likely curve fitted.

There are also many ways to define 'fail'. The system might lose money, but if it loses money in the same way that it lost money in your back-testing, then it's still potentially OK.

If it loses more money or quicker or longer or whatever that is uncharacteristic of your backtesting, then it's curve-fitted.

 

[Dacamic]

Rather than search for a single best-performing parameter set, I prefer finding a set that is within a range of sets that perform well. Using your example, I might settle on Stoch(14) if 12,13,15 and 16 also show promise. On the other hand, I would avoid Stoch(14) if its parameter "neighbors" perform poorly. In cases where a suitably robust parameter range is not found, it might make sense to abandon the system.

With the above being said, Pedro01 is correct implying that a static system might not be successful across a variety of market conditions. Speaking for myself, I find it necessary to anticipate market conditions, and then match systems -- including parameter sets -- to those conditions.

 

[rathcoole_exile]

I have created a simple system based on Stochactics

whooo hoooo !! more money for me !!!!!!

seriously.
thanks for donating your hard earned sovs to the market. I'll drink to you when I'm spending them....

 

[pedro01]

whooo hoooo !! more money for me !!!!!!

seriously.
thanks for donating your hard earned sovs to the market. I'll drink to you when I'm spending them....

Sarcasm runs deep in this post.

 

[SuperDriveGuy]

whooo hoooo !! more money for me !!!!!!

seriously.
thanks for donating your hard earned sovs to the market. I'll drink to you when I'm spending them....

Hi RE,
I understand where you are coming from

I have been on these forums long enough to know that "most" do not recommend indicators for basing trading decisions on, as they lag, but for confirmation maybe..

I agree with pedro, dacamic, hardy et al when they say that the best parameters need to be known in advance, which is a tall task(IMHO) as they are only apparent after the event. Unless somehow one has knowledge of the cyclic patterns on market and what osc parameter would it well on a day to day, week to week basis!!!

Just thinking aloud here, has anyone tried for example to run an optimizer for half(quarter/two thirds etc etc) a day, week, month etc on an osicallator and found that the best(or near best) parameters perform well for the rest of day, week or month respectively?

The main question being assuming that a cyclic pattern is what we are after, is this pattern apparent early in day or even after 3/4th of a day, because if it is and it is identifiable then trading profitably with for even quarter of a day would be worth it.

I will try this myself as well, but most people on this forum have already been through this phase and can provide valuable inputs.

Thanks in advance.
Regards

 

[Stewenson133]

absolutely yes... imo

 

[SuperDriveGuy]

absolutely yes... imo

Hi Stewenson,

I post was rather long, you agreed to which part? please elaborate.

Thanks

 

[alan5616]

Thanks Hoggum and RedGreenBen,

So, if I understand correctly, I should optimize and only use those values that stand the test of "unseen" data i.e. forward testing etc.

I believe this approach has "value", but I being newbie and all I am not sure because of the optimization results and the results from forward testing(unseen data).

Maybe I am being naive, but I believe this is part of my learning curve, that simple systems work and this is very simple.

Thanks again for your comments.

I'm sorry to tell you this but, you are being naive. In choppy market conditions, a stoch indicator can be a useful trading tool. However, it's a major loser in a trending market.
Trying to curve fit the settings, after the event, is a futile exercise.

Very few technical indicators are worth spending much time with. You'll, probably, find that multiple MAs and crosses are as good as anything on back test.

 

[rathcoole_exile]

I understand where you are coming from

put all your lagging indicators into a big pile and set fire to them,
from the ashes will rise a new, stronger, better SDG

 

[SuperDriveGuy]

put all your lagging indicators into a big pile and set fire to them,
from the ashes will rise a new, stronger, better SDG

Alan, RE,
Any advice then what I should be stuying instead of indicators?

Price Action? where do I find info on that? Is it basically support and resistance, trendlines? have you used these in mechanical(read automated) trading systems?

I am not looking for a shortcut, but a hint towards the correct path even if long arduos will keep one motivated in the long journey that there is definitely the destination at the end of the road

Thanks for your inputs.

Regards

 

[Dacamic]

Alan, RE,
Any advice then what I should be stuying instead of indicators?

Price Action? where do I find info on that? Is it basically support and resistance, trendlines? have you used these in mechanical(read automated) trading systems?

I am not looking for a shortcut, but a hint towards the correct path even if long arduos will keep one motivated in the long journey that there is definitely the destination at the end of the road

Whether good news or bad, each person must on their own figure out which techniques will work well for them. For example I use indicator-based systems, while others obviously do not. Even within a given subset of traders, it still fascinates me how widely diverse their approaches are. Regardless, it is not enough to find something that works ... you have to find something that works for you.

 

[Stewenson133]

Hi Stewenson,

I post was rather long, you agreed to which part? please elaborate.

Thanks

of course. not read all 3d but:

in my opinion, i simply think that changing parameters every week is curve fitting. more than two parameters can drive to overfitting, test your system over muliple time series (e.g. all dji components and so on)

 

[alan5616]

Alan, RE,
Any advice then what I should be stuying instead of indicators?

Price Action? where do I find info on that? Is it basically support and resistance, trendlines? have you used these in mechanical(read automated) trading systems?

I am not looking for a shortcut, but a hint towards the correct path even if long arduos will keep one motivated in the long journey that there is definitely the destination at the end of the road

Thanks for your inputs.

Regards

Dacamic's reply is spot on. There are no short cuts.

Nobody is born a successful trader. Those that make it to the top of the profession do so after many years of making mistakes and losing money. They, eventually, find a strategy that works for them and stick with it.

 

[meanreversion]

You need to backtest in order to better understand the system. During this process there will be periods of performance and underperformance, and it's important to understand why and when the system works/fails.

One way of making the system continuously "adaptive" without adjusting parameters every week is to incorporate volatility into your stop loss and take profit targets. So for example, you could stop out 2 ATR away and take profit 4 ATR away (in whatever time frame you choose to operate). That way you can keep the period of stochastic (I prefer RSI myself) constant as the system automatically allows for greater volatility.