Avoid equity curve fitting with ProBacktest trading strategy optimisation

Optimisation of a strategy is a major part of automated trading development. Despite its simplicity with the ease of use of ProBacktest optimisation module, there are a lot of traps where algorithmic traders almost always falls.

Optimising already made strategies can seriously lead to equity curve fitting phenomena. How to avoid this and take advantage of it, this is what this article is all about.

Everyone who rely on automatic trading wanna find  the best algorithm that almost always sweet the best performance over the little among of time possible. Within these two factors, we try to build perfect performance over past data, while adjusting indicators parameters or even better stoploss protection. Building the most the better strategy is not a good idea, we must try to find an edge between risk adjustment and not so greedy objectives of performance.

But, many traders may find the optimisation process a good idea to find the best parameters over past periods of the instrument they created their strategy for. This irremediably leads to equity curve fit of the best smoothing and good looking of past equity curve performance, the more is climbing without doing stair steps, the better it is, but of course, as you may know, it is not as simple to build your first hedge fund !

Optimizing a trading strategy, where does it fail ?

Let’s imagine we are on the last day of December, year 2014, we have just finished elaborate a new trading strategy and we are going to make an optimisation of it through the last 2 years of data we have.

We build a simple RSI 50 value cross, with simple long only strategy on the SP500 on daily timeframe. The RSI “n” period is optimising from 3 to 60.

Optimisation of the strategy for the last 2 yeas gave us a good looking result, very impressive indeed : +964% with a RSI period of 4, everything look alright, but definitely “curve fitted” for sure :

Now that we believe that RSI [4] periods is best to fill all trading opportunities in the future, let’s give it a try :

Oops.. Seems that we have failed somewhere ..

The conclusion is that the parameters that suit best in the past, will never give you an advantage on the market. Remember that any optimised trading strategy over a determined time series will give you the same result in the future.

Now for the whole :

Strategy still in gain for sure (because first part were optimised), but who will suffer that 40.000$ loss in less than 6 months ?

So why don’t we optimise then the whole time series for future trading purpose ? Because tomorrow is not yesterday, and you’ll be in the same worry than for now, so we’ll have to move on next part to learn how to get rid of that phenomena.

Avoid equity curving :

Because of past data behaviour, we cannot only rely of past performance. Each minor price change in the past is the conclusion of millions of traders interactions each previous days. So, we can conclude that the 24 moving average periods we found for filtering our entries in the past, while optimising, is just an average of what it would be OK in the past for this exact purpose. As we were optimising a strategy over, 90 days as an example, how do our 24 periods would react for a 120 days optimisation process ? I got the answer : it would surely fail, because of new data include in this new time series, and it would be exactly the same with future data that do not exist at this moment.

To avoid this, we can try to divide our time series into 2 different series to test our optimisation, let’s call them :

• the “in sample” time series (that represent 2/3 of our whole data)
• the “out of sample” time series (the last 1/3)

It’s always a good idea to divide our time series with the last 1/3 as the out of sample period.

For that example, we are using almost 5 years of data from 2010/01/01 to end of October 2015.

Let’s build a more “sophisticated” strategy with a real cross of the 50 level of RSI with trailing stop % with parameter (p) from 0.5 to 5% within 0.5 step on optimisation :

1/. First, let’s optimize the 2/3 of data, from 2010/01/01 to 2013/03/31 (in sample)

We find that the best fit is an RSI period of 3 with 0.5% trailing stop :

With a decent profit of +915%, let see how this RSI[3](close) and 0.5% trailing will act in the last 1/3 of data.

2/. Test this parameters from 2013/04/01 to end of our data series (out of sample)

As you see, OOS (out of sample) time period act as a validation, or not in this case of the “in sample” best fit. We ran on a margin call at the beginning of 2014.

3/. While in the first step, we get a report with a lot of optimised pass, now that we know that the best IS fit doesn’t work in OOS, we have to test other parameters that fit also well in the past :

Just try to get result from the second optimised parameters, as RSI[34] with 0.5% trailing stop :

OK, that’s better, at least we have not run out of money this time, and we stand till the end of the period as well :), but oh OK, that’s a crappy strategy…

Conclusion :

Now you have understood the power of “in sample” and “out of sample” optimisation and/or validation, if you think you had made a good automated trading strategy that run very well in the past, you should test it on at least 1/3 of future/present data before making a real investment on it.

Of course there are a lot of other more sophisticated process to avoid curve fitting (e.g: walk forward analysis, Monte Carlo tests, etc.), but that’s a story i will tell later.

Share this