Walk Forward analysis allows you to optimize a trading system and to validate its robustness and stability over time.First, the walk forward analysis optimizes a set of variables on an initial period called “in-sample data”, then it tests the best parameters on the following period called “out of sample data”, and repeats the process by shifting forward the time windows. The optimization periods (in-sample) may have a single starting point (Anchored mode), or different starting points (Non-anchored mode).
By comparing the results of these 2 periods, the method allows you to:
- Check that the performance of the strategy on the optimization periods (in-sample data: shown in blue) are consistent with the test periods (out-of-sample data: shown in grey). This avoids the risk of “overfitting”
- Check how the strategy behaves in changing market conditions
- Test the strength of the strategy on past data
Practical example of Walk Forward analysis
This section shows how to optimize an existing trading system using the Walk Forward method.
First click on the ‘wrench’ button at the top of the code editor window shown below.
The window below will then appear. Once your variables are defined, you can activate the Walk Forward method by clicking on the “Active” button and choose the Walk Forward parameters. Select Non Anchored mode for different starting points, or Anchored mode for a single starting point for all test repetitions.
You can also define the number of Walk Forward repetitions (one repetition consists in optimizing the parameters on the optimization period and testing the best set on the test period), and the ratio between the optimization period and the test period.
Once the parameters are defined, close the window above and launch the Walk Forward analysis by clicking on “ProBacktest my system”.
During the Walk Forward analysis, the platform starts by optimizing the strategy on a first sample to determine the set of variables that generates the highest performance. Then the performance of this set is evaluated on an additional sample that was not included in the optimization sample. The process is repeated 5 times. The aim is to determine whether the optimal set of variables has generated consistent or greater performances in different market conditions.
Depending on the number of variables and repetitions, the calculation can be more or less complex. The backtest execution time will depend on this complexity.
After calculation, a chart containing an equity curve will be displayed together with the detailed report of the system performance. In addition to the usual detailed report tabs, an additional Walk Forward tab compares the results for each test repetition.
In order to determine if the strategy is robust, the WFE ratio (or Walk Forward ratio) compares the annualized gain of the test period (out-of-sample) to the annualized gain of the optimization period (in-sample). A ratio above 50 or 60% is necessary to consider that the strategy is not over-optimized.
The equity curve aggregates and displays the first in-sample (or optimization) period and the 5 out-of-sample (or test) periods on the chart.
When moving your mouse over the lower bands, the corresponding period will be highlighted with the parameters used, along with the performances of this time period. On the above screenshot, the mouse is over the first test period. The values shown above for the variables PROTECTION, ORDERS, and NUMBER gave the best results during the 1st
optimization period (in-sample #1) and were then applied on the test period (out-of-sample #1), resulting in a gain of 2,000 € during test period #1 (highlighted in the image above).
(this article is a copy/paste of the ProOrder documentation, I will make videos later to explain it better).
Looks promising, thank you Nicolas for your role in helping develop such a tool !
Thanks. But all credits go to all prorealtime developers who contribute everyday to make a better and reliable trading platform for all of us! 🙂
Amazing work and addition to the tool set, should prove to be very valuable. Thank you very much for this Nicolas.
Wow just Wow!
Big Thank You to the PRT Developers responsibe for this most useful feature!
GraHal
Hi Nicolas
I am using v10.3, When this amazing function avaiable?
It’s already available for prorealtime software/IB customers. It will be released in a few weeks for other users (IG/PRT-CFD).
Thanks Nicolas.
Yes! Thats it! I love PRT !!
Amazing! Impatient to discover and explore this function
Alleluiaaaaaaaaaaaaaaaaaaaaaaaaaa 🙂
I don’t remember the first time I wait for it.
Do you know if the MulTiFrame will be come soon too ?
Thanks
Zilliq
Soon but not sooner.
Anche per me questa nuova funzione potrebbe attenuare il problema della sovra-ottimizzazione delle nostre strategie. Aspetto impaziente di poterla utilizzare.
Un saluto a tutti. Fill
I’ve been expecting this functionality for years… great to read that it’s coming to PRT!
Any videos on these in english ?
Dear Nicolas,
Sorry to bother you about this, but I would like to have a perfect understanding about the PRT Walk Forward Analysis and how to best interpret WFA with five tests at once.
I would like to ask what exactly is being tested in the second test (or third or fourth or fifth test) in the example above ?
As far as I understand the second test takes a second In sample period and a second Out of sample period (same timescale as the first, but later in historical data) and for this second period the parameters for the In Sample test are optimized again, and for those parameter settings the second Out of Sample result is given. (You can see the renewed optimised parameters for each consecutive test, ticking on the (5) ” x ” below “Variables” at the Walk Forward Statistics tab)
If my understanding is correct, why comparing 4 independent out of sample tests with 1 (the first) dependent out of sample test, all together, with the first In sample test ? Is it maybe to quickly compare if the optimized parameters of the five consecutive tests are more or less the same, which would indicate something about the robustness?
The question is if a equity graph should be drown from the first In Sample result plus five consecutive Out of Sample results.
Thank you very much in advance.
The OOS are not compared with any others. You can consider each IS+OOS independent.
Then In my opinion a equity graph is very confusing, even misleading, it suggests comparabilitiy between the consecuitive tests, while in fact they are completely independent. Why adding Out of Sample results of completely independent test to the initial test in a graph ? Could in this case the equity graph not be presented ?
With 1 IS and 1 OOS it is good to present the graph, and when the parameters of each consecutive test are the same (with a dummy variable this is possible))
Kind regards, Jan
If you have X good repetitions of IS+OOS, then you have a clear idea of when you need to make new optimization. That’s the purpose of this analysis.