Walk forward repetition quantity.

Forums › ProRealTime English forum › ProOrder support › Walk forward repetition quantity.

Tagged: 

Viewing 15 posts - 1 through 15 (of 23 total)
1 2
  • 02/07/2018 at 6:42 PM #61883
    Vonasi

    So finally today I got around to transferring everything out of my old IG PRT account and into my new PRT sponsored IG account – and what a faff it was. I lose all my old history of both live and demo strategies that have had running so had to record it all manually with pen and paper(!) as there is no way to export the results. I then had to set up all the live and demo strategies again one by one to run in the new account. I still have to import all my indicators but I’ve had enough for one day.

    So now at least I have a few new toys to play with such as 200k bars for testing and more repetitions for walk forward optimization and it is this last point that I have a question about. Before I would just use 70/30 and 5 repetitions with a dummy variable and get a base figure for strategy robustness but now I can do 20 repetitions! So what is likely to be better for analysing robustness 5 repetitions or 20? With 20 I will obviously get more OOS periods that perform badly and more IS periods that perform astoundingly. Which is best 5 or 20 or both?

    02/07/2018 at 8:59 PM #61897
    GraHal

    Not an answer to your question, but another question, sorry Vonasi

    As an example, we optimise several variables over 10k bars and then use the optimised variable value in the code.  The use a dummy variable (variable does not appear in the code?) to enable the WF to run.

    Attached are the results of above.

    What / where are you seeing this base figure for strategy robustness that you mention above??

    Thank You
    GraHal
    PS Anybody can answer of course

     

     

    02/07/2018 at 9:29 PM #61907
    Vonasi

    GraHal – when I say base figure I really mean base level as a starting point to decide whether changes we make improve or detract from a strategy. The overall Walk Forward Efficiency is our first base level and then we can look at how that is compiled by comparing each of the out of sample period annualized percentage ratios and see how evenly spread the results are. A perfect Walk Forward result would be 100% Walk Forward Efficiency for Overall Efficiency and 100% Walk Forward for each annualized OOS period. We then know that our strategy will do exactly in the future as it has done in the past. We will never achieve this but the closer to it we are the more robust our strategy is.

    My question is whether lots of OOS periods tested is better than a few?

    I guess I know the answer – any test is a good test and so there is no right or wrong number of repetitions. When you add the combination of repetitions to the possible ratios of IS and OOS (i.e. 10/90, 20/80, 67/33 etc) we have an awful lot of possible test combinations and it is always interesting to have other’s opinions on things like this and what they use or feel should be used.

    02/07/2018 at 10:08 PM #61911
    GraHal

    Thank you and I agree; I also like others opinion and fresh eyes in case I am missing something! 🙂

    Where are you seeing … The overall Walk Forward Efficiency?

    I can see efficiency figures for each OOS period, but not an overall WF efficiency? Do you mean add up the OOS % figures and divide by the number of OOS % figures. So in my case above this would be 982 Total / 5 OOS figures = 182 for overall WF efficiency??

    If yes to above, do you consider 182 is good / realistic or maybe too good to be true and therefore unlikely to be seen in real live trades??

    Thank You
    GraHal
    PS I just noticed I used Anchored mode and I usually always use Not-Anchored.

    02/07/2018 at 10:20 PM #61913
    Vonasi

    I’m talking about the Walk Forward Efficiency Ratio that replaces the Time In Market on the Detailed Report…. but now you mention it when I check the maths I am struggling to work out what figures this is calculated from. Any ideas….. (……Nicolas)?

    02/07/2018 at 11:30 PM #61920
    GraHal

    I was missing the WF Efficiency Ratio altogether! I have my screen well zoomed in and the WF ratio is below the visible area unless I scroll down!

    I tried a few ratios and I can’t suss out where it comes from either!

    Cheers

    PS Sorry, but I cant think of anything meaningful re repetitions / more is better etc, I’ll sleep on it! 🙂

    02/08/2018 at 11:54 AM #61988
    verdi55
    Vonasi wrote:

    I lose all my old history of both live and demo strategies that have had running so had to record it all manually with pen and paper(!) as there is no way to export the results.

    Sorry, I don’t get this. From my demo account, I can of course export all strategies as an .itf file and reimport them elsewhere; in case this should not work, you can always do simple copy and paste with any codes and transfer them this way.

     

    02/08/2018 at 11:59 AM #61991
    verdi55

    Also, when PRT should freeze (which it never does on my computer), you can always close any position immediately on the IG website. Using the IG website even allows you to stop a PRT strategy without closing a position : when you change a stop or a limit on the IG website, this will stop the corresponding PRT strategy, but will keep the position open.

    02/08/2018 at 1:25 PM #62016
    GraHal

    verdi55 I think Vonasi is talking about the performance history / profit / loss on the Strats that he has lost.

    Yes I manipulate / close Strats a lot via the IG Platform. Another little dodge I do is if I can see that the market structure is favourable for a retrace of a loss making Strat I will set  a manual limit in IG and of course this closes the Strat but keeps the  Trade open. I have many times rescued a Strat this way that otherwise would have closed at a loss due to it’s coded Limits / Stop Loss etc.

    I guess I am a manual trader at heart!? 🙂
    GraHal

    02/08/2018 at 1:45 PM #62018
    Vonasi

    Yes I did the export/import thing for strategies and indicators which was a pain as you cannot highlight all and you also end up having to deal with duplicates as some are in both LIVE and DEMO and some are not. A right faff. Then I ended up with such a mass of files to search through that I found it was quicker to cut and paste (CTRL A CTRL C CTRL V) than try to find each file that I wanted to import and set running. Plus it used far less of my 4G data!

    GraHal is right – it is the history of the strategies that I had on forward test in DEMO and the history of the strategies that I have run LIVE that I have lost by migrating to a new account. So now I have to check the new results in my new demo account and then add the results together with those that I have written on a bit of paper from the old account – how high tech!

    02/08/2018 at 1:48 PM #62020
    verdi55
    GraHal wrote:

    verdi55 I think Vonasi is talking about the performance history / profit / loss on the Strats that he has lost.

    Ahhh, ok. But you can still export the order history (ctrl o – automated trading) by dragging and dropping it into Excel. It remembers the last 500 orders, their result and the name of the corresponding strategy.

    02/08/2018 at 2:22 PM #62022
    Nicolas

    The calculation of the WFE presented on the summary page of the detailed report is as follows:

    Σ annualized earnings for OS / Σ annualized earnings periods for IS periods

    It is wrong if the selected end date of a ProBacktest is real time. If the date is fixed, it is correct. The correction is in progress and will be available in the next update of the platform.

    About how many iterations you should make with WF is up to you and the results you get when you change it. If you find that doing 3 optimizations of your variables (3 IS+OOS) each year is better (according to your own appreciation) than doing 10 ones, then do a new optimization each 4 months.

    1 user thanked author for this post.
    avatar Vonasi
    02/08/2018 at 2:36 PM #62023
    Vonasi

    Thanks for that info Nicolas. Regarding the number of repetitions my interest was more in using the WF as a test of strategy robustness using a dummy variable rather than using it for optimization of variables. Lots of repetitions obviously breaks the IS and OOS periods down to smaller sizes and can give us some idea of the extremes of how badly or wonderfully a strategy might perform over short periods and less repetitions will give us a longer term view of strategy performance. Kind of looking at the short game and the long game I guess.

    02/08/2018 at 7:55 PM #62069
    Henrik

    Vonasi, what do you mean by doing a wf with a dummy variable? Feels like it’s no use to do a wf if the strategy is already optimized over the hole period.

    About the repetitions I think it depends on the type of strategy.

    Regards

    Henrik

     

    02/08/2018 at 8:37 PM #62071
    Vonasi
    Henrik wrote:

    Vonasi, what do you mean by doing a wf with a dummy variable? Feels like it’s no use to do a wf if the strategy is already optimized over the hole period. About the repetitions I think it depends on the type of strategy. Regards Henrik

    Quite simply all you are doing is setting one variable – call it ‘Dummy’ that is set from 1 to 1 step 1. This way you test your strategy with your chosen fixed variables over the test period and break it down into in sample and out of sample periods. The result is an indication of robustness with those fixed variables.

    Personally I feel that using Walk Forward testing with changing variables for each IS and OOS test is of little use. You just curve fit the variables to the recent data in the last IS OOS run. If the variables have a big enough range to choose from then you could make every IS and OOS period look pretty damned good – but pick fixed variables and test over the complete period and you have a good idea whether those variables have a history of working well or what your likely short term hit or gain could be.

    I am quite happy to be proven wrong on this.

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 23 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
drawsegment 5 bars forward

Create your free account now and post your request to benefit from the help of the community
Register or Login