Strategy Robustness Tester
Forums › ProRealTime English forum › ProOrder support › Strategy Robustness Tester
- This topic has 161 replies, 20 voices, and was last updated 3 years ago by Roger.
Tagged: casino, robustness, walk forward
-
-
09/29/2019 at 7:42 PM #108805
Vonasi
- Is the the logic behind using Qty / Random combo to give more than 100 results so that we input the worst 100 results into the Robustness Tester and so we have a worst case scenario??
- What do you think of the result attached for the code attached discussed on the link
https://www.prorealcode.com/topic/backtest-results-change-when-i-stop-using-variables/page/2/
09/29/2019 at 9:07 PM #108813- No – we should never have more than 100 tests as we need to see the bad ones – the bad ones tell us more about a strategy than the good ones! The optimiser results only ever give us the best 100 results.
- I had a look at that strategy and started checking it out but then dinner, a few drinks and the girlfriend wanting to watch some American drama on the laptop sort of got in the way!
1 user thanked author for this post.
09/30/2019 at 9:00 AM #108863I have created yet another robustness tester. I call this one the ‘Robustness Tester – WF Style’ because it splits the history being tested into lots of smaller sections and tests just on those sections.
To make the tester work it needs to know the value of barindex for the last bar on the chart. You can simply get this with the indicator RETURN BARINDEX and then enter the last value into the tester as BARSONCHART. You can also set a start date. Set this to zero if you want to test on all of history.
As with all the other robustness testers add the tester to the start of your strategy (under any DEFPARAMs) and ensure that all variables in the strategy are fixed values. Then add TRADEON to your entry conditions (being aware of how to exactly do this as previously discussed in this thread). Then you optimise the strategy with two variables DIVI and DIVI2.
DIVI one is how many bigger pieces the history being tested is divided into and DIVI2 is how many smaller pieces these bigger pieces are then divided in to. So with DIVI set to optimise from 1 to 0 step 1 and DIVI2 set to optimise from 1 to 4 step 1 we will have 40 different sections of data to test on. Be careful of setting the maximum values for DIVI and DIVI2 too high as otherwise the data is split into very small data samples that will not tell us very much at all when tested.
The results can then be dragged into Excel and analysed exactly the same as for the other testers.
1234567891011121314151617181920//Robustness Tester - WF Style//By Vonasi//20190930barsonchart= 2286// total number of bars on chartstartdate = 19950101// earliest start date of test period - set to zero to test all of historyif opendate = startdate thentestrange = barsonchart - barindextestsize = (testrange / divi)startindex = barindex - (testrange / divi)endifstartbar = startindex + (testsize * divi)endbar = startbar + ((barsonchart - startbar) / divi2)tradeon = opendate > startdate and barindex >= startbar and barindex <= endbar//graph barindex10/26/2019 at 8:39 PM #11128710/26/2019 at 9:30 PM #111298Yes it is but that strategy has more BUY and SELLSHORT orders than I think I have ever seen in one strategy and it looks like it has the possibility to swap directions too and so adding the TRADEON condition to all the right places and still allow direction changes is going to be a bit of a mammoth task (possibly better done when I haven’t had a couple of glasses of wine!)
Changing trade direction always dilutes any robustness test a little – the more you change direction the less the robustness test can be trusted. Also because it is a 5 minute strategy the amount of data available for robustness testing is very limited compared to all of history so I am not sure exactly how conclusive the test will be but it is worth a look.
I’ll try to find time!
1 user thanked author for this post.
10/26/2019 at 9:47 PM #11129910/28/2019 at 9:43 PM #111465I finally managed to find some time to try to add the ‘Robustness Tester – Random’ to fifi743 ‘s Vectorial DAX M5 code. I think that I added it correctly and it turned out not to be as difficult as I first thought despite the masses of BUY and SELLSHORT instructions in the code! In reality not many of them were buys or sellshort when not reversing direction.
The test on this strategy is very slow!!! – so results took quite a lot of time to get.
I tested with settings of 10/5 and 20/5 for QTY and RANDOM. So that is 45 random samples and 95 random samples. See the attached images for the results. The equity curves are for the lowest resulting average gain per trade in each test.
At first glance the results look pretty good. In the 20/5 test the average gain score is not very high but all tests in both tests resulted in positive equity curves and equity curves that closely resembled the original equity curve for the back test on all candles. The combined score in the 20/5 test was still pretty good thanks to the win rate% score being very good.
It would seem that the strategy on the data sample tested is pretty robust especially considering that the sample is very up and down and not just a bull market or a bear market. My only reservation being that the data sample of 100k 5 minute candles is a very small data sample considering the amount of history that precedes it that we have not tested on! I have attached the ITF file for the strategy modified to include the robustness tester random code. If anyone has 200k bars and wants to test it then it would be interesting to see the results.
10/28/2019 at 10:08 PM #11147510/28/2019 at 10:43 PM #111478I found myself copying and pasting the VRT snippet from one System to another and it got on my nerves, so …
Here’s the VRT snippet as an .itf … if you make it available in your System backtesting list then it’s real easy to copy into the various strategy codes.
All thanks goes to Vonasi of course!
Hi GraHal, could you give us this bot to try it on demo? please? Thank you
10/28/2019 at 11:48 PM #111479could you give us this bot
What bot are you referring to?
I included the VRT.itf in the post that you quoted.
1 user thanked author for this post.
10/29/2019 at 11:11 AM #11151710/29/2019 at 11:19 AM #11152010/29/2019 at 11:48 AM #11152310/29/2019 at 1:52 PM #111540Are you saying that you do not know how to use the vrt.itf in a bot in order to test your own bot’s robustness?
No, what I want is to put the bot in demo mode and it gives me an error
10/29/2019 at 2:39 PM #111550 -
AuthorPosts
Find exclusive trading pro-tools on