Strategy Robustness Tester
Forums › ProRealTime English forum › ProOrder support › Strategy Robustness Tester
- This topic has 161 replies, 20 voices, and was last updated 3 years ago by Roger.
Tagged: casino, robustness, walk forward
-
-
07/12/2020 at 3:45 PM #139086
No robustness test is a bad test compared to not robustness testing at all. However we should always keep in mind to ignore the extremes especially those test results that have very few trades in them. Better to delete them from the spread sheet results and not include them in the the analysis charts at all. Set a minimum trade quantity level and delete all test results that don’t meet it.
It is always a nice idea to include GRAPH TRADEON at the end of your strategy so you can see the sorts of trading sequences that your QTY and RANDOM settings are producing.
The need to take an average of lots of different robustness tests and the fact that they still might not see every possible trade along with this need to filter out the extreme test results is why I much prefer my ‘strategy every bar’ testing. Unfortunately this SEB method does not work easily on faster time frame strategies due to the massive number of strategies you are required to run to ensure that you miss as few trades as possible.
1 user thanked author for this post.
07/13/2020 at 8:38 AM #13911109/24/2020 at 8:55 PM #14527809/24/2020 at 10:58 PM #145285Dumb question: if the strategy have “not onmarket” before entry conditions, is RT useless?
No. The TRADEON condition is the condition that decides whether a trade is allowed to open or not and it can be used with any other conditions including only opening a trade only if “not onmarket”. TRADEON must be checked only for the first market entry and not for following market entries if accumulating positions are part of the strategy.
1 user thanked author for this post.
10/25/2020 at 7:08 PM #14843710/26/2020 at 12:53 PM #148502@ullle73, those are v unusual results. Either you have produced the most robust system ever … or there’s an error in your testing. It’s quite rare for the av gain per trade to go above 50, and I don’t think it’s possible for either result to go above 100 (i may be wrong about that).
10/26/2020 at 5:29 PM #148548would this be an okay result according to you vonasi?
There’s clearly something wrong, check it again
10/27/2020 at 9:48 AM #148630As others have said – those results are so good that there is a possibility that they could be too good to be true. You would need to double check how you have applied the robustness code. Not all strategies are suitable for this type of robustness testing – for example the robustness results for a strategy that opens a position and then is always on the market switching from one direction to the other will be meaningless.
10/27/2020 at 11:54 AM #14864901/02/2021 at 3:54 PM #155977Hi
Phew, been through this thread today and found it incredibly interesting and useful….
I’ve put one of my algos through this and have a result below – thanks to Vonsai, Paul (and everyone) for the algos and templates.
Can somebody please take a look and give me their feedback please. I’m not looking for a “go ahead” or “dont go ahead”, just an opinion from someone who has done this before as it will help me read results better in the future.
I used QTY 2-21, RANDOM 1-5 and used all 100 rows of data from each.
many thanks!
01/02/2021 at 4:12 PM #155981Sorry to bring bad news but that is a v poor score.
The evaluation is always going to be subjective but personally I like to see something above 35 in the top box and preferably in the 40-50 range. It rarely goes above 50. The % win is always a higher score and usually less problematic but 56 is not great – I’m usually looking for 80-90+ there.
Sorry to say but I think it’s back to the drawing board on this one. Hey ho … 🤔
NB i usually run it qty 2-10 and random 1-10 so results may vary, but not that much.
01/02/2021 at 4:24 PM #155983Sorry to say but I think it’s back to the drawing board on this one. Hey ho … 🤔
Hey, no worries! and thanks for the feedback
01/03/2021 at 12:18 PM #156067to ignore the extremes especially those test results that have very few trades in them. Better to delete them from the spread sheet results and not include them in the the analysis charts at all
With this in mind, I’m thinking it might be better to run it with something like qty 2-12 / random 1-10, giving 110 iterations of which PRT shows us the top 100? This would automatically remove the outliers, which in my experience tend to be at the bottom of the Avg Gain results.
What do you think?
01/03/2021 at 12:23 PM #156069That’s an option but I prefer just to see all 100 results and then manually delete any with hardly any trades from the spreadsheet rather than possibly lose 10% of results that could all also be fantastic and give me even more confidence in the strategy.
01/20/2021 at 10:48 AM #158640How about doing the parameter optimization (i.e., all the curve fitting) with the aid of a position entry scrambler like the robustness tester ? That means, during parameter optimization, you block several entries randomly, ideally using different blocking patterns as the robustness tester generates ? Shouldn’t this result in more robust systems from the start ?
So, could it be possible to apply position scrambling already during parameter optimization, not only in hindsight, as the robustnesss tester does ? I haven’t thought about how to code this, but would this be a feasible way to obtain more robust parameter sets for a given system ?
-
AuthorPosts
Find exclusive trading pro-tools on