Technique help to reduce over-fitting
- This topic has 63 replies, 12 voices, and was last updated 3 years ago by
.
-
I’m not sure if you grasp the mechanism of VRT test, but I think is important to know it first, and in short, ProBacktest is limited by the fact that once a trade is on market, it will not open another trade (unless you activate cumulateorders), so during a trade, you may have another signal, but not taken into account by ProBackTest. Those missing signal could have killed your strategy but you do not know as they were skipped. Thus, VRT is born, to tried to skip some barindex, in order to give opportunity for those missing trades to appear and see if they still work well. (though from software point of view, this can be improved by having multiple instances to monitor per trade, and consolidate each signal that are unique, but I doubt PRT will do that).
I wonder if it is because my strategies are faster than Vonsai and use more trades per day.
There is no problem if you have more trades per day. It is more about if during the VRT test, is your trades reduce drastically? E.g. from total 300 trades to become 50 trades. If it is reduce to 50 trades, are the trades still consistent? If yes, then your PRT result will be still good. If no, the chance of curve fit could be higher, it means you might have few big winners (in relative to your average gain), and since those big winners have disappeared in VRT, then weakness appear. And how big chance of those big winner to appear again in live trading? Without those big winners, is your profit become poor? This is quite subjective, because your strategy might be a scalper, thus more trades are in your strength…then you might consider to adjust the parameters.
can you recommend VRT QTY and Random settings please?
The game is the higher value of QTY, the less trades it will be, after you finalize with QTY, then you try to adjust Random so to have total 100 combinations (as PRT optimization window allows only 100 combinations), so you may adjust to your need, my recommendation is to have at least 100 trades for 1m TF. My preference for 1 min TF strategy is to get a score >60 in QTY 2-10/random 1-10, and further test it with QTY 2-5 /random 1-21 with >70, based on experience from my 1m strategy that have profit in live for sometime. To be honest, running VRT is already the last step, the difficult part is are you willing to give up your strategy if it scores badly, as you might have spent several weeks on it? So even it scores badly, maybe just put it in demo and let it run for sometimes and see. If VRT scores well, at least more confident to put it on live sooner… (and I can tell you for my 1m strategy, I have run many rounds of test, improve, test prove, in VRT, WFA, and other tests 🙂 )
Hope it helps, and don’t hesitate to come back and share your opinion and new findings.
Those missing signal could have killed your strategy but you do not know as they were skipped.
and that’s why often strategies have a big stoploss (= many missing signals) and a smaller profit target and fail live.
maybe if a strategy has a simple fixed stoploss & profittarget (1:1?) , you could test each signal simultaneously (and cumulative) with help of arrays in v11. Now if you have a high winratio, then it’s a true high winratio.
Then after finding best settings go back to non cumulative, add additonal exits such as a trailingstop and the system will be more robust. Doubt it’s possible though.
1 user thanked author for this post.
Not all strategies are suitable for applying the VRT code to and also sometimes the VRT code is applied incorrectly. So without the actual code and seeing how VRT has been applied to it it is very difficult to comment much on the results.
‘Qty’ from 2 to whatever you want and ‘Random’ from 1 to whatever you want (maximum = maximum qty value).
That is how it should be and the values should result in 100 or less optimisation results.
There is no correct set of values – you need to capture as many trades as possible so more tests at more different RANDOM and QTY settings can only be a good thing.
103 trades is not a lot. If they are not on the market for long then the VRT test might not find many new trades and so not tell you very much about your strategy. As I said VRT testing does not suit every strategy and if you have questions about the results then you have to look for reasons why you have those questions. We must question every strategy as much as possible to understand how it works as much as possible before we put a £1 on it.
1 user thanked author for this post.
Find exclusive trading pro-tools on 
