25 minutes later … you could say I found the culprit. At first I was happy, but this lasted 30 seconds and then I realized that it requires PRT-Premium to solve it.
All right (not), it’s all in my posts in this topic, but it was only after 25 minutes before I thought of that myself : the live data. Or better said : the not-historical data;
When we run Backtests in V12, doing so on the normal data everybody with PRT-Complete has, will fail. And not a bit – I don’t know how many variations I have seen. Below .itf can be run on S&P500, which I did with a 1 minute TF on the chart in PRT-IB. It is set up such that you have some variation in the output (Optimization Result) and can keep on clicking in there. Subsequent clicks on the same output line for the maxOrdersL Parameter will mostly yield different results. Or say half of the time. But it is really more bad than imaginable because I have had at least 3 base situations with different “switchable” results. Thus, whatever you do, the results will switch between two sets (say one with 8 trades and the other with 16), but reload the backtest in the chart and it changes to 6 and 12 (the results are completely different). Of the latter you see an example below, looking at the lines with 6. Thus where you see the 6, clicking that line may show a Detailed Report with 6 trades (good in itself) or 12 trades (wrong). … This is what everybody sis saying all the time in this topic.
Notice that you can not subsequently click the same line – instead you must switch between two output sets (say the top line and the 3rd line in what you see below).
What I try to say with the above, is that the base set – of which you see one version below – can change completely when you reload the backtest in the chart. Gain is different and the two result lines per backtest are also different. This makes backtesting 100% worthless because I can’t tell which is correct. Okay, *I* now can see which one is correct, because I can yield the absolute correct one, but this requires PRT-Premium. … With History data (the selection of 1M bars) all is fine. And this is how I never noticed the issue, because I always use History data these days. Why ? because else I see changing results throughout the day, just because the data changes (adds with new bars). Again, you see the hints it in my previous posts, although I could not guess the solution.
… which is no solution for most because most won’t have Premium …
With Complete hence no-History data, the exhibits vary quite a bit. Below you see the gain 537 which is the good result for clicking the Result Line with 537 on it. But it can also show the 1050 when the 537 line is clicked. Or it shows the DD belonging to the 1050 line. This with variations of the number of trades, which is actually what people talk about as the only culprit (as far as I recall) because that is so obvious. But it is thus worse, and will depend on what you show in the Detailed Report anyway.
For PRT : What internally goes wrong with the live data is that elements are not reset. You can see this happening because when reset, the whole Detailed Report form briefly flashes (it disappears from screen for a fraction of a second). When it does not reset, you see it cumulating (which is a nice gag). Funny is that with the Historical data you always see it cumulating. And thus what kind of happens in the wrong situation is that it cumulates on top of the previously shown situation.
I have been trying to find “resets” but did not succeed in that so far. And when I found the solution in Historical data I stopped. As it is now, everybody will be able to test it with the below .itf and focus on reset possibility. And then still it won’t be solved, because reloading all from start just yields completely different results, unless we can find the reset of that too.
Again for PRT : When I had issues with the Detailed Report myself in V12 (say 6 months back) I described “smart caching” which failed. I don’t even know whether this was ever solved, because when I could not backtest because of nothing showing in the Detailed Report, I hopped back to V11 and worked there for a couple of months. And in there I started using Historical data, just because of the amount of data I needed newly (for my means of optimizing). Now back in V12 with backtesting for yet another couple of months, I only use Historical data there too. And so I think it was never solved, except for data showing again.
I should be able to point to this post with reference to the same ticket from back then, because apparently it was never solved. But also, I have direct access to Support because I am an IB user, and not IG in the base.
I will file a new ticket (Technical Report) as well, and will do that somewhere tomorrow (Sunday). Maybe someone finds the reset possibility in the mean time. And otherwise : back to V11.
.
