How many possibilities with ProBacktest for a system with three parameters?

Forums › ProRealTime English forum › General trading discussions › How many possibilities with ProBacktest for a system with three parameters?

  • This topic has 13 replies, 2 voices, and was last updated 2 years ago by avatarJS.
Viewing 14 posts - 1 through 14 (of 14 total)
  • 04/01/2022 at 10:39 PM #191011
    JS

    Suppose I have a system with three parameters and I want to optimize that system, how many possible combinations are there?

    For example, suppose parameter 1 is related to the system’s look-back period and runs from 0 to 5000.

    I want to backtest all periods so parameter 1 runs from 1 to 5000 with steps of 1.

    Suppose parameter 2 and 3 are a kind of thresholds, related to buy and sell, and run from 1 to 10 with steps of 1.

    (Of course you can run the different parameters between other values, but it is now about the calculation of the amount of combinations)

    How many possible combinations are there?

    Is that 5000 x 10 x 10 = 500,000 combinations???

    04/01/2022 at 11:45 PM #191013
    PeterSt

    Hey JS, – Because you are too much of a math guy to know the answer to your own question (which would be Yes, 5000 x 10 x 10 = 500,000), what is the underlaying real question hence culprit ?

    I don’t want to be your pain, but probably am ?

    My own point would be that most often we can “see” that a large number of options are mutually exclusive, but we can’t eliminate them. Like Bollinger Bands but not knowing the mid point yet. So indeed, if PRT would be able to cover for that, it would save us ages of backtesting (and would save as much server capacity).

    Is that what you are hinting at ?

    04/02/2022 at 12:02 AM #191014
    JS

    Thanks @PeterSt

    No, I was wondering when there are 500,000 possible combinations in this case and the Optimization report shows the 100 best combinations, then that is 0.02% of all possible combinations.

    What I would like to know is how the rest of the combinations score?

    04/02/2022 at 12:12 AM #191015
    PeterSt

    Aha. But are you saying that only the best 100 show ? … I think long ago I made a remark in this realm (want to see them all !). But to be honest I forgot about that while today I would explicitly be the most interested in the “extremes” and see the equity curves.
    So are only the 100 best results shown ? I really forgot about that.

    PS: I vaguely remember that something else is going on. Like at first all the results show, but later (during the optimisation) results are taken out. *This* is still so, but I can’t tell why that happens. Can’t mimic that either. I just see it happening sometimes.

    04/02/2022 at 12:21 AM #191016
    PeterSt

    Probably not what you mean, but there’s “intelligence” in knowing redundant results. The below incurs for 1001 combinations, but still only one shows. This is because all the results have the same outcome. I am not even sure I like that, but at least it is faster because of it.

    04/02/2022 at 12:27 AM #191018
    JS

    @PeterSt, during the optimization process, the “lesser” combinations are replaced by “better” combinations until you have the 100 best ones left. (So the “lesser ones” fall out)

    I’m actually looking for data that shows me if a system is robust.

    Based on 0.02% of the combinations, I don’t think I can say that.

    04/02/2022 at 12:38 AM #191019
    PeterSt

    Or this one. Only 21 results implying a real difference. And the 21,000 combinations really only show 21 results (you can count the results in the list).

    The third attachment is interesting (??) because it shows Dummy2 with a result of 1 again, while it should not. But see the fourth attachment and where the 23 comes from.

    04/02/2022 at 12:40 AM #191024
    PeterSt
    JS wrote:

    I’m actually looking for data that shows me if a system is robust. Based on 0.02% of the combinations, I don’t think I can say that.

    Yes, agreed.

    04/02/2022 at 12:54 AM #191025
    JS

    @PeterSt

    True, and that’s why I’m looking for those other 99.98% of the combinations but I can’t find them anywhere 🙂

    04/02/2022 at 1:08 AM #191026
    JS

    I have also looked at the 3d graphs but they are totally incomprehensible to me…

    How do you set that up and what are you looking at???

    04/02/2022 at 4:17 PM #191055
    PeterSt

    Hi @JS,

    Yes, it is true that the 3D graph would show all. But how to explain its working without an extensive video … seems undoable.

    I have a result list which is 441 (results) long here. Notice that you must first make is sensible, in the sense of it not showing 300 results which are all 200K negative, because then all is compressed and you won’t be able to make anything of it. Making anything of it means : hover your mouse over the various results and find the correlations.

    First of, zoom OUT so you will see the captions of the axises. By default at least the bottom one is out of view.

    Now drag the graph such that you have a clear view on the most gain. This is only easy when you know it in advance, and you do by means of the normal table view of the results. I know from my situation that this is 5215.62. You can see that I positioned the graph such that I am able to find it visually to begin with.

    Never forget that besides the two (optimisation) variables you can choose, there’s also the choice for the z-axis, by default filled with the Gain (Winst as it shows here).

    You can see that wherever your mouse is at, the X- and Y-axis show your two chosen variables and it highlights the current position. So I can see that the best RSIEntry length is 17.45 and the best RSIEntryDepth is 0.668. Would I move my mouse a bit to the right, to the next peak visible, I would find that that too has the gain of 5212.62, but with a different set of variable values (attachment 2).

    After repositioning the cube somewhat (att 3), the third attachment shows nicely the “Linearity” of my chosen range and that the worst combination follows from the lowest value of both variables. Notice the yellow 2D graph on the z-axises (both the backwall and the right-hand side wall).

    More to come …

     

     

     

     

    04/02/2022 at 4:41 PM #191060
    PeterSt

    For the other end of the spectrum (spectra – all overshooted) you can see in att.1 that I put my mouse on a kind of random position on that straight line. But notice again the two z-axis planes. Also look at the white vertical line in the left-hand plane. It is representative for the 13.47 you see on the EntryLength axis. Also notice that the yellow graph you see in the right-hand plane resembles the red graph going upwards from the mouse.

    Att.2 shows me how the worst part for the EntryLength axis ends at a lower value of 17.37 and where you can again see its yellow and red graph correlating (in the 2D and 3D map).

    Would I put my mouse at the other end of the 3D map, then you’d see in att.3 how I actually now am looking at the 2d graph in the left-hand plane. Again compare the yellow line on that plane with the red one in the 3D map.

    I’m thinking on what to explain more, but all becomes the most explanatory when moving the mouse and looking at the axises which numbers highlight.

    Fact would be that the 3D map will show you all the results.

    One more post to follow.

    04/02/2022 at 4:45 PM #191064
    PeterSt

    What I wanted to explain because the 3D graph showed it already (by moving the mouse), but which is too hard to explain, now is visible in the 2D map. Look :

    This tells me what I saw in the 3D map – the RSI Length is less important than the RSIDepth. In att.1 you see an equal-ish spread. Not so in att.3 ! It is clear that the middle of the spectrum has the preference. And you should be able to see this at a glance (ahem) in the 3d graph.

     

     

    04/02/2022 at 7:02 PM #191081
    JS

    @PeterSt

    Thanks for the time and effort to explain the 3d charts.

    What complicates it for me is that I have three parameters instead of two.

    What I would like to see is how parameters 2 and 3 behave in relation to parameter 1 (the look-back period).

    There are four variables above the 3d chart: the three axes x, y, z, and the heat map (the rectangular “blurry mirror” that reflects only color) with a variable.

    How do I set these variables now to see how parameter 2 and 3 behave relative to parameter 1.

  • Author
    Posts
Viewing 14 posts - 1 through 14 (of 14 total)

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
2 trading strategies in one system
a completely simple nas macd system
auto tradeing system
bug probacktest duplicate limit orders
cascade ordering system
créer probacktest avec indicateur ssl hybrid
dax dayly: system that works.
default prt vwap bands (daily) not selectable for trading system
divergences for many indicators pinescript
expiration date for my system

Create your free account now and post your request to benefit from the help of the community
Register or Login