Backtesting kaput ? (IB)

Viewing 7 posts - 1 through 7 (of 7 total)
  • #226899

    I just spent over 2 hours filing a ticket about something being totally wrong today, unless I am blind and don’t see the cause. So if anyone seems to notice similar as this …
    Side note : this is for V11 and V12 and in two accounts of IB. I did not check IG.

    Backtest results are not the same, I think after the platform reset 06:30 (Amsterdam) this morning. But, this depends very much on what the code does. A small quote of what I wrote to support :

    If we look back in ##PRTzz304c, we can see suspects like StrategyProfit, Floor, Min. I use these throughout the program code as well, including Round and Max.
    What I am saying is : it is not necessarily the data because it can be the commands as well. I could add that these commands are almost (or fully ?) exclusively used in the MM code, which is much larger than the small part I showed. I have no reasons to use these commands elsewhere, except for StrategyProfit which I use to determine the exits of trades. Because that all keeps working (as far as I can tell) it should not be that.

    The kind of fun is that I can compare the backtests with Live from the past 3 weeks, and nothing is the same any more. This counts for 3 running Systems I checked.

    When it stopped working I did not restart PRT somewhere – the backtest results still on screen from earlier this morning, just don’t work the same any more for certain situations.
    If someone else also has issues suddenly, it would be good to know about it ?

    #226928

    This appears way more complex than I could initially see. It is possibly (probably ?) related to the last line of code you see in this snippet pulled from the program of concern :

    Obviously I am making my observations from backtesting. In this case I find that where > should work (it is what I want because I want to do things when making profit in the last trade), that doesn’t do a thing and I notice that <> does do something with effect. As if losses are now incorporated too. Special environmental to this “problem” : the backtests do not show losses and so making it <> as a temporary solution is for that moment fine with me. It only means that when there is this one loss out of 100 trades (or whatever I exactly expect), the contract amount will be increased after a loss as well. This is insignificant.

    In practice different things occur than what the eye can see; in practice, with backtesting in IB (!), there are TWO occasions when one trade unwinds/exits and when the code is called in two subsequent events (bars)  this happens – the first change of StrategyProfit is when the trade indeed exits and the second change of StrategyProfit is when the backtesting mechanism adds the broker fees. Careful, this is how it works for IB. For IG this does not happen because IG works with spread for income (in this situation which is about Futures, Indexes for IG).
    This implies a kind of cancellation of the reported profit to the code, because in the first call (at the Exit) there will be profit (can also be a loss), while the second call implies a loss (broker fee is a deduction).

    This is how backtesting works.
    In Live no such thing happens … In Live there is just less gain reported, as the broker fee is right away deducted from the gain of the trade.

    We now arrive at the more complex situation that the contracts-adding code is called twice – which out of all I already noticed and what made me halve the added contracts, so net it would be fine (Peter, this is a wrong thing to do, man). But code like this is everywhere in the program, so many more things go wrong.
    Try to envision that I had to adjust the MM code which originally was triggered by losses only, and which I could not get to work in a situation with “profits only” (MM did not do a thing now). So now I changed the MM by means of letting it act on profits, but because profits also imply losses each time, each trade reports a double virtual profit (envision what that code snippet does on the condition you see) and the MM is down the drain.

    The net effect I can not reason yet, but the outcome of it is that Live performs way way better than backtest, while backtest actually was “fit” to what I see happening. Thus, what I see happening should increase the number of contracts gradually and if that goes too fast for what the portfolio can bear, I slow it down in the code. And that slowing down does not work in Live because of the StrategyProfit acting differently (no “no-profit” because of broker fee).

    And so all ‘n all backtesting is fine. Live is also fine, but very different. Net for the situation, Live is not so fine at all because after each profit 1 contract is added. Totally crazy stuff you see below, and which makes me sit there and wait till I deem it too much and kill the system (which I did twice so far from of Jan 4).
    … And all I thought was an “oh well, *that* is working nicely !”, meanwhile knowing that I must be doing something wrong because I am not really into blowing an account by this strange act;
    In the other topic (about Money Management) I see myself writing about how I approach that, actually not being able to justify this craziness, where you can see things go out of whack. But hey, I derived that from Live and not from Backtest. Backtest works as how I made it (for net result) and there nothing crazily grows. The 2nd attachment shows how it turned out in Live and where I could not deem that wrong. This coincidentally relates to the profit per trade which turned out way better for the latest market situation than I envisioned (better : saw happening in backtest) when designing this, and so I thought that was the reason and let it go for 3 weeks. … Until I wanted to showcase this per backtest in that MM topic and simply could not mimic the Live situation. And then I created the topic you are reading right now …


    Almost the worst is that despite you see something totally illegal in the 2nd attachment, it would work on the longer term just the same. Thus, it would not be so that a loss would make the system go under water. The loss(es) would be huge, but the profits have been more huge to begin with. However, the growth just makes no sense and eventually it would blow up the system itself because of a lack of funds in the account. This is what we talk about in that MM thread … and that it is harmless in itself. But hey …
    Money Managment ideas
    More specifically this post this post and beyond.

    #227218

    On my latest backtest, I am not reproducing last week’s results.

    See above for similar / same Issue

    #227226

    Hi GraHal, Hello @plbourse,

    I hope it is not too confusing, but what I wrote about in this topic can not be related to plbourse’s problem. PRT ne donne pas les mëmes résultats que la semaine dernière

    My last post in this topic (“Backtesting kaput”) , tries to describe that Backtest is not the same as Live for the IB version of PRT. Initially I thought things had changed. But nothing changed with backtesting … the behavior of StrategyProfit is just not the same in both environments Backtest vs Live. Very inconvenient if one does not know, but it is another problem.

     

    2 users thanked author for this post.
    #232111

    Hi Peter, have you solved your problems with IB ?I’ve studied strategies to implement for IB but I’m very confused for many things. first of all I don’t find any code for  a possible trailing stop, I don’t mean function Ptrailing, just anyone. Do you use something in particular for IB?

    thank’s

    #232113

    Hello Alessio,

    Yes, I solved that problem, but by means of a backdoor solution. In this case it cannot be applied “generally”, hence I can’t tell about any solution for the by me unknown code.
    Please notice that I am not even sure that you are bothered by the issue I ran into, which – summarized- is only related to wanting to know whether the last trade made profit or not. Yes, that simple, and that does not work *at all*, mainly because there is no PRT command for it. I complained about *that* a dozen times (also in this forum), but that never helped. This very issue, which I had in a combination in the code that made it untraceable, could be solved because I recalled the base issue from longer ago : The Broker Fee which is charged in backtesting after the last exit, thus in the next bar. And because Broker Fee is always a cost, it is always a loss once observed in that next bar.
    You could say that out of the blue a loss occurs, which only gets detected by you (as the coder) if you ask for the difference in StrategyProfit.

    Need I say more ? it requires hoopla x 10 to solve such an issue, because of all the combinations which may occur. One example of this : if you go out on an Exit in bar #10314, but in-bar – thus in that same bar #10314 you Enter again (which is a possible combination with a Limit order) – and the Exit lead to a profit, then the Broker Fee as a loss would not even be detected (assumed the Broker Fee is smaller than the profit).

    If you understand the elements in order here, you will be able to mimic it with backtesting and graph and such. In normal situations you will always see an extra trade after the formal trade exited, and it is a loss at the height of the Broker Fee.

    Matters as these will never be solved for PRT because the developers are 100% hidden from us, the user, and while I could easily have it solved together with a developer, it now just won’t get through because the person you talk to can not code a single bit. It just will not get through.
    … Just as this post, unless you can mimic it with graph as just told. And then you will need to solve it yourself for your specific situation.

    If not clear yet : this situation causes that no backtest is ever the same as Live, because in Live this extra trade is not implied. I could also say : avoid comparisons with StrategyProfit[1] because it will never work as should. Also, with IG this will never happen because no Broker Fee is charged. And in Live it is fine, because the Broker Fee is incorporated in the cost f the trade, thus in the StrategyProfit reported right after the trade. So, the problem is that you can’t backtest in PRT-IB.

    This is the base of the solution. Further down the line you can now use StrategyProfitChanged (it won’t cover for the in-bar two trades). But interpret this part correctly :

    The first line is unrelated because it solves the throwing out issues and restarting the System (and preserve the gain so far). If not in order it can be initialized at 0.
    The second line is for yourself to determine somewhere, because it is related to the accumulation of the gain over trades. Example from my own code a few 100 lines further down the line :

    And Yes, you can see that I am actually still struggling with it. Or at least 10 days ago I was. Of course this is all with the added dimension of Money Management, but hey …

    Now, if you have the feeling that this can help you, please go ahead. If you (anyone) have the feeling that this indeed bothers you, please create a ticket for the PRT engineers so they can provide a solution which is transparent to us, the users.

    In any event have fun (not).

    #232114

    first of all I don’t find any code for  a possible trailing stop, I don’t mean function Ptrailing, just anyone. Do you use something in particular for IB?

    Alessio, I see that in the other topic you’re asking for a self-coded solution for the IB environment, but I wouldn’t know why such a provided solution won’t work with PRT-IB. Unless of course, comparisons with StrategyProfit are in order. 😉

    Anyway, No, indeed no standard solutions for Trailing are available in PRT-IB.
    And if you need PRT-IB specific solutions, then probably nobody provided that, because people like Roberto work with IG. And Yes, the both (IG and IB) may not be compatible in all cases because IG would mostly be too simple, with Contract Values always at 1 (a constant which just lacks in PRT, as in “Qty x 1000” for Forex with PRT-IB and PointSize and PointValue which in PRT-IB always matter and have to be incorporated (if you use real Futures with IG you already have those incorporated, but the provided trailing examples will not, as far as I know by heart).

    Regards,
    Peter

Viewing 7 posts - 1 through 7 (of 7 total)

Create your free account now and post your request to benefit from the help of the community
Register or Login