same code but opposite behaviour on real and demo accounts

Forums › ProRealTime English forum › General trading discussions › same code but opposite behaviour on real and demo accounts

Viewing 12 posts - 1 through 12 (of 12 total)
  • 12/02/2024 at 3:27 PM #240981
    oliTR

    Hello,

    after 3 years of intensive practice of PRT trading systems on IG, I wonder about the relevance of all these simulations:

    For example, this morning at 9 am on the DAX, the same system took, at the same second, a long position on the demo account and a short position on the real account. Result +1000€ in the first case, -1000€ in the second.

    Am I the only one who sees this kind of thing?

    Thank you

    Oli

    12/02/2024 at 3:46 PM #240983
    GraHal
    1.  Were both Systems not onmarket at 08:59am this morning?
    2.  If No to 1. … did both Systems start at exactly the same time on Demo and Real?
      3.   Had either System been stopped and restarted recently?

    Just a few early thoughts.

     

    12/02/2024 at 4:15 PM #240984
    oliTR

    Thanks

    well, the positions were reported to enter at the same time, 09:00:00, and the system was set on the real the day before and duplicated just after on the demo, both for the first time.

    The time unit is 5 seconds, quite short,but I don’t know if that is related to this UT.

    12/02/2024 at 4:44 PM #240985
    GraHal

    What spread did you enter / type in on the Demo settings?

    Btw what does UT stand for?

    12/02/2024 at 5:52 PM #240989
    oliTR

    No spread was entered as it was not backtest but system operating on the real account and the demo account. Si the spread was the one applied by IG, unknown.

    UT= unit of time

    1 user thanked author for this post.
    avatar GraHal
    12/02/2024 at 6:52 PM #240992
    GraHal

    Does seem weird and I can’t, at this stage, think of anything further that might have a bearing.
    Let us know if it happens again?

    Anybody else feel free to add your thoughts.

    12/03/2024 at 6:31 AM #241009
    PeterSt

    Hi Oli,

    Personally I don’t see anything strange so far. Indirectly this relates to you not showing any, say, proof of oddness. In other words, would you have shown where the Demo system went off vs the Live system, then I wouldn’t have a case.

    It is a bit strange to tell this in aftermath, but what you still can do is this :
    Provide your code with Graph and GraphOnPrice all over the place for key variables, and run it in backtest from the 100% exact time as you did with your Live system. This means, to the second (which you already can’t really because it is per the minute, but alas). You will soon see that your backtest does different things than your Live system (and Demo for that matter), but the point is : it will make you wonder why. And you will find out. Please notice that you can run this same backtest in Demo and in your live account.
    You will run into a dozen of questions (which you may put over here in this thread) and you will learn what’s all going on.

    You said that you have been testing thoroughly for the past 3 years and of course I believe that. But what I tried to say in between the lines in my text above is : if you had done that per the means I now suggest, you would have come up with the proof of oddity long gone. And thus : you did not test your systems the past 3 years like this ?
    None of my systems is ever doing something strange suddenly. I could add : PRT is decent enough to not let this happen. However, *if* this happens, I immediately run my available backtest against it in retrospect, and I will immediately find out the culprit. Always. And with that said : none of my systems is ever doing something strange suddenly. Ah, I said this already. But the point is : it is always me who did something wrong or strange or whatever it is, causing the Live system do something unexpected. Anyway, it can always be explained.

    Now what if you already did this all ? …
    Look for oddness in the data itself. Obviously you will have looked whether the data in both Live and Demo was different (of course for Live you’d need to look back in “history” but that is no problem). Assumed it is not different : look for spikes. A spike in this case could be a huge one – one which is not realistic, hence did not happen for real on the Exchange, but still is there – on both Demo and Live. Now think of different treatment of oddities in Demo vs Live …
    Data providers and especially beyond (like the broker – like PRT) can filter out unrealistic spikes. But did that happen in Demo ? No, that most probably did not happen in Demo. This means that your code in Demo will have seen the spike and act upon it, while the code in Live never saw it. Still you would be able to look at it today (from Monday morning I suppose). Set your timeframe to 1 second or even ticks perhaps.

    I am not making this up, and when I dig very hard I can find a ticket for this from 6 years ago or so, where my Demo system indeed was molested forever, because it lost 100K in one trade. The price went up for a full USDollar (on the 1,20 or so for the EUR/USD Fx), it won a million or whatever it was, and next it went short after that (Reversed IIRC) and lost 1.1 million. Thus net a 100K loss.
    But the spike was wrongly provided in the data and this can happen. … And as you can imagine, all systems MUST be able to filter such oddity and they thus do. But the Demo in PRT does not. Btw this was with PRT-IB, and while this is a decent environment, I can imagine that with IG and their CFD environment this can be worse.

    Regards and please let us know your findings !
    Peter

    1 user thanked author for this post.
    avatar inverse
    12/03/2024 at 7:04 PM #241050
    oliTR

    Hi Peter,

    many thanks for your thoughts.

    In your last part you mention spikes filtering differences between Live and Demo. I didn’t know that could be but I am happy to read this because it really looks in line with I’ve seen.

    What to do to improve the robustness of the code, to make it less sensitive to spikes ?  Could it be using multiple similar codes, but very slightly different for the entry in position, with smaller positions, in order to dilute the risk of being trapped ?

    Regards

    Oli

     

     

    1 user thanked author for this post.
    avatar PeterSt
    12/04/2024 at 1:23 AM #241059
    PeterSt

    Hi again Oli.

    Please notice that in Live (not Demo) these spikes will never be an issue (and if it is, you can claim money back when you lose because of it). Thus, this is only about the realism of Backtesting.

    I must say, my personal largest effort goes into realism of Backtesting indeed. It is almost more important than the good strategy itself (nothing new to you 🙂 ). But this one with the spikes ? I don’t think anything can be found to make that more real during backtest. One thing to mention perhaps : other platforms may show this as “outliers” up to even don’t count them in the backtest result. PRT does not contain this “statistic” but they could make it for our better insight. Thus, my own example with a profit of 1 million and a loss of 1.1 million upon normal profits and losses of say 1000, are outliers. They should not be counted because they are “not normal”. Still, in normal life and Live, outliers exist all over, like with (Fed) news. That too urges for huge swings your backtest in the past and Live for today will capture. Still, your backtesting – optimising actually, will always be in favor of that outlier. Thus, it will always “form” the trades such that when normally profits of 100 occur, that outlier of 1000+ will form your trades (in Live no “forming” can happen of course).
    This is more difficult to explain, so I hope you get the gist of this ?

    Both feet on the ground, anomalies because of these spikes are rare. I have seen them on the charts a couple of times, but really only a few times (which does not tell all because I can not see the charts of all instruments). However :
    What I do know for sure is how historical data fails very often on minor unrealistic spikes. It is, for example, no exception at all to find data with the last trade price beyond bid and ask, which can not be in real life. This happens many times per day with regular data providers, and your backtesting will rely on it. Thus, bid was 6000.25, ask was 6000.75, but price at the same time is 5995.75. This simply can not exist. This is the problem of historical data providers. PRT is actually also one (of its own) and I don’t know at all whether PRT does a good job here; we just can not tell without the bid/ask quotes data (which PRT does not provide).

    So is the latter a real problem with backtesting ? … not that I have seen, as long as you don’t want to make profit over 0.25 or 0.50 points.

    Stick with the advised proof of what happened with the Graphing tools (ran afterwards through backtest) and I take it you will be sufficiently good.
    Regards,
    Peter

    12/04/2024 at 6:27 AM #241064
    PeterSt

    As if the devil plays with it (we say), here is one. Look at the strange shape/level of the price line, and observe the large green spike downwards at the start of (Dec) “04”.
    The 2nd picture shows it zoomed-in.
    The 3rd picture now shows it with 1-second bars and here you can clearly see how things are wrong in the data. You can also see how it recovers slowly. The 4th picture shows that this recovery occurred after 12 seconds (the bar prior to where the hair cursor is). And, that for a time period of 10 seconds no data was present at all.

    This ~23 points for this Russel2000 is impossible after this one hour of no trading (23:00 – 00:00 Amst).
    What Live Autotrading would have done with this – no idea, but if all is right this was filtered out. But was it ? I can’t tell until (e.g.) I call, and PRT will repair the data and tell me to restart the platform (and then the spike has vanished). But, they may also tell me that it can’t be repaired because this is what the exchange presented for real (through the data provider of PRT – which is themselves as far as I am concerned). In that case my live trading would have been in trouble too. And Demo ? Demo will always have reacted to this.
    Please keep in mind : this is PRT-IB (not IG).

    N.b.: The price point of 2408 will be real(alistic) – at the top of the pin of the first green bar.

    Regards,
    Peter

    12/04/2024 at 6:37 AM #241069
    PeterSt

    Here you have it at the tick level;
    The more experienced eye can see that the oblique red downwards like just shows like that because of the graphics rendering. It is all the same ne “tick”, reaching the bottom at 00:01, staying there until the 11th second.

    It is hard to tell for me what happens after that (say until in the 12th second) because I am not experienced with looking at the Russel2000 at the re-opening, while this looks like normal trades to me. The first larger jump-up after the “stall” is 5 points. This seems too much to be realistic.

    12/04/2024 at 6:58 AM #241074
    PeterSt

    To complete the fun, here the S&P500 (Micro) on PRT-IB at this same moment in time. This looks like a normal re-opening to me – and anyway my program code responds to this with a Buy.
    The second picture in more detail on a 1 second chart. The third more zoomed in. This looks all completely normal.

    And a bonus … see the 4th picture. This is the Russel2000 again, but now on PRT-IG. The very same thing. This tells that at can’t be PRT’s data being wrong/off, because the data on PRT-IG is from IG herself. It’s the derivative in CFD form.
    So if this all has not happened for real (nobody sold a million contracts of Russel2000 or whatever it takes to drop 23 points in one go/tick/sale) then the data is the only truth (thus whether for real or not) and you could have traded on it.

    Compare to what happens each day at 14:00 Amst on PRT-IB. Heavy movement of about everything (mainly stocks which operate well during ETH, e.g. Tesla, AMD and the like). This goes up-down-up-down for several seconds (like 10) and you can NOT trade on that. So this is false data of some sort, and it is there each day.

     

    Now you know all.
    –there’s no more lyrics–
    🙂

  • Author
    Posts
Viewing 12 posts - 1 through 12 (of 12 total)

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
2 accounts limitations on ib
buy on certin dates, not on technical singal
buy on retest on a certain candle
how to keep current window open on the same monitor
real time exits on monthly bars
same script but different results in demo and live ig cfd account
two different strategies on same market and timeframe
2 bars late on entry and exit
andrew cardwell strategy on rsi
arrays & same candle programming

Create your free account now and post your request to benefit from the help of the community
Register or Login