Trades being executes in backtest but not in Live autotrading

Forums › ProRealTime English forum › ProOrder support › Trades being executes in backtest but not in Live autotrading

Viewing 15 posts - 1 through 15 (of 21 total)
1 2
  • 03/18/2024 at 6:02 PM #229969
    Batty

    Hi all … as of this morning it appears that some trades that are being triggered in my chart backtests are not being triggered in my live accounts, which is really concerning. So I wonder if anyone else has just started experiencing the same …
    In more detail. I have two completely separate accounts, one business and one personal (both via IG).  Currently I am running the same two auto systems in each account. When I am running an auto system live I always have it running as an ongoing backtest on the relevant chart, so I can keep an eye on whether they are matching up well or not.  Sometimes the opening and closing times of a trade might vary a small amount between bactkest vs live, but instances of a trade opening in either backtest or live and not at all in the other are, in my experience, extremely rare (as they should be).

    Today however, for one of the systems I am running, there have been 6 trades in backtest and just 2 in Live trading.  For the other system there have been 2 trades in backtest and 0 in Live trading. I should also mention both of the systems I am running are close variants of the same underlying systems that I have been running for many months with experiencing this issue.

    Obviously this is very concerning as such a level of discrepancy makes a asolute, total nonsense of any backtests. I have already raised a support ticket and their first response was that they could not see anything wrong, so I have sent them details of all the relevant trades.

    It would be very useful to know however if anyone else has experinced the same issue today?

    03/18/2024 at 6:18 PM #229970
    Batty

    just posting a quick edit to my first post on this ..

    I should also mention both of the systems I am running are close variants of the same underlying systems that I have been running for many months with experiencing this issue.

    Should read

    I should also mention both of the systems I am running are close variants of the same underlying systems that I have been running for many months without experiencing this issue.

    1 user thanked author for this post.
    avatar GraHal
    03/18/2024 at 6:19 PM #229971
    PeterSt
    Batty wrote:

    I should also mention both of the systems I am running are close variants of the same underlying systems that I have been running for many months with experiencing this issue.

    Without, right ?

    Edit : Haha.

    1 user thanked author for this post.
    avatar Batty
    03/18/2024 at 6:23 PM #229972
    Batty

    Yes, thanks Peter that’s correct 🙂 …. without

    03/18/2024 at 8:38 PM #229978
    GraHal

    You 100% sure you have not changed some setting or spread value or unticked a box etc?

    Have the Systems you are comparing been started at exactly the same date and time … with some codes this can make a big difference to entry and exit times, with other codes it makes little or no difference.

    03/18/2024 at 9:53 PM #229979
    Batty

    I have been testing this further and I think I have found what the problem relates to but I cannot make sense of it … Here is a detailed account of this.

    If I run these two systems on a chart set to a small number of units/bars, then the backtest matches the Live trades. If the chart has more units, the backtest shows trades that were triggered earlier in the day, but which were not triggered in the Live trading. The cutoff point where the chart backtest either shows the earlier trades or does not is just under 67,000 units/bars.

    Does this have something to do with DEFPARAM PreLoadBars?

    • The chart is a 5 second chart.
    • The longest (multiple) timeframe used in the code is 4 hours.  This is 14,400 seconds (4*60*60), which is 2,880 bars (14,400 / 5 sec underlying chart timeframe)
    • The code has DEFPARAM PreLoadBars set at 10000

    So DEFPARAM PreLoadBars should comfortably cover the 4 hour timeframe and the number of units the chart is set to should not make a difference.

    The only other piece of code that could possibly relate to this is as below

    My calculation is that this requires 5,040 bars preloaded

    • 60sec / 5sec per bar to get the 60 sec timeframe = 12
    • 20 bars to calc the exponential avg
    • 21 bars to calc the momentum
    • 12 x 20 x 21 = 5,040

    As far as I can make out, If DEFPARAM PreLoadBars = 10000 means 10,000  5 second bars will be preloaded, then the above anomalies should not be happening.

    Or maybe it is the case that DEFPARAM PreLoadBars  in a 5 second chart is actually calculated on a 1 second basis … if that’s the problem then why can’t I find an explaination of this anywhere?

    But even then, the cut-of point on the charts .. i.e. where the earlier trades show or do not show, is just under 67,000 units/bars.  When the chart has 66,500 units/bars the earlier trades do not show, when the chart has 67,000 units, the earlier trades show. DEFPARAM would not be relevant here, so what’s behind these anomolies?

    And even then, it does not make sense of the But it is the only explaination I can come up with.

    Am I missing something here?  Any input on this would be gratefully received.

    03/18/2024 at 10:14 PM #229981
    GraHal

    You are doing excellent with your analysis!

    You say that all has been good for many months running prior to very recently.

    Has your analysis revealed anything that you changed that has caused the recent problems or are you convinced that something has changed on the PRT Platform or the data?

    1 user thanked author for this post.
    avatar Batty
    03/18/2024 at 10:34 PM #229982
    Batty

    Thanks GraHal, I have not changed anything in the systems that would affect the issue outlined. I guess that if I am misreading something in the way PreLoadBars works in a 5 sec chart, and if this is the root of the problem, then it might be that none of the previous versions of these systems have ever tried to place trades so soon after being started. But I really don’t think that is the case. And I cannot see how that would relate to the charts showing the trades or not showing the trades depending on whether they have 66,500 units or 67,000 units …. that number of units should easily cover all the multi-timeframes contained in the codes and all the calculations that run under them.  Either I’m missing something or something isn’t working right.

    I have updated my support ticket with the same information so hopefully this will get to the bottom of it and I will post an update here when I hear back from them.

    1 user thanked author for this post.
    avatar GraHal
    03/18/2024 at 10:42 PM #229983
    JS

    Hi Batty,

    The calculation of an exponential average requires 2 to 3 times the amount of history than, for example, a simple average…

    If you assume the worst situation, use at least:

    12 x (3 x 20) x 21 = 15.120 bars

    1 user thanked author for this post.
    avatar Batty
    03/19/2024 at 5:14 AM #229988
    PeterSt
    Batty wrote:

    it might be that none of the previous versions of these systems have ever tried to place trades so soon after being started.

    If you take that for assumption, it will be your best bet for understanding what’s going on further (instead of thinking that something suddenly is amiss environmentally).

    N.b.: With a 5 second TimeFrame on the chart, it will be just that which is the base for the PreLoadBars. Thus not 1 second or anything else you did not use.

    Otherwise :

    Could you test easily whether the threshold is 65536 ? (or very close to that). If Yes, it definitely is a bug within PRT. And to me it already looks like that …
    Edit : hmm … with between 66000 and 67000 you already answered that question. Still I think my 65536 is involved. 64 bars of difference. If your 66000 is really what you noticed, that 64 is also such a “bit” (from bytes) number.

    1 user thanked author for this post.
    avatar Batty
    03/19/2024 at 10:14 AM #230003
    Batty

    Thanks JS, I appreciate the information on the EMA, it’s very useful. To update, there has been a further trade this morning which shows in the chart backtest (provided the chart is set to enough units/bars), but did not trigger live. Both systems were started at open (spot gold) on Sunday evening so the PreLoadBars issues is not relevant. There were also the two Live trades that were triggered yesterday afternoon. I follow this up further in my reply to PeterSt below.

    03/19/2024 at 10:47 AM #230007
    Batty

    Thanks Peter, I’m now certain it has nothing to do with none of my previous versions of these systems having tried to place trades so soon after being started. I’m aware that 5sec bars should behave like 5sec bars but there appears to be something wrong environmentally so maybe they are not?  Just exploring around it – V12 in particular seems not always to do what it should.

    Following a further trade this morning that was “missed” in Live but shows in the chart back test (see reply to JS) , I looked at the chart timescales again. The “cut-off” between the chart showing the two different sets of results is now different!  With the chart set to 76,000 bars / units, there are 7 trades showing. With the chart set to 76,000 units / bars, just the 2 trades of yesterday afternoon show.

    I ran a further test on the chart bactests, this time with a system I was running Live until last week. Last week the back test trades were matching the Live trades exactly. If I run the back test over 75,000 bars units the number of trades if far fewer than if I run it over 76,000 – see screenshots. The differences in all the system’s charts go back to the start of the chart, they don’t just apply to this week.

    All thoughts welcome, I am updating the support ticket again with the above.

     

    03/19/2024 at 11:05 AM #230011
    PeterSt

    On a side note : This is funny; I see that you’re working with Spot Gold. I was working with that too the other week, but could not understand a hoot of what’s going on with it. So a strange hint in between : could it be so that something in there struggles with pips and pip values which somehow does not match between Backtest and Live ? I never came ’round to doing it live, because I just did not understand the backtesting (it made profit all right, but that does not satisfy me when I don’t understand all of it).
    For me this was with IB. I think you’re on IG, right ?

    I will now look into your last post and screenshots.

     

    03/19/2024 at 11:15 AM #230012
    PeterSt

    In between the lines another hint which could be related :
    In the PreloadBoars phase, do notice that your code is fully executed, except for the trades. Thus all lines of code just execute (10000 times in your case), but Buy/Sell Short lines are skipped.
    I can’t give the example any more, but this can cause different behavior in Backtest vs Live.

    03/19/2024 at 11:25 AM #230013
    PeterSt
    Batty wrote:

    so the PreLoadBars issues is not relevant.

    Another hint :
    What PreloadBars can do or can not do for Backtest, will be consistent in Live. Think about this :
    If you have a chart loaded with 10K bars in order for your 5 second TF which does not deliver all that much days in the first place, then the PreLoadBars will not – or may not do a thing. This will especially be so for the maximum like 200K bars, or 1M bars for historical data. You just won’t be provided with more.
    But now Live …
    Live WILL comply to it. This is because Live starts “today and right now”.

    So this might not be your issue, but it is at least something to consider.

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 21 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
live trades do not match those of the backtest
autotrading – latent gain will not show on charts
backtest vs live – different results recently??
backtest vs live trade
graphical functions not being recognized
proorder autotrading performance report not opening
v12 for prt-ig now live on live
6 jan 24 prt not available – too long to respond
accountid does not have permission for epic + locale combination
alert not triggered

Create your free account now and post your request to benefit from the help of the community
Register or Login