Bug ProBacktest duplicate LIMIT orders
Forums › ProRealTime English forum › ProOrder support › Bug ProBacktest duplicate LIMIT orders
- This topic has 11 replies, 4 voices, and was last updated 8 months ago by nevtek.
-
-
04/10/2024 at 1:23 PM #231347
Hi,
I ran into some weird behaviour while backtesting some code. I was only buying multiple times intraday with LIMIT orders, and closing the position EOD. So no shortselling, but I ended up with a growing short position…
So I dived into it and found the problem. The problem occurs when the code triggers multiple LIMIT orders at the opening bar with the same amount of shares.
Some simplified working code as example1234567891011DEFPARAM CumulateOrders = truedefparam flatafter=145500 //EST//spy 11-01-2024TIMEFRAME(1 hour)if Dopen(0) > Dhigh(1) thenif not longOnMarket thenBuy 10 SHARES AT Dhigh(1) LIMITBuy 10 SHARES AT Dclose(1) LIMIT //11 shares okendifendifSo when the stock opens above yesterdays high then it creates 2 buy LIMIT orders at yesterdays high and close. When the stock immediately drops below yesterdays close at the opening bar, then these 2 LIMIT orders should be both executed. But because it is the opening bar and the code runs after the bar closes, the actual orders will be executed at the next bar open. The result of this is that both orders now have the same execution price and become ‘duplicates’.
Now it goes wrong, only 1 order (10 shares buy) is executed in the proBacktest(probably because it thinks it’s the same order) meanwhile the Orders list(detailed report) knows it were 2. Then when the position is closed EOD, it sells 20 shares. Result is a -10 short position.
The problem does not occur when I use different size per order.
See screenshot for example (SPY 1HOUR 11 january 2024)04/10/2024 at 9:22 PM #231378It’s because you are using LIMIT pending orders, even when they should be STOP orders.
I want to remind you, as from the attached pic, that LIMIT orders are to be used when the entry price is better than the current one, STOP orders are to be used when it’s worse, instead.
12345678910111213141516171819DEFPARAM CumulateOrders = truedefparam flatafter=145500 //EST//spy 11-01-2024TIMEFRAME(1 hour)if Dopen(0) > Dhigh(1) thenif not longOnMarket thenIF close > Dhigh(1) THENBuy 10 SHARES AT Dhigh(1) LIMITELSIF close < Dhigh(1) THENBuy 10 SHARES AT Dhigh(1) STOPENDIF//IF close > Dclose(1) THENBuy 10 SHARES AT Dclose(1) LIMITELSIF close < Dclose(1)THENBuy 10 SHARES AT Dclose(1)STOPENDIFendifendif04/11/2024 at 2:04 PM #231410Hi,
I think you are describing another ‘issue’. The code is simplified, just for showing the bug. It’s not my actual backtest.
Whatever I do in code should never result in a discrepancy between the order list on the left(2×10 buy) and the position size right(1×10 buy). That’s prorealtime code, not mine. Please check my screenshot in the original post.
Also when I use different size per order, there is suddenly no problem….
I will add this as screenshot as well.1234567891011DEFPARAM CumulateOrders = truedefparam flatafter=145500 //EST//spy 11-01-2024TIMEFRAME(1 hour)if Dopen(0) > Dhigh(1) thenif not longOnMarket thenBuy 10 SHARES AT Dhigh(1) LIMITBuy 11 SHARES AT Dclose(1) LIMIT //11 shares ok, 10 NOT OKendifendif04/12/2024 at 3:30 PM #231461It’s because when you use a wrong type of pending order, it is opened AT MARKET.
04/13/2024 at 1:11 AM #231474Below the text I wrote in response to your first post in this topic, Roberto. But I did not post it because the explanation would be too complicated. In other words, I could not finish that text.
Point is : although the situation is challenged because putting an order at the wrong side of the price, what happens as result is no reason to just say “you should not do it”. As you will read below in brief – I do it all the time. But in controlled fashion.My text from yesterday summarized (in attempt), we can reason that the broker and PRT in combination enter an impossible situation, of which I could show longer essays of me and ProRealTime reasoning this out, BUT, this is about Live situations – not Backtest. The Live situations are about orders being filled partially, what PRT won’t anticipate – nothing does. But it is challenged for with two orders in the same bar (candle).
In my situation, which is related to Eurex and an extra layer of “Exchange” I can mimic the situation easily and it causes the system to go short while the strategy as such does not even Sell Short anywhere. It is about left overs of the order of a previous bar which could not complete, but which completes later. Even several bars later.It is NOT the situation of nevtek, but think of the simple example of your Market order not filled (or partially filled), while your code simply says Sell at Market. It only requires a redundant registration of the system having done such, and the misery is there. There is no solution to this, other than an If Not OnMarket, in combination with resets I applied all over, because a few lines back I simply *know* I am out of position (while it appears I am not at all).
Again, this is not nevtek’s situation, but it is illustrative for what is challenged with the means of ordering you, nevtek, apply (which is odd to say the least, but which would be allowed anyway AND WHICH WOULD NOT BVE POSSIBLE WITH MANUAL TRADING – please think of this … your position would be gone because the (first) limit order at the wrong side of the price, would eliminate the position ride away).The reason why I could not finish my initial response is because this cannot happen in Backtesting. Or at least not that I experienced. In Live, Yes (Demo, probably also not).
The problem – thus in the same realm – is more challenged for with IB than IG, because IB is more capable of not filling orders than IG. With IB it can therefore be so that pending order should be eliminated, but it is not because another pending order already was given to IB while the former is not cancelled (this is about the principle of pending orders in the first place). Thus mind you, the cancellation is an explicit activity, and nothing knows it should be done twice (it is an unexpected situation). It is is also about the informal OCO principle (one Cancels the Other) which is not formally present within AutoTrading, bit which works out 99 of 100 times as should anyway because you don’t pose a strange act to PRT and/or the broker. Anyway just envision :You Exit a Long position at 08:10 and at 08:25 you suddenly have a Short position and you won’t know the heck where it came from. But it comes from the still pending Exit (which is a virtual Short against a Long and which is perfectly allowed as Exit means), while something else caused the position to Exit long gone, and the other Exit still pending because it was never cancelled. It is now only a matter of waiting until the price is touched (could be a year later, you never seeing the order, always pending).
All you can do against it is a. knowing about it and b. not challenge the situation. And b. is what is happening here, although I can’t see how precisely – and especially not with backtesting. But envisioning the Manual situation is sufficient – you won’t be able to have both orders in because the one at the wrong side of the price will exit the position. What’s left is the other Limit order and now thing how that one will or at least can unwind …
Peter
Nothing prevents from putting a limit at the wrong side of the price, so Roberto, if you think that causes the culprit within PRT, please elaborate ?
I am using Stops and Limits at the wrong side of the price all of the time, though it would be coincidental (I can’t always know where the price is going). Nothing goes wrong with that.I have been staring at this too, and all I can come up with is that something goes wrong because of the usage of two the same type of pending orders at the same time. Point is, manually this is allowed, but, nobody will do that manually at the wrong side of the price (or else you lose your position right away because it becomes a Market order).
With the context of Roberto being correct but without him further explaining, I come to this reasoning (and could show examples from Live Autotrading if necessary) :
When two Limit (or two Stop) orders are given to the broker, there is no way for PRT to know that the one has been filled while the other is still not. Theoretically this can obviously be known, but it is an unexpected situation. Thus, when the one order is filled, the other is still pending and that can be filled too, later. Especially in the realm of one hour, this can easily happen. But when I buy 10 and 10 go out as well […]
04/13/2024 at 1:31 AM #231475Also when I use different size per order, there is suddenly no problem….
This can be explained by PRT doing her stinking best to match orders which actually can’t be matched (see my previous post).
In communication between a “API user” like PRT is to IBKR, orders have Identifications. This is also about the OCO principle. But if that leads to unexpected situations (read : is just has not be coded by at least one of the participating parties), communication stops, and in this case the order would remain at the broker without PRT having the knowledge of that. This is the subject of my previous post;I could show you (if I could ever find it) situations of nice systems which work like a loose running train, just because they keep on “servo-ing” positions which are just noticed to be there, and acted upon. Mind you, this is in Live again, but can be explained by backtesting in aftermath just the same. Thus, the fact that your position is Short, need not to be noticed by you a all, just because you have the knowledge it can be short. Meanwhile, all s*cks because the initial short was not intended, but because it is, it is Exited Short, or worse, sold Long in the double quantity because you intend to go Long, and did not know about the Short. This is all a matter of doing both in the same call of the code (your per 1 hour (bar)) and it happens. This is far away in the same realm of you having one If, while Roberto made that an If Else.
Because I again can’t provide the real example, just think of this situation, please taking for granted I am correct with it :
If you invert the position in the wrong fashion in your code, you will need the money twice, because nothing can see it goes from -1 to + 1 (or vise versa) with net the same result. Thus, you will need 2 to do it, which is not justified. Still net your order will be for 2 and not for 1 and another 1.If this is grasped, it is better understood why these situations should not be challenged for.
We could indeed call the net result a bug, but what can’t be dealt with, can’t be dealt with; all is always about communication between a provider like PRT and the broker. … And how that must be simulated in Backtesting on top of it. This is how I say that 90% of more rare situations (like yours) can’t be tested in Backtesting or Demo / Papertrading; it can only be done in Live. Testing a strategy, yes, that is fine for backtesting. But testing the technical real life happenings … impossible. A more sad pint is, that what you do is “strategy” as such just the same. Again, I do these things too, for a good reason (ahem). Now find out how Live deals with it, and set it up slightly different so Live will bear it. Mat take a year throughput (per broker). But that’s AutoTrading life …04/13/2024 at 2:01 AM #231476This is from a situation I describe to PRT support, on Jan 15 2024. It implies the same situation as you deal with in the base, nevtek. The story is of course longer, but in brief you see two red triangles. Now, the one partial fill is from the one order, and the other partial fill is from the other order. Both orders were there in this one bar.
Without further elaboration, this lead to a Short position while the system does not even contains the commands for it. Although I proved things in retrospect with backtesting (see 2nd picture and the pink triangle overlaying the top red one) I never checked backtesting results for what you experienced. Fact is, this happened in Live and possibly backtesting acted the same.Nothing will be able to cope with that (the system was thrown out by PRT) and after learning this could happen, I took precautions that it never would happen again. The system got less smart (bringing theoretical less $), but an other reason was eliminated it can be thrown out (and I promise you and everyone a 100 more reasons).
04/14/2024 at 10:48 PM #231509Hi Peter, I don’t know if the problem is the same but I’m running on paper traing a system on Nasdaq ( IB ) and I noticed that after opening long and closing position in profit, the stop loss limit remain opened. Now I closed it but it seemed strange to me. Could that be a problem for the strategy?
Alessio
04/15/2024 at 5:25 AM #231510Hi Alessio,
Yes, that sure would be a problem in practice (Live), would that happen; if the order remains, it just will be active and when the price touches it, you will have an unwanted position (and your code won’t be able to deal with it, although OnMarket will be True). It will always be an unexpected situation for your program logic.
It is very hard to test your Systems on Papertrading for IB because it is too wacky. I can’t tell whether your occasion is wrong for the Papertrading version, but it would be hard to be wrong because such situations – AFAIK – always spring from thrown out systems (which only happens in Live). What I try to say is : my example in my post would go wrong in Backtesting just the same (two orders eat from each other’s filling, so to speak) BUT in backtesting I would not notice it unless there would be an odd situation all together which requires analysis (which seems undoable to me in the first place). In other words : Papertrading would show it too and you would observe it in real time.
If the “stop loss limit” remains open each time, it will be a Papertrading problem. On the other hand, if your order is of the wrong type, you may be the cause yourself; I don’t see how yet (similar to the subject of this forum topic), but your description is ambiguous for now. A real Stop Loss (like Set Stop $Loss) will be an OCO situation. If it is a Pending Limit Order, it can’t exist because at the wrong side of the price (when you went out with profit – just think about this). When it is a Pending Stop Order it should go out, at least at the next bar. Think about this too, because PRT does not deal with this properly in IB, again because it is not OCO (in IG it does !). But but but … if your bars are e.g. one hour, you will think it did not go out if you don’t wait out this hour. But this is exactly the problem : in one hour of time, that other order may still be in when the price reaches it. And that is exactly what I described (without words). Then things will go wrong and you will end up with an inverted position. And in my example, partly on top of it which probably is a prerequisite to let it go wrong (and can’t be solved as far as I can reason – we (you, me, everybody) should not challenge for it).
Regards,
Peter04/15/2024 at 1:27 PM #231527It’s because when you use a wrong type of pending order, it is opened AT MARKET.
I understand that it is opened at market because both limit buy orders ended up out the money and therefor executed immediately at open. That’s fine! And is not the problem I’m describing here. (And otm limit orders are not wrong, I trade them manual all the time as alternative for MARKET orders. Is the prorealtime BUY ASK button wrong as well? )
Let’s keep it simple, why does the Order list (left part screenshot 1st post) know it are 2 orders, and the position chart (right part screenshot 1st post) not? Has this to do with the IB API? Is that even simulated in the backtest?
So is this a feature or is it a bug?
04/15/2024 at 3:28 PM #231539Your first code is working fine:
123456789101112131415DEFPARAM CumulateOrders = truedefparam flatafter=145500 //EST//spy 11-01-2024TIMEFRAME(1 hour)if Dopen(0) > Dhigh(1) thenif not longOnMarket thenBuy 10 SHARES AT Dhigh(1) LIMITBuy 10 SHARES AT Dclose(1) LIMIT //11 shares okendifendifset target pprofit 20set stop ploss 20graphonprice Dhigh(1) coloured("Green")graphonprice Dclose(1) coloured("Fuchsia")Pic X1 shows a correct order placed because the price was ABOVE the entry price, so the LIMIT orders were corret. One of the two irders was triggered, the other one wasn’t.
Pic X2 shows two incorrect orders both triggered because the price was BELOW the entry price, so STOP orders should have been placed; as they were placed of an incorrect type, both pending orders were triggered AT MARKET (with DEFPARAM CUMULATEROTDERS=FALSE directive only one of them would have been triggerd). There’s no support for OCO pending orders.
04/15/2024 at 6:35 PM #231554Your second example works fine because the condition is met during the day and not at the opening bar. See my 1st post, I explain it there.
You have to look for opening bar conditions. You can easily spot them when you run my code on any stock intraday, and then zoom out. See attachment -
AuthorPosts
Find exclusive trading pro-tools on