ProBacktest Results Incomplete

• 02/04/2023 at 6:13 AM #209078
  Pieter_T

  I have experienced over the last few months on a few occasions some issues with ProBacktest in that it does not complete the backtest. The concern for me is that there is no indication on any screen I am aware of that actually show you or the backtest was completed fully. The backtes report at the bottom of the screen show the green tick with the words Optimization complete but the actual test did not complete in full. I had times were I watch the screen and after 5% progress the screen close and show the results in the Optimize report with the green tick on and Optimization complete. If I then redo the test it will complete and the results are OK.

  This is very concerning as I am now not anymore sure or the Optimze results are always the most optimized as it could have been a result from a backtest that did not complete.

  Is it possible for PRT to show at the end of the backtest in the Optimization report how many combinations was requested (at start of test) and how many was tested (on completion of test). This would help to confirm all requested combinations were actually tested.

  

  Pieter_T

  Participant

  Topics: 5

  Replies: 27

  Been thanked: 5 times
  02/04/2023 at 7:06 AM #209084
  PeterSt

  Hi Pieter,

  This is so since (July 2022) the Optimization Result form remains on screen and a next Result populates in the same form. It is a matter of observing that small “message-like” window with a progress bar in it BUT which easily falls behind another window. So teach yourself to have that in sight always. And … never click the button on it on that small window, because then the backtest will stop and the Result form will show that it’s done anyway.

  Regards, Peter

  

  PeterSt

  Participant

  Topics: 73

  Replies: 2503

  Been thanked: 805 times
  02/04/2023 at 7:37 AM #209088
  Pieter_T

  Hi Peter,

  I am aware of the “message like” window you mention. It show the progress in percentage of the back test. It is in this window that if at times the backtest only reach a certain percentage like 5% or 35% and then the backtest stop itself. The results in the report is then only the best results for 5% or 35% of testing done. There is no indication on the screen as to why the backtest stopped and how many of the combinations requested was actually tested.

  This morning again I ran a backtest and after 5% it stopped itself. I repeated the test and then it went all the way to 100%.

  My understanding in talking with PRT before  is that the backtest are actually running on the PRT server and we only get the results populated on our screen in the report as the server complete this. You will not know about this problem unless you keep watching the “message like” window and see that 100% is actually reached. After 5% or 100% completion everything on the screen look just the same.

  

  Pieter_T

  Participant

  Topics: 5

  Replies: 27

  Been thanked: 5 times
  02/04/2023 at 8:33 AM #209089
  PeterSt

  Thank you for your accurate description. But hmm …I never experienced that. Looks like something chokes in there (sorry for the obvious).

  I wouldn’t know how to approach this either. But say your code is not a secret – or you change a few things significantly while the problem persists. You could upload your .itf to here and people (e.g. me) may test x times to determine it happens here too. You must determine x (as in : after x times you should have run into the issue for sure).
  When you changed the code for the purpose, you should measure on x again yourself (it might suddenly be OK).

  1 user thanked author for this post.

  Pieter_T

  

  PeterSt

  Participant

  Topics: 73

  Replies: 2503

  Been thanked: 805 times
  02/04/2023 at 9:56 AM #209092
  GraHal

  For the record, my optimisations do not stop part way through.

  pietertal after what period are you 5% through your optimisation?

  1 user thanked author for this post.

  Pieter_T

  

  GraHal

  Participant

  Topics: 211

  Replies: 8778

  Been thanked: 2183 times
  02/05/2023 at 6:46 AM #209122
  Pieter_T

  I do not think it is related to the code as I can run the same backtest at another time and it will work just fine – complete 100%. The times I experience this issue the most is over weekends and a few hours before and after the markets open. This might have to do with more users being online before and after market open (UK/France markets) and over weekends maybe when maintenance work is done on servers. This is just a guess. I however do no loose any connection with PRT to be able to say it could be this. If backtest is done during the week it gets slower as we approach market open time and for a few hours after market open. In this time period it occurs more.

  I will write a simple code and try and execute a backtest if I get another backtest failure and see or I can replicate this. It occurs very random and not always on the same code.

  The reason you do not see this in Europe is perhaps because you are closer to the servers with less lag and intermittent dropouts.

  

  Pieter_T

  Participant

  Topics: 5

  Replies: 27

  Been thanked: 5 times
  02/05/2023 at 6:47 AM #209123
  Pieter_T

  The backtest stoppage over weekend occur at around 5% which in time was after about 3 minutes.

  1 user thanked author for this post.

  GraHal

  

  Pieter_T

  Participant

  Topics: 5

  Replies: 27

  Been thanked: 5 times
  02/20/2023 at 3:38 PM #209984
  Pieter_T

  This now resolved. It seems it was again some maintenance works on the IG server or something. Tried same code after a few days and all good.

  1 user thanked author for this post.

  GraHal

  

  Pieter_T

  Participant

  Topics: 5

  Replies: 27

  Been thanked: 5 times
• Author
  Posts