PRT and CPU Usage
- This topic has 60 replies, 10 voices, and was last updated 1 year ago by
.
-
no backtesting, real trading.
If I do backtest where will my memory go?
30 minutes I opened PRT1 user thanked author for this post.
I tried with the internal Java (version of two years old and also a today updates version) and I see no difference.
1 user thanked author for this post.
Now with high(er) volatility :
1st is the 40 charts IB – Mixed TFs mostly 5 minutes, 11x 2 seconds, 5x 1 second, 1x 10 Ticks.
2nd is a 4 charts IB – all 5 minutes;
3rd is a now 2 charts and IG (no forward Backtesting or anything active) – both 1 minute. This is “self”-Java (no difference with PRT-Java).Only PRT will know what they are doing to transfer the IG data in such a special way onto the charts. But the difference is crazy.
The 2nd screenshot is FYI.
1 user thanked author for this post.
So today is Saturday and this gives insight in how the system behaves when all is at rest.
First off, yesterday I have been backtesting a large part of the day, and I couldn’t run more than 4 or 5 200K backtests or the system choked. Just hung. Kill with Taskmanager and such. Always the memory used is then over 11GB, an amount of which we know PRT/Java can’t deal with (the limit is into the 8GB somewhere).
I noticed that the memory used could jump from 6GB to into that 11GB in one go. I also noticed that the memory used could double in one go. One go : once instance of (visible) time.
I noticed that a new order (Live) would imply heavy usage, hence all would stall for maybe 20 seconds.
I regularly noticed the dreaded “memory” message while not backtesting at all (also see previous bullet point).
I noticed a complete different behaviour compared to before (and I say : since the 23 Dec update, received somewhere the last days of December).
I noticed a constant high CPU usage, coming down to 100% for one core. Very often more than 100% for one core (read : probably more processes now are on the loose).
FWIW, I noticed that reloading the platform (per means in the File menu), would not incur for the more and more memory usage. It stayed at a high level, but I could normally backtest again. Notice that the backtesting itself does not consume the memory. It is just the chart(s) sitting there – moving once per minute (in my case).
And today I notice that all is fine and that the CPU consumption is virtually zero. Memory consumption is how it always was (not good and increasing, but we are used to that as PRT will never find the cause of that anyway (if they search for it to begin with))
Logical explanation : the Indicators. Read : all stands still – there are no Indicators to update at the tick level.And so something must have changed in the area of the indicators.
After more confirmation on Monday (shut off the Indicators and see whether that helps – dig more into the cause) I will report this to PRT.1 user thanked author for this post.
I have an addition to this which may be unexpected.
Memory consumption is how it always was (not good and increasing, but we are used to that as PRT will never find the cause of that anyway (if they search for it to begin with))
It looks like I need to withdraw that. Yesterday I saw this happening “like always” but soon after I realized that maybe PRT needs to build up some base data in the beginning *or* that this is related to two Java (PRT) instances running at the same time. I told something about that as an issue earlier on (years ago). Later, the consumption stabilised at 3.2GB and only by the end of the day it was at 3.6GB. The 200K backtest I ran were uncountable (think 200-300). This experienced like never before. Never slower. Never needed to restart PRT for any reason.
Today, at 40 or so backtests, the consumption is still at 2GB and it doesn’t seem to move. Today this PRT-IG version is the only one loaded.
On a side note I saw the garbage collection running (this an I think sole is a Java thing) and it even did something good. So small portions of memory are given back regularly. And because this doesn’t happen at enormously high consumption in the first place, it feels good (say knowing a bit about this stuff).
We should remember this for when things go odd again, knowing that PRT finally nailed this ?
This leaves the behaviour during markets being open, which is now to be put in another perspective.
All is unacceptably slow, as soon as it is about screen activity, also that which you don’t necessarily see (in-bar Indicators). Backtesting does not slow down from this, but since the memory is very fast eaten, backtesting is virtually impossible (in my situation at least).
I am stipulating that the whole mechanism of dealing with the Indicators is “off” somewhere, which can be seen the best by scrolling (very much stuttering) if only markets are open. Self-scroll because a new bar opens tears down the platform so much that sometimes for many seconds nothing moves. This is also the time the memory will be eaten fast. Complete memory copies are made to (re-)position the chart ? something like that.If PRT acts fast (hence if I provide them with this behaviour fast) they will know what they did in the advancing to the Dec 23 (?) update.
1 user thanked author for this post.
So I sorted this out and it is not the Indicators. It is the sheer movement of the price (and the updating of the chart because of *that*).
I recall that ever back I “claimed” that this was caused by the extra layer which applies to PRT-IG, which although logical, does not fit my today’s thinking. So Yes, they will have that extra layer (which very maybe needs to be converted by PRT to make all suitable for PRT), but today I don’t see how that would be client(-platform) related. Server side Yes, but we would not notice anything of that (for own CPU usage).
I recall from the past the same as I notice today : memory consumption is all normal, the platform being up since 12 hours or so. But, would I start one backtest the lot will start to collapse (not tried today). So this in itself is not new, but what will happen since the Dec 23 update is that only a couple of backtests are allowed per session when markets are open. This, while “infinite” backtests could happen with markets closed (seen this weekend).
So it is the combination which “fights”, as it seems.1 user thanked author for this post.
Hey Guys,
Seeing similar CPU hogging here as well.
Only noticed it when I went to an older template for some water divining activities lol.
Deleted all the charts and saved it. Reloaded the now naked template and it was still hogging CPU!
<hr />
Very odd.
IG-AUS here.
<hr />
Many (all) older templates exhibit similar symptoms with CPU R3600 6c/12t / 16GB ram cycling thru to 100% … gave me the wills ngl.
If you save and exit on the template you wish to use, then restart PRT and open on the last template used (as it does) CPU activity on the once dogged template returns to an all G state.
Cycling (reloading) back to what your day-to-day template was / is should be all G as well w/o necessitating a PRT restart.
It could be tied into where it’s caching new / old loaded templates and it’s ability to resource the environment well enough with an old template (housekeeping).
<hr />
Unfortunately, starting on the ‘most used and latest’ template and re-visiting the older template triggers the problems again.
All the old templates, while maintaining calcs and water drilling points (yes I’m on the turps), have lost all there colours and look really odd. We’re not talking a few templates here, maybe 20 odd.
It’s had a right old rogering.
If we recreate the old templates (in the new env) all is well again and we can roll between templates no problems, no matter the last template used. The old template will however always exhibit CPU hogging.
ALL custom indicators can then be reapplied and the world is round again.
<hr />
Below is a dump of what we see when going from ‘new template’ and back to ‘old’. All are identical but the older hadn’t been used for a few months. Perhaps this is a symptom of the upgrades you chaps have mentioned and perhaps also Gra is right in that it’s been an issue for a little while now too.
Not sure if it’s applicable to the latest desktop rev. but did the damage for us here, hope it helps others. Even if your templates look all G recreate the smallest one and see if it resolves your issues.
We’ve also seen odd bugs where custom indicators have needed to be removed and reapplied – so reinventing the wheel might be a thing when we move rev. levels etc from time to time or MS ream us. Keep a change log of breakages and fixes.
If anyone responds to this post because it’s incoherent, i apologise now, as i mightn’t be able to reply today coz i might just be driving the bus soon enough ….. 😐
This process eliminated the problem entirely for me.
Fortunately / unfortunately running IG supplied v11.1 – 1.8.0_22 with last update 02-Jan-2023 that leverages Java and on the latest here on w10 22H2
Cheers
1 user thanked author for this post.
Find exclusive trading pro-tools on 
