Scoring your report data

Forums › ProRealTime English forum › ProOrder support › Scoring your report data

Tagged: , ,

Viewing 15 posts - 1 through 15 (of 26 total)
1 2
  • 11/27/2021 at 11:11 AM #182380
    nonetheless

    Looking at the info we’re given in the detailed report, I’m thinking of a way to combine the relation between those values to arrive at an overall score for any particular backtest.

    It has to result in a number where ‘higher is better’, but maybe some factors should be given an additional weighting?

    This is my starting point, based on the values I tend to pay attention to:

    av.Gain / Av.Loss * %Win * MaxRunup / MaxDD * GainLossRatio / Duration (months)

    This would depend on some standardization of position size between instruments based on exposure value (for the DAX, NAS, DJ, SP I use: 0.5, 0.5, 0.2, 1.66 respectively)

    What do you think? other factors to consider?

    Is such a thing useful? does it mean anything? or too much fussing about ?

    If there’s anyone out there with a background in statistics, maybe you know of a better way to construct such a formula ???

    1 user thanked author for this post.
    avatar GraHal
    11/27/2021 at 11:46 AM #182383
    robertogozzi

    That’s avery interesting point which might be worth analyzing in depth.

    It’s food for the brain.

    Thank you for sharing 🙂

     

    1 user thanked author for this post.
    avatar nonetheless
    11/27/2021 at 12:26 PM #182390
    GraHal

    Sounds an extremely useful benchmark measure against which to compare strategies.  It is something I’ve attempted using excel pivot tables, but I have to be in the mood for all the faff doing it ‘off PRT’.

    Wouldn’t it be fantastic if we had a ‘free text column’  in the Detailed Report where we could enter such formula as you propose above and PRT computed automatically?

    The same measure could be applied to bots Forward Tested on Demo / Live Accounts as I find myself all the time saying … “now which of these 25 bots is the best performer today / this week etc”.  I know this can be done drilling down in the Detailed Report and taking Notes off several widgets – then compare all 25 bots etc –  but it all feels like so much faff doing it this way?

    I put in a Suggestion to PRT re adding a ‘Period widget‘ to the Auto Trading Window, but your idea does so much more and is a new exciting step forward!

    I reckon we should progress your idea and not worry too much (at this stage) if the formula is the best it can be?  We can explore how we would achieve display of the results and if PRT might include in the V12 release … that would be an exciting Christmas present for us all!!?? 🙂

    11/27/2021 at 12:33 PM #182391
    PeterSt

    Thoughts (apart from it being a great idea !) :

    I think what you’d want to know is the performance per time unit. At least that is how I try to maximise the performance of the AutoTrades. (OT : For example, an extremely deep SL may work out for net result, but only for that trade. Usually not for the overall net result because of the ages it takes to taking profit.)
    And thus the exposure amount should be eliminated from the formula**. This in itself means that the invested amount should be linear to the result. At least I think that would be the prerequisite, because obviously (??) we are going to compare our results ? (being anonymous in the money we spend) – If that is too difficult (it is !) then all should be normalised to a standard exposure amount. Thus, we could agree that 1000 is the exposure amount, and if mine coincidentally is 2000 then my “result score” should be divided by 2. To this regard, notice the complication with a loss, because a loss will then also be divided by 2 (and my experience tells me that this is not fair – not at all sure though).

    The Duration could be problematic with dozens of trades per day. But then – again – it should be about the performance per time unit.

    What could be wrong in my thinking is that I clearly speak “over trades” while you possibly observe one trade only (though not sure). There would also be a problem with running trades (open position) because in my view they just count (sadly PRT statistics don’t understand that), also knowing that for my normal trading I seem to have an iron 30% of gain in the open positions. Thus for example, might PRTs statistics show that I have 30K profit, then I have 10K of additional profit throughout time and PRT statistics thus never show that. This is very significant (for me it is). And if you talk about “months” as the duration unit, in my view it should be about open positions just the same (am I right ?).

    **) : Which could be difficult for any formula in general, when the gains are exorbitant.

    PS: I am staring at what you see in the 2nd attachment – Return on Initial Capital – and wonder whether that already tells all ? –> No, because it does not tell how long it took me to obtain that figure. Thus square that with the duration somehow ?

    11/27/2021 at 12:58 PM #182395
    PeterSt

    I will try to work out two examples, based on the “PS” in my previous post. It is just to test whether it would work. In this post the first example :

    My exposure is USD 224000;
    The attachment shows that the profit of USD 314,40 (including payed spread) was obtained over the period of 7 hours. Thus, 314,40 / 7  = 44,91 per hour.

    I need to normalise to a standard exposure amount, which is set to USD 1000. 44,91 / 224 = 0.200491 per hour (per 1000). Thus my score is 0.200491;
    Because of the small number, we could work it out per day. Thus x 24 = 4.81178. This looks to be a nice number to work with.

    Next example in the next post …
    (I don’t want to hijack the thread – it’s just ideas)

    11/27/2021 at 1:03 PM #182398
    nonetheless
    GraHal wrote:

    Wouldn’t it be fantastic if we had a ‘free text column’  in the Detailed Report where we could enter such formula as you propose above and PRT computed automatically?

    yes! i was thinking the same thing. For something like this to be useful it would have to be automated, so you just get the number as a part of the report, relating all the stats.

    PeterSt wrote:

    The Duration could be problematic with dozens of trades per day

    I was thinking of the duration of the backtest rather than the trade, eg. looking at the performance of a 1min algo with a 3 year BT compared to a 1 hour algo with a 10 year BT

    11/27/2021 at 1:19 PM #182399
    PeterSt

    Normal trading day from yesterday (thus no AutoTrading) :

    I multiplied all figures by a consistent though unknown number. But the score figure would be comparable.

    Exposure was 390000 (390K);
    In this case I’d need to say that the trading day lasts 24 hours, never mind it is not so at all for any equity. The fact that all (USA) closed at 7pm my time (instead of 11pm) should not interfere because it gets too complicated otherwise. Futures, stocks, USA, Europe … thus, 24 hours.
    My profit was 1700. This is 70,83 per hour.
    The exposure should be normalised to 1000, this 70,83 / 390 = 0.18162 per hour.
    Per 24 hours this is 4.35876.

    From my previous post :

    PeterSt wrote:

    Thus x 24 = 4.81178.

    and I am startled. My Autotrading may outpace me.

    The fact that I manipulated the figures twice (one time the multiplier for obfuscation and one time the normalising to 1000 exposure) is not important.
    So my score yesterday was USD 4.81178.

    BTW in euros (https://a2a3f5h6.rocketcdn.me/ 1.12) this would be  4.29623.

    11/27/2021 at 1:47 PM #182400
    PeterSt
    nonetheless wrote:

    I was thinking of the duration of the backtest rather than the trade, eg. looking at the performance of a 1min algo with a 3 year BT compared to a 1 hour algo with a 10 year BT

    Clear.
    So the 7 hours I showed for a back test of the past month (20 days) comes to

    -1456.94 / 224 / 20 = -0.32521 (for a day).
    Once we would be used to such a number, we could see that it is quite harmless to have a minus of -0.32521 (it is negative). However, what I would do myself is incorporate the law of large numbers, and somehow integrate the fact that this is over 578 trades, which make the loss diminish. Otoh it would also tell me that large profit is impossible just the same. What it also tells me is that this algo is harmless to run and can be observed in live for improvement.

    If this would have been spread over two trades, it would be a dangerous beast. And thus I would incorporate the 578 trades somehow. … Normalise that to 1000 ? maybe.
    -0.32521 / 578 * 1000 = -0.56265.
    Would this have been 2 trades :
    -0.32521 / 2 * 1000 = -162.60500 (ouch – not-dangerous numbers should be between -1 and 1 ?).

     

     

    11/27/2021 at 2:48 PM #182404
    nonetheless

    Peter, I think we may be at cross purposes. This is an example of what I had in mind.

    The attached image shows the results from 2 different backtests. The one on the left is SP500 with a position size of 1.66, on the right is DJI position size 0.2

    The exposure of each is therefore broadly comparable: 4572 x 1.66 = 7589, 34640 x 0.2 = 6928

    As we’re dealing with ratios between respective values, the difference in the absolute value shouldn’t be too significant (???)

    Based on the above formula, this would give the following scores:

    Algo 1: 19.19/41.82 * 81.71 * 7703 / 436 * 2.05 / 72 (months) = 18.86

    Algo 1: 55.59/35.78 * 53.48 * 11546 /467 * 1.79 / 137 (months)  = 26.84

    The score would be an abstract value where higher is better.

    11/27/2021 at 2:57 PM #182406
    nonetheless

    (typo: the second calculation is Algo 2, on the right)

    11/27/2021 at 4:24 PM #182413
    robertogozzi

    According to the orginal formula (I only changed duration in DAYS, but it’s just a timeframe to be changed to use months, weeks, etc…) by nonetheless, I wrote this snippet with an example (let me know if I interpreted the formula incorrectly or you find any error in calculations):

     

    4 users thanked author for this post.
    avatar Nicolas, avatar GraHal, avatar nonetheless
    11/27/2021 at 4:37 PM #182415
    robertogozzi

    I added a multipier in lines 74-77 to be able to weigh each parameter differently.

    I edited line 57 (but it doesn’t change the output).

     

    11/27/2021 at 4:53 PM #182416
    JS

    @nonetheless First nice idea but when I look at the formula I am not sure what the meaning of your calculation is 🙂

    The first term: Average Gain / Average Loss = (already) the Risk / Reward Ratio

    Then multiplying the Risk / Reward Ratio with %Winning what does that mean???

    When you work this out (Risk/Reward Ratio * %Winning) you get: Average Win * Losing Trades / Average Loss  * Total Trades (what is the meaning of that?)

     

    11/27/2021 at 5:02 PM #182417
    nonetheless
    JS wrote:

    Then multiplying the Risk / Reward Ratio with %Winning what does that mean???

    there is a definite relation between these 3 factors. When %Win is high (+80%) then the average loss will also be high (compared to average gain)

    An average %win of 50% might look poor, but if the average win is higher than the average loss it still works.

     


    @Roberto    , the code looks amazing, thanks for that – I would never have thought of doing it that way. I’ll run some tests as soon as I get the time.

    1 user thanked author for this post.
    avatar GraHal
    11/27/2021 at 5:43 PM #182424
    JS

    What I mean is:

    Average Gain = Only Profit / Winning Trades

    Average Loss = Only Loss / Losing Trades

    %Winning = Winning Trades / Total Trades

    When you divide: Average Gain / Average Loss and multiplying it with %Winning then your Winning Trades will fall out (of the equation)

    What is left is: Average Gain * Losing Trades / Average Loss  * Total Trades (what is the meaning of that?)

     

  • Author
    Posts
Viewing 15 posts - 1 through 15 (of 26 total)
1 2

Find exclusive trading pro-tools on ProRealCode Marketplace

Similar topics:
very concerned about ig’s data and/or prt execution of data
analysing ranges of data
asx stock live data needed
comparing instruments and using the data on indicators or systems
conversion of vold data and vold ratio code from trading view to realtime
detail report sometimes showing double the number of trades in the optimise repo
detailed report in different currency
downloading data to external files. is possible?
end of week data
incomplete historical data

Create your free account now and post your request to benefit from the help of the community
Register or Login