Z-Score to improve strategies
Forums › ProRealTime English forum › ProOrder support › Z-Score to improve strategies
- This topic has 18 replies, 6 voices, and was last updated 9 months ago by Brad.
-
-
07/30/2021 at 11:58 AM #174383
I am posting my version of the code to calculate Z-Score and manage it, in order to embed it into your strategies. It also comes with a simple test strategy.
What is Z-score?
Z-score measures the distance between the mean of some set of the statistical results and the given observation.
A trader might suspect a dependency if he or she experiences a run of several consecutive profitable trades, or a run of several consecutive unprofitable trades. “Obviously, there was some kind of dependency or serial correlation among your trades [in this case], where winners were followed by winners and losers were followed by more losers,” Thomas Stridsman wrote in the April 1998 issue of Futures. “If this happens again, you’ll want to exploit the good times and perhaps avoid trading altogether in bad times.”
In Forex systems, traders are interested in Z-score not of a trade’s return (profit/loss size) but rather in a Z-score of the outcome — was it a profitable one or a losing one?
Z-score, calculated using winning and losing streaks, measures the dependence between the outcome of the previous position and the outcome of the next position.
If we consider profitable positions as positive results and losing positions as negative statistical results, counting the total number of all wins, losses, overall trades, as well as the number of win and loss streaks, we can calculate a Z-score for a given
trading strategy.
Usually, Z-score fluctuates between -3 to +3, but sometimes, can go above and below these “limits”. A Z-score value of 0 means that we are dealing with completely random results.
Each Z-score value has also a probability of dependence (P) associated with it, which informs us of how probable dependence between the trades is. P values below -2 and above +2 have high (>95%) probability of dependence between trades:
- a positive Z-score means that:
– a profitable position is likely to be followed by a losing one
– a losing position should probably be followed by a winning one.
Basically, it means that the probability of long winning and losing - a negative Z-score means that:
– profitable positions are likely to be followed by more profitable positions
– losing positions are to be followed by more losing positions.
Basically, it means that winning or losing streaks are probable.
It should also be noted that Z-score calculation makes sense only for sufficiently
large samples. Math literature suggests a sample size of no less than 51 (some talk about 30) to get a reliable Z-score value.
Z-Score123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////Z-Score (code)////The results can be INCORRECT when, on the SAME bar, the following 3 conditions are met://// 1) a trade is closed// 2) a new trade is opened (usually due to a Stop & Reverse)// 3) the new trade hits TP////Furthermore, optimization should be avoided when accumulating positions, unless it's//possible to consider all of them as one.////(streaks)-https://www.forextraders.com/forex-education/forex-money-management/using-the-z-score-to-determine-trade-size///(streaks)-https://www.earnforex.com/guides/z-score-optimization-in-forex-trading/////(runs) -https://www.mypivots.com/dictionary/definition/233/z-score//(runs) -https://www.referenceforbusiness.com/encyclopedia/Val-Z/Z-Score.html//// Counting STREAKS (+ are Winning trades - are Losing trades)////trades: +-++--++-----+---+++- (21 trades)//tally: 1 2 3 4 5 6 (6 streaks) a STREAK is a TWIN pair followed// by a different sign////// Counting RUNS (+ are Winning trades - are Losing trades)////trades: +-++--++-----+---+++- (21 trades)//tally: 12 3 4 5 67 8 9 (9 runs) a RUN is any sign followed// by a different sign//// --- start of Z-Score codeONCE Periods = 0ONCE Streaks = 0ONCE CurTrade = 0ONCE TotalWin = 0ONCE TotalLose = 0Periods = Periods + 1MyProfit = StrategyProfit// tally Total winning + losing streaks (streak = consecutive winning or losing trades)IF MyProfit <> MyProfit[1] AND (BarIndex > 0) THENIF MyProfit > MyProfit[1] THENCurTrade = 1ELSIF MyProfit < MyProfit[1] THENCurTrade = -1ENDIF//---------------------------------------------------------------------------------// code using STREAKS (it seems to be returning only negative values)// (a sign change preceded by a twin, i.e. a couple of the same sign)////Twins = (CurTrade = CurTrade[1]) //TWO identical trades to make a STREAK//Streaks = Streaks + ((CurTrade <> CurTrade[1]) AND Twins[1])//now Different trade +// prior candle's TWINS////---------------------------------------------------------------------------------// code using RUNS (like PRT seem to be doing)// (a run is each sign change, no matter twins)Streaks = Streaks + (CurTrade <> CurTrade[1]) //now Different trade//---------------------------------------------------------------------------------//// tally Winning & Losing tradesTotalWin = TotalWin + (CurTrade = 1)TotalLose = TotalLose + (CurTrade = -1)N = TotalWin + TotalLoseENDIFIF Streaks > 0 THENP = 2 * TotalWin * TotalLoseR = StreaksZscore = ((N*(R-0.5))-P) / sqrt((P*(P-N)) / (N-1))Zscore = round(Zscore * 100) / 100ELSEZscore = 0ENDIFZpos = Zscore > 0 //or whatever else > 0Zneg = Zscore < 0 //or whatever else < 0// --- end of Z-Score code////*********************************************************************************// Z-Score managementONCE SkipOneTrade = 0ONCE MinTrades = 30 //30 or 51 trades minimum to consider Z-Score Reliable// as suggested on some websitesIF Zpos THEN//Positive Z-ScoreIF (CurTrade = 1) AND (Curtrade[1] = -1) THEN //Winning trade following a Losing oneIF N >= MinTrades THEN //use Z-Score only when MinTrades+ have been talliedSkipOneTrade = 1ENDIFENDIFELSIF Zneg THEN//Negative Z-ScoreIF (CurTrade = -1) AND (CurTrade[1]= 1) THEN //Losing trade following a Winning oneIF N >= MinTrades THEN //use Z-Score only when MinTrades+ have been talliedSkipOneTrade = 1ENDIFENDIFENDIF//*********************************************************************************//// S T R A T E G Y//ONCE MA = 100 //100ONCE T = 1 //1=emaIF close CROSSES OVER average[MA,T] AND Not OnMarket THENIF SkipOneTrade THEN //Do not open any trade when SkipOneTrade is TRUESkipOneTrade = 0 //clear it for the next tradeELSEBUY AT MarketENDIFELSIF close CROSSES UNDER average[MA,T] AND Not OnMarket THENIF SkipOneTrade THEN //Do not open any trade when SkipOneTrade is TRUESkipOneTrade = 0 //clear it for the next tradeELSESELLSHORT AT MarketENDIFENDIFset stop ploss 250 //250set target pprofit 2500 //2500////*********************************************************************************// Nicolas' trailing stop functiontrailingstart = 25 //25 trailing will start @trailinstart points profittrailingstep = 5 //5 trailing step to move the "stoploss"//reset the stoploss valueIF NOT ONMARKET THENnewSL=0ENDIF//manage long positionsIF LONGONMARKET THEN//first move (breakeven)IF newSL=0 AND close-tradeprice(1)>=trailingstart*pipsize THENnewSL = tradeprice(1)+trailingstep*pipsizeENDIF//next movesIF newSL>0 AND close-newSL>=trailingstep*pipsize THENnewSL = newSL+trailingstep*pipsizeENDIFENDIF//manage short positionsIF SHORTONMARKET THEN//first move (breakeven)IF newSL=0 AND tradeprice(1)-close>=trailingstart*pipsize THENnewSL = tradeprice(1)-trailingstep*pipsizeENDIF//next movesIF newSL>0 AND newSL-close>=trailingstep*pipsize THENnewSL = newSL-trailingstep*pipsizeENDIFENDIF//stop order to exit the positionsIF newSL>0 THENSELL AT newSL STOPEXITSHORT AT newSL STOPENDIF//*********************************************************************************//graph Zscore coloured(255,0,0,255)graph TotalWingraph TotalLoseOne of the attached pics shows correspondance between these calculations and PRT’s.
I tested it on DAX, h1, 200K units.
In the attached PIC about the formula, PRT stands for “the above formula in PRT Language”, not “PRT Formula”.
07/31/2021 at 12:00 AM #174413While the computation is working fine, I had to slightly change the Z-Score management and the strategy, as follows (you can copy & past it or you can download the correct version of the ITF file):
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182//*********************************************************************************// Z-Score managementONCE SkipOneTrade = 0ONCE SkipFlag = 0ONCE MinTrades = 51 //30 or 51DirectionSwitch = (ShortOnMarket AND LongOnMarket[1]) OR (ShortOnMarket[1] AND LongOnMarket)IF Zpos THENIF (CurTrade = 1) AND SkipFlag = 0 AND ((Not Onmarket) OR DirectionSwitch) THENIF N >= MinTrades THENSkipOneTrade = 1SkipFlag = 1ENDIFENDIFELSIF Zneg THENIF (CurTrade = -1) AND SkipFlag = 0 AND ((Not Onmarket) OR DirectionSwitch) THENIF N >= MinTrades THENSkipOneTrade = 1SkipFlag = 1ENDIFENDIFENDIF//*********************************************************************************ONCE MA = 100 //100ONCE T = 1 //1=emaIF close CROSSES OVER average[MA,T] AND Not OnMarket THENIF SkipOneTrade THENSkipOneTrade = 0ELSEBUY AT MarketSkipFlag = 0ENDIFELSIF close CROSSES UNDER average[MA,T] AND Not OnMarket THENIF SkipOneTrade THENSkipOneTrade = 0ELSESELLSHORT AT MarketSkipFlag = 0ENDIFENDIFset stop ploss 500 //500set target pprofit 5000 //5000////*********************************************************************************//trailing stop functiontrailingstart = 50 //50 trailing will start @trailinstart points profittrailingstep = 5 //5 trailing step to move the "stoploss"//reset the stoploss valueIF NOT ONMARKET THENnewSL=0ENDIF//manage long positionsIF LONGONMARKET THEN//first move (breakeven)IF newSL=0 AND close-tradeprice(1)>=trailingstart*pipsize THENnewSL = tradeprice(1)+trailingstep*pipsizeENDIF//next movesIF newSL>0 AND close-newSL>=trailingstep*pipsize THENnewSL = newSL+trailingstep*pipsizeENDIFENDIF//manage short positionsIF SHORTONMARKET THEN//first move (breakeven)IF newSL=0 AND tradeprice(1)-close>=trailingstart*pipsize THENnewSL = tradeprice(1)-trailingstep*pipsizeENDIF//next movesIF newSL>0 AND newSL-close>=trailingstep*pipsize THENnewSL = newSL-trailingstep*pipsizeENDIFENDIF//stop order to exit the positionsIF newSL>0 THENSELL AT newSL STOPEXITSHORT AT newSL STOPENDIF//*********************************************************************************//graph Zscore coloured(255,0,0,255)graph TotalWingraph TotalLose07/31/2021 at 8:34 AM #174418Link to above added as Log 295 here …
1 user thanked author for this post.
07/31/2021 at 6:50 PM #174462Roberto, I copy-pasted your Z-score code (partly from your 2nd post), but something seems to go wrong / off ? The strategy is my own.
-1.13 (PRT) vs +11.2 (yours). Notice the ever increase throughout the chart (of ~15 trading days).
The difference I see is that I am dealing with Brokerage Fees (which are significant) which influence the amount of gain.
Thanks !
PS: The one but last attachment is redundant.
07/31/2021 at 8:28 PM #174469Fees, spread etc… do not affect the results.
Only accumulation of positions and the three conditions stated in the comments (before the code starts) can affect Z-Score.
08/01/2021 at 4:30 AM #174474a new trade is opened (usually due to a Stop & Reverse)
That happens (happened) 3 times, and I figured it unimportant. After switching off that feature, no changes except for the results somewhat. The Z-Score PRT reports also changed a little (1st attachment).
a trade is closed
That does not happen as far as I am aware of.
the new trade hits TP
Impossible because I control this myself (no Set Target $Profit, etc.).
I assed a part of the chart which better shows what happens (2nd attachment) . The red graph is obviously your Z-Score.
I will try to look into it later. But if you have more hints, please don’t hesitate.
08/01/2021 at 4:43 AM #174477It should be something with the streaks. You probably can tell (I so far can not);
The previous post showed a winning streak of 4, which increases the Z-Score multiple times. In the attachment below there’s a losing streak of 2, and this decreases the Z-Score.
I obviously have more winning streaks than losing streaks (also see 2nd attachment in my first post from yesterday), zo net the Z-Score increases. … Without looking into the code, it seems that a 3rd win (or loss) in a streak again increases the Z-Score, which should not be ? (because it is the same streak). Probably …More later.
08/01/2021 at 9:35 AM #17448208/01/2021 at 10:17 AM #174486And if I apply my own method of tallying Runs, I again arrive at your exact number.
Something else is, that the feeling of it all should tell that I never would have such a low Z-Score as PRT is telling me. Thus Roberto, your remark that PRT would be using the Runs method could be correct, but the way PRT is calculating that should be (way ?) off.
What remains is why your calculated Z-Score *is* the same as PRT’s …
08/01/2021 at 11:38 AM #174492It’s worth while testing it with several strategies on different TF’s. I have, and I think the formula is the same one.
PRT, on the other side, can overcome my limitations (as stated in the comments), as they can exactly tell if more than one trade was exited on the same bar, which we can’t, and manage accumulation (though accumulation is discouraged as a general rule according to some websites).
Anyway, being able to manage Z-Score and also drawdown (https://www.prorealcode.com/topic/when-to-stop-a-strategy-and-money-management-code-snippet/#post-174460) may help improving strategies.
Neither is the Holy Grail, a poor strategy is likely to remain as such, but a good enough strategy could be improved a bit.
08/01/2021 at 1:17 PM #174495I would love to have it readily available in my strategies because it would in real time show “what’s next”. Actually as how the real Fx guys apply it. So I guess I won’t rest until I have it going as you intended it. It intrigues !
Thank you again for making this available. 🙂 It appears not all that difficult, but someone has/had to do it (invest the time).
08/01/2021 at 6:54 PM #174521I think I found the culprit.
1IF MyProfit <> MyProfit[1] AND (BarIndex > 0) THENIt goes wrong at that line (or because of that line used as a trigger);
The first attachment shows a small drain of money at the entry (this happens at each entry). The difference you see reported as -60.
The second attachment shows the used BrokerFee (this is IB). It is 60 … (counts for entry and counts for exit again – this latter does not bother regarding the issue).
And so each entry is seen as a “Streak” and counts as such. Also, it will count as a loss and it confuses everything.
When I eliminate the BrokerFee, all is fine (fourth attachment) – I’ll grant PRT the small difference (Reverse Orders is inactive though measurement is over 157 trades so maybe rounding errors are in place somewhere.Remedy : Let trigger the Z-Score code by a means which does not incorporate the entry. It is not easy (for me) to see how to do that, knowing that we’d want general code for that (like Roberto’s example which nicely has it all at the top of the program code). I myself probably would dare to code something like :
12345678910111213// Untested :BrokerFeeAmount = 60 // Set to what's in the Editor/Backtest startup screen.DontIncreaseStreaks = 0If MyProfit < MyProfit[1] then // We lose some ?// Is it the entry which charges the BrokerFee (Commission) ?If MyProfit - MyProfit[1] = -BrokerFeeAmount then // This compares at 5 decimals.DontIncreaseStreaks = 1endifendif[...]I hope this addition helps someone (BackTesting with PRT-IB or other brokers who charge commission).
1 user thanked author for this post.
03/27/2023 at 1:39 PM #212191P = 2 * TotalWin * TotalLose
Do you see a problem changing line 67 to “P = max(1, 2 * TotalWin * TotalLose)” ?
There’s a Division by zero error after the first trade since only win or lose has a 1 and the other is 0. Figured this way was more appropriate than “IF Streaks > 0 AND TotalWin > 0 AND TotalLose > 0 THEN”.
1 user thanked author for this post.
03/27/2023 at 5:38 PM #212243Well spotted paisantrader. Thank you 🙂
01/23/2024 at 5:19 PM #226771Hi Robertogozzi,
I am stumbling upon this interesting idea;
But I cannot see how I can use it concretly in an ON OFF strategy management;Can you explain it on this example drawn directly from your code and strat.
Thanks
- a positive Z-score means that:
-
AuthorPosts
Find exclusive trading pro-tools on