Is this system too good to be true?

• 08/27/2019 at 7:59 PM #105669
  Thally

  Hello all,

  I have developed a simple trend following system which I apply to USDJPY. The system is very simple. It marks out the opening price at 22:00 (UK time). It then buys or sell an m5 close either side of this line in the sand. The initial stop loss is behind the most recent swing high/low and the position size is determined from the distance to the stop and the max risk percentage of equity (1% in this example). There is a breakeven stop loss which kicks in after price has moved in our favour by a certain multiple of the 10 period ATR. Positions are then held until 20:00 (1 hour before the futures markets close) or until they are stopped out.

  The backtest delivers incredible returns – >4000% between 2016 and 2019 from a 5k initial equity and 0.8 point spread (average spread from IG is 0.79 for USDJPY) – see attached performance report. It has the typical drawdowns expected from a trend following system, but nevertheless the system performance in the 200k backtest is pretty incredible. I have done no optimisation so the performance can’t be driven by overfitting (or at least I think it can’t). So my question for you is… is this system too good to be true? Surely anything that looks this good has to be wrong somehow? If its not then merry Christmas here’s a new trend following system for your Arsenal.

  Thally

  // strategy looks for break above or below previous days 20:00 price // stop loss behnind swing high or low // position size is 1% risk based on distance to stop loss // positions managed with a breakeven stoploss after an ATR multiple move in our favour // positions are exited at 20:00 DEFPARAM CumulateOrders = False // Cumulating positions deactivated DEFPARAM FlatAfter = 200000 // Spread ONCE sp = 0.8 // money management ONCE Capital = 5000 ONCE Risk = 0.01 equity = Capital + StrategyProfit maxrisk = round(equity*Risk) PositionSizeL = abs(round((maxrisk/StopLossL)/PointValue)*pipsize) PositionSizeS = abs(round((maxrisk/StopLossS)/PointValue)*pipsize) // window for calculating yesterdays 22:00 price ONCE starttime = 220000 ONCE endtime = 220500 if opentime = starttime then op = open endif if opentime = endtime then lastop = op endif graphonprice lastop as "op" // Trade Counter ONCE TradeCount = 0 IF INTRADAYBARINDEX=0 then TradeCount = 0 ENDIF // Time Filters ExitPos = Time >= 200000 // Long IF Not ONMARKET AND close CROSSES OVER lastop AND TradeCount < 10 THEN BUY PositionSizeL PERPOINT AT MARKET TradeCount = TradeCount + 1 StopLossL = abs(close - LOWEST[10](low)) + sp SET STOP LOSS StopLossL ENDIF // exit longs IF LONGONMARKET AND ExitPos THEN SELL AT MARKET ENDIF // short IF Not ONMARKET AND close CROSSES UNDER lastop AND TradeCount < 10 THEN SELLSHORT PositionSizeS PERPOINT AT MARKET TradeCount = TradeCount + 1 StopLossS = abs(close - HIGHEST[10](high)) + sp SET STOP LOSS StopLossS ENDIF // exit shorts IF SHORTONMARKET AND ExitPos THEN EXITSHORT AT MARKET ENDIF // Breakeven stop loss //reset the stoploss value IF NOT ONMARKET THEN newSL = 0 ENDIF // Parameters for determining when BE stop kicks in TrailingStart = 4 * AverageTrueRange[10] * pipsize TrailingStep = 1 * AverageTrueRange[10] * pipsize // Longside IF LONGONMARKET THEN // move to breakeven IF newSL = 0 AND CLOSE - TRADEPRICE(1) >= TrailingStart THEN newSL = TRADEPRICE(1) + TrailingStep ENDIF ENDIF //Shortside IF SHORTONMARKET THEN // move to breakeven IF newSL = 0 AND TRADEPRICE(1) - CLOSE >= TrailingStart THEN newSL = TRADEPRICE(1) - TrailingStep ENDIF ENDIF //stop order to exit the positions IF newSL > 0 THEN SELL AT newSL STOP EXITSHORT AT newSL STOP ENDIF

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

  // strategy looks for break above or below previous days 20:00 price // stop loss behnind swing high or low // position size is 1% risk based on distance to stop loss // positions managed with a breakeven stoploss after an ATR multiple move in our favour // positions are exited at 20:00   DEFPARAM CumulateOrders = False // Cumulating positions deactivated DEFPARAM FlatAfter = 200000 // Spread ONCE sp = 0.8 // money management ONCE Capital = 5000 ONCE Risk = 0.01 equity = Capital + StrategyProfit maxrisk = round(equity*Risk) PositionSizeL = abs(round((maxrisk/StopLossL)/PointValue)*pipsize) PositionSizeS = abs(round((maxrisk/StopLossS)/PointValue)*pipsize) // window for calculating yesterdays 22:00 price ONCE starttime = 220000 ONCE endtime = 220500 if opentime = starttime then op = open endif if opentime = endtime then lastop = op endif   graphonprice lastop as "op"   // Trade Counter ONCE TradeCount = 0 IF INTRADAYBARINDEX=0 then TradeCount = 0 ENDIF   // Time Filters ExitPos = Time >= 200000   // Long IF Not ONMARKET AND close CROSSES OVER lastop AND TradeCount < 10 THEN BUY PositionSizeL PERPOINT AT MARKET TradeCount = TradeCount + 1 StopLossL = abs(close - LOWEST[10](low)) + sp SET STOP LOSS StopLossL ENDIF   // exit longs IF LONGONMARKET AND ExitPos THEN SELL AT MARKET ENDIF   // short IF Not ONMARKET AND close CROSSES UNDER lastop AND TradeCount < 10 THEN SELLSHORT PositionSizeS PERPOINT AT MARKET TradeCount = TradeCount + 1 StopLossS = abs(close - HIGHEST[10](high)) + sp SET STOP LOSS StopLossS ENDIF   // exit shorts IF SHORTONMARKET AND ExitPos THEN EXITSHORT AT MARKET ENDIF   // Breakeven stop loss //reset the stoploss value IF NOT ONMARKET THEN newSL = 0 ENDIF   // Parameters for determining when BE stop kicks in TrailingStart = 4 * AverageTrueRange[10] * pipsize TrailingStep = 1 * AverageTrueRange[10] * pipsize   // Longside IF LONGONMARKET THEN // move to breakeven IF newSL = 0 AND CLOSE - TRADEPRICE(1) >= TrailingStart THEN newSL = TRADEPRICE(1) + TrailingStep ENDIF ENDIF //Shortside IF SHORTONMARKET THEN // move to breakeven IF newSL = 0 AND TRADEPRICE(1) - CLOSE >= TrailingStart THEN newSL = TRADEPRICE(1) - TrailingStep ENDIF ENDIF //stop order to exit the positions IF newSL > 0 THEN SELL AT newSL STOP EXITSHORT AT newSL STOP ENDIF

   

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/27/2019 at 8:09 PM #105671
  Magifina

  Hi Thally,

  Your strategy is very risky. Your max drawdown is 10 times higher than your initial capital and your position size is too high. If you cumulate several losing trade you will burst your account.

  

  Magifina

  Participant

  Topics: 2

  Replies: 41

  Been thanked: 12 times
  08/27/2019 at 8:25 PM #105673
  Thally

  Thanks for the comment. The risk per trade is always 1% of the current account equity. Those large drawdowns only occur late on in the time series when the account size is much much larger than the initial equity. Because the system uses a % of equity the position sizing grows as the account size grows. How is that risky?

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/27/2019 at 10:20 PM #105676
  GraHal

  How come I don’t get anywhere near your results??

  Attached with spread = 2 (currently on my screen usdjpy spread = 5! )

  

  GraHal

  Participant

  Topics: 213

  Replies: 8791

  Been thanked: 2190 times
  08/27/2019 at 10:31 PM #105679
  GraHal

  oh sorry I dropped the Lot size  to 1 in above results.

  Below results are as per the original code over 100k bars with spread = 2

  

  GraHal

  Participant

  Topics: 213

  Replies: 8791

  Been thanked: 2190 times
  08/27/2019 at 10:55 PM #105683
  Thally

  Its the spread. I also get shite results for spread = 2. I guess the question is whether a spread of 0.8 is reasonable. The strategy won’t trade between 20:00 and 00:00 (UK time) so that should avoid periods like now when the spread is large.

   

  IG quote 0.79 as the average UJ spread (https://www.ig.com/uk/help-and-support/spread-betting-and-cfds/fees-and-charges/what-are-igs-forex-spread-bet-product-details). However I have not monitored it meticulously over 24h and IG say they can’t give the numbers broken down over 24h. Given that UJ is quite volatile after 00:00 I’d imagine the spreads come down to the average after Sydney and Tokyo open. My live (demo) testing is in line with the backtest which suggests spreads of 2 are unrealistically large.

  1 user thanked author for this post.

  GraHal

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/27/2019 at 10:57 PM #105684
  Thally

  with a more conservative spread of 1  I get the attached (over 200k m5 bars). Still pretty epic performance.

   

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/27/2019 at 11:25 PM #105689
  robertogozzi

  Have you tried to do some math, as someone suggested?

  My rule is to have in my account a least (MaxDrawDown * 3)+RequiredMargin. In your case you should have about £ 80,000 on your account, while you decided to backtest with just £ 5,000! Try again entering £ 80,000 to spot the difference, then spread it over 2 and a half years.

  That’s what you are going to gain per year, provided you have £ 80,000 and the future performs like the past!

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17153

  Been thanked: 4040 times
  08/27/2019 at 11:58 PM #105691
  Thally

  Hi Roberto,

  attached is backtest starting with 80k. It still does very well, but of course the % gain is lower. I chose 5k because it is more representative of my reality. Perhaps Im not understanding yours and Magifina’s logic, but I fail to see the logic behind setting your account relative to the max drawdown  in a system that capitalises on compounding? The position size function always risks a fixed % of equity so as the account equity grows the risk (in $ terms) grows too. The max drawdown in the strategy backtest does not happen when the account size is 5k, it happens when the account size more like 50k, so in % terms that’s a 50% drawdown. Unless the system lost continuously I think an account blowout risking 1% of equity on each trade is very unrealistic. As you can see the win rate is about 40%, which is pretty good for a trend following system.

  I guess it would be a useful addition for PRT to give max drawdown in percentage rather than absolute terms to avoid these misunderstandings.

  Cheers for the comments though…

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/28/2019 at 2:01 PM #105739
  Thally

  So… it turns out the results were too good to be true. The bug was in the position size calculation. As Nicolas kindly pointed out in another thread, because I call the position size function before the stop loss function the position size is always delayed by one bar and therefore calculated incorrectly. When we calculate positions size correctly the results are more modest, but still pretty good – not tradable yet in my opinion but not bad. Here’s the results with the correct position size calculation.

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/28/2019 at 2:41 PM #105743
  Vonasi

  I would suggest always developing, back testing and forward testing with a fixed position size. If a strategy is profitable then position size is irrelevant. Once you are happy that a strategy is capable of making profit – even if it is only small due to low position size then run a back test with some more dynamic position sizing to see how things like draw down would have been affected.

  Throwing position sizing in at the beginning is just throwing an extra spanner in the works from the start. The only time that position sizing should be included at the start of development is if you are trying to build a casino strategy that uses position sizing to create its profit or if you are trying to use dynamic position sizing based on calculated probability of return.

  1 user thanked author for this post.

  Thally

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  08/28/2019 at 2:45 PM #105744
  Thally

  Good point well made.

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
  08/28/2019 at 2:47 PM #105745
  Thally

  I never added the new results as an attachment. Here for completeness.

  

  Thally

  Participant

  Topics: 25

  Replies: 56

  Been thanked: 8 times
• Author
  Posts