Strategy Robustness Tester

• 09/01/2019 at 1:34 PM #106072
  Vonasi

  Following on from a discussion elsewhere regarding walk forward testing I had the idea to create a bit of code that you can add to any strategy and see how it would have performed if you had traded on only odd days or only on even days, or odd or even months or odd and even years or any combination of the three. By comparing the output of all 23 tests we can get an idea of how consistently a strategy makes profit. The gain figure in the optimisation results should be ignored as the number of trades for each test will be very different. The average gain per trade, win rate and equity curve for each test can tell us a lot about the strategies robustness though.

  You just add the code to the start of your strategy code and then insert the extra condition ‘and tradeon’ in the conditions whenever you send buy or sellshort orders to the market. Then just optimise with a variable ‘choice’ with values 0 to 22 step 1. Then just click on each result in the optimisation report to see the equity curve for each test.

  Here is the code to add to the start of your strategy in case it is of use to anyone:

  //Day Month and Year Robustness Tester //By Vonasi //20190901 //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //odd days only if choice = 1 and oddday then tradeon = 1 endif //even days only if choice = 2 and evenday then tradeon = 1 endif //odd months only if choice = 3 and oddmonth then tradeon = 1 endif //even months only if choice = 4 and evenmonth then tradeon = 1 endif //even years only if choice = 5 and evenyear then tradeon = 1 endif //odd years only if choice = 6 and oddyear then tradeon = 1 endif //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150

  //Day Month and Year Robustness Tester //By Vonasi //20190901   //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only   oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0   tradeon = 0   if choice = 0 then tradeon = 1 endif   //odd days only if choice = 1 and oddday then tradeon = 1 endif   //even days only if choice = 2 and evenday then tradeon = 1 endif   //odd months only if choice = 3 and oddmonth then tradeon = 1 endif   //even months only if choice = 4 and evenmonth then tradeon = 1 endif   //even years only if choice = 5 and evenyear then tradeon = 1 endif   //odd years only if choice = 6 and oddyear then tradeon = 1 endif   //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif   //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif   //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif   //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif   //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif   //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif   //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif   //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif   //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif   //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif   //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif   //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif   //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif   //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif   //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif   //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif

   

  Then alter your strategy code to use the TRADEON variable so that trades are only opened if the day, month and year meet the test criteria.

  if (your long entry conditions) and tradeon then buy 1 contract at market endif if (your short entry conditions) and tradeon then sellshort 1 contract at market endif

  1 2 3 4 5 6 7 8

  if (your long entry conditions) and tradeon then buy 1 contract at market endif     if (your short entry conditions) and tradeon then sellshort 1 contract at market endif

   

   

   

   

  6 users thanked author for this post.

  Meta Signals Pro, mr blue, keewee, nfulcher, jeremyben, Nicolas

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/01/2019 at 2:08 PM #106078
  Vonasi

  I added some extra tests based on the day of the week and the month of the year. Only trade on Mondays and only trade in January etc. 41 tests now!

  //Day Month and Year Robustness Tester //By Vonasi //20190901 //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only //23 = day of week = 0 //24 = day of week = 1 //25 = day of week = 2 //26 = day of week = 3 //27 = day of week = 4 //28 = day of week = 5 //29 = day of week = 6 //30 = month = 1 //31 = month = 2 //32 = month = 3 //33 = month = 4 //34 = month = 5 //35 = month = 6 //36 = month = 7 //37 = month = 8 //38 = month = 9 //39 = month = 10 //40 = month = 11 //41 = month = 12 oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //odd days only if choice = 1 and oddday then tradeon = 1 endif //even days only if choice = 2 and evenday then tradeon = 1 endif //odd months only if choice = 3 and oddmonth then tradeon = 1 endif //even months only if choice = 4 and evenmonth then tradeon = 1 endif //even years only if choice = 5 and evenyear then tradeon = 1 endif //odd years only if choice = 6 and oddyear then tradeon = 1 endif //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif //day of week = 0 if choice = 23 and opendayofweek = 0 then tradeon = 1 endif //day of week = 1 if choice = 24 and opendayofweek = 1 then tradeon = 1 endif //day of week = 2 if choice = 25 and opendayofweek = 2 then tradeon = 1 endif //day of week = 3 if choice = 26 and opendayofweek = 3 then tradeon = 1 endif //day of week = 4 if choice = 27 and opendayofweek = 4 then tradeon = 1 endif //day of week = 5 if choice = 28 and opendayofweek = 5 then tradeon = 1 endif //day of week = 6 if choice = 29 and opendayofweek = 6 then tradeon = 1 endif //month = 1 if choice = 30 and openmonth = 1 then tradeon = 1 endif //month = 2 if choice = 31 and openmonth = 2 then tradeon = 1 endif //month = 3 if choice = 32 and openmonth = 3 then tradeon = 1 endif //month = 4 if choice = 33 and openmonth = 4 then tradeon = 1 endif //month = 5 if choice = 34 and openmonth = 5 then tradeon = 1 endif //month = 6 if choice = 35 and openmonth = 6 then tradeon = 1 endif //month = 7 if choice = 36 and openmonth = 7 then tradeon = 1 endif //month = 8 if choice = 37 and openmonth = 8 then tradeon = 1 endif //month = 9 if choice = 38 and openmonth = 9 then tradeon = 1 endif //month = 10 if choice = 39 and openmonth = 10 then tradeon = 1 endif //month = 11 if choice = 40 and openmonth = 11 then tradeon = 1 endif //month = 12 if choice = 41 and openmonth = 12 then tradeon = 1 endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264

  //Day Month and Year Robustness Tester //By Vonasi //20190901   //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only //23 = day of week = 0 //24 = day of week = 1 //25 = day of week = 2 //26 = day of week = 3 //27 = day of week = 4 //28 = day of week = 5 //29 = day of week = 6 //30 = month = 1 //31 = month = 2 //32 = month = 3 //33 = month = 4 //34 = month = 5 //35 = month = 6 //36 = month = 7 //37 = month = 8 //38 = month = 9 //39 = month = 10 //40 = month = 11 //41 = month = 12   oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0   tradeon = 0   if choice = 0 then tradeon = 1 endif   //odd days only if choice = 1 and oddday then tradeon = 1 endif   //even days only if choice = 2 and evenday then tradeon = 1 endif   //odd months only if choice = 3 and oddmonth  then tradeon = 1 endif   //even months only if choice = 4 and evenmonth  then tradeon = 1 endif   //even years only if choice = 5 and evenyear then tradeon = 1 endif   //odd years only if choice = 6 and oddyear then tradeon = 1 endif   //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif   //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif   //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif   //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif   //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif   //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif   //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif   //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif   //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif   //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif   //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif   //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif   //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif   //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif   //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif   //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif   //day of week = 0 if choice = 23 and opendayofweek = 0 then tradeon = 1 endif   //day of week = 1 if choice = 24 and opendayofweek = 1 then tradeon = 1 endif   //day of week = 2 if choice = 25 and opendayofweek = 2 then tradeon = 1 endif   //day of week = 3 if choice = 26 and opendayofweek = 3 then tradeon = 1 endif   //day of week = 4 if choice = 27 and opendayofweek = 4 then tradeon = 1 endif   //day of week = 5 if choice = 28 and opendayofweek = 5 then tradeon = 1 endif   //day of week = 6 if choice = 29 and opendayofweek = 6 then tradeon = 1 endif   //month = 1 if choice = 30 and openmonth = 1 then tradeon = 1 endif   //month = 2 if choice = 31 and openmonth = 2 then tradeon = 1 endif   //month = 3 if choice = 32 and openmonth = 3 then tradeon = 1 endif   //month = 4 if choice = 33 and openmonth = 4 then tradeon = 1 endif   //month = 5 if choice = 34 and openmonth = 5 then tradeon = 1 endif   //month = 6 if choice = 35 and openmonth = 6 then tradeon = 1 endif   //month = 7 if choice = 36 and openmonth = 7 then tradeon = 1 endif   //month = 8 if choice = 37 and openmonth = 8 then tradeon = 1 endif   //month = 9 if choice = 38 and openmonth = 9 then tradeon = 1 endif   //month = 10 if choice = 39 and openmonth = 10 then tradeon = 1 endif   //month = 11 if choice = 40 and openmonth = 11 then tradeon = 1 endif   //month = 12 if choice = 41 and openmonth = 12 then tradeon = 1 endif

   

  2 users thanked author for this post.

  Brad, Elio_58

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/07/2019 at 8:22 AM #106748
  Vonasi

  I felt that using days and weeks and months added some possible seasonality to the robustness testing so I created a ‘Robustness Tester – Lite’. This one just runs three tests:

  • On all the data
  • Only open trades on odd barindex numbers.
  • Only open trades on even barindex numbers.

   

  It is obviously much faster to run and removes all possibility of influence from seasonality.

  As with the other tester add it to the top of your strategy (but under any DEFPARAM instructions) and then add TRADEON to all your first entry conditions.

  //Robustness Tester - Lite //By Vonasi //20190907 //1 = odd barindex //2 = even barindex oddbarindex = barindex mod 2 = 1 evenbarindex = barindex mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //even barindex if choice = 1 and oddbarindex then tradeon = 1 endif //odd barindex if choice = 2 and evenbarindex then tradeon = 1 endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

  //Robustness Tester - Lite //By Vonasi //20190907   //1 = odd barindex //2 = even barindex   oddbarindex = barindex mod 2 = 1 evenbarindex = barindex mod 2 = 0   tradeon = 0   if choice = 0 then tradeon = 1 endif   //even barindex if choice = 1 and oddbarindex then tradeon = 1 endif   //odd barindex if choice = 2 and evenbarindex then tradeon = 1 endif

   

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/07/2019 at 8:39 AM #106749
  Vonasi

  Here is an example of the ‘Robustness Tester – Lite’ on a very simple strategy that just buys dips in the DJI daily since the beginning of 2000 and then sells them if in profit or if a close is greater than the previous high.

  The first image is starting trades on all days, the second on just odd days and the third just even days.

  Yes the equity curves are pretty ugly with horrible draw down but when trading on just even or just odd days the win rate was almost identical and the average gain per trade is actually about a third better. So it seems pretty robust and also highlights one very important thing. If we have a strategy that only allows one position to be open at a time and that position can be open for a few bars then we might be missing out on other potential winning trades. Launching two strategies where one only starts trades on even bars and the other only on odd bars would seem a very sensible thing to do to increase trading opportunities and profit and maybe smooth out the equity curve a bit. In fact on a daily strategy having a strategy for each day of the week would be a good idea as long as it is a robust strategy.

  1 user thanked author for this post.

  Wilko

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/07/2019 at 9:07 AM #106753
  Vonasi

  So I decided to test the exact same simple strategy on just each of the days of the week to see how it worked out. Not quite so robust but still every day was profitable and the win rates were all good. I have no idea how any trades got opened on a Saturday but two did! Unclean data I’m guessing or was there some strange times in history when the stock markets opened on a Saturday?

  Anyway ignoring the Saturdays it is interesting to note that we were able to test the strategy on 1946 trades instead of just the 967 that we could when testing on all the data. Increasing your amount of trade tests by 50% has to be a good thing.

  It is also interesting to note that the strategy definitely performed better on Sunday’s, Monday’s and Friday’s and performance dipped off midweek. Perhaps as a start of the week and end of the week only strategy it has some potential but then we start to enter the world of data mining and we don’t want to go there.

  It would be great if the platform had some way to show a combined equity curve of all the results or any that we selected.

  Note: In the image choice -1 = all and 0 to 6 are Sunday to Saturday.

  2 users thanked author for this post.

  Bard, GraHal

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/07/2019 at 10:28 AM #106758
  Vonasi

  Developing the idea of running several strategies at the same time but just trading on different days of the weeks I thought that if you have a strategy that trades very regularly then just setting one running from different start points would be a good idea. So if it is a Sunday start a strategy and if it is a Monday start another and so on. At some point they may come into synch when they all are off the market at any one time but I adjusted the Robustness Tester to do this anyway to see how it worked out.

  You have to set a start date if you are not testing on all of history. Set it to zero if you want to test on all of history. Set ‘choice’ to be whichever day you want the strategy to start running on or if back testing use it as an optimised variable from -1 to 6. As always with these Robustness Testers add TRADEON to your entry conditions in your strategy.

  Image attached is results for the same DJI daily strategy as above but just starting on different days.

  //Robustness Tester 8 starts in 1 //By Vonasi //20190907 //-1 = tart any day //0 = start if day of week = 0 //1 = start if day of week = 1 //2 = start if day of week = 2 //3 = start if day of week = 3 //4 = start if day of week = 4 //5 = start if day of week = 5 //6 = start if day of week = 6 StartDate = 20000101 //0 = all data Choice = 0 if opendate >= startdate then if choice = -1 then tradeon = 1 endif //day of week = 0 if choice = 0 and opendayofweek = 0 then tradeon = 1 endif //day of week = 1 if choice = 1 and opendayofweek = 1 then tradeon = 1 endif //day of week = 2 if choice = 2 and opendayofweek = 2 then tradeon = 1 endif //day of week = 3 if choice = 3 and opendayofweek = 3 then tradeon = 1 endif //day of week = 4 if choice = 4 and opendayofweek = 4 then tradeon = 1 endif //day of week = 5 if choice = 5 and opendayofweek = 5 then tradeon = 1 endif //day of week = 6 if choice = 6 and opendayofweek = 6 then tradeon = 1 endif endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56

  //Robustness Tester 8 starts in 1 //By Vonasi //20190907   //-1 = tart any day //0 = start if day of week = 0 //1 = start if day of week = 1 //2 = start if day of week = 2 //3 = start if day of week = 3 //4 = start if day of week = 4 //5 = start if day of week = 5 //6 = start if day of week = 6   StartDate = 20000101 //0 = all data Choice = 0   if opendate >= startdate then if choice = -1 then tradeon = 1 endif   //day of week = 0 if choice = 0 and opendayofweek = 0 then tradeon = 1 endif   //day of week = 1 if choice = 1 and opendayofweek = 1 then tradeon = 1 endif   //day of week = 2 if choice = 2 and opendayofweek = 2 then tradeon = 1 endif   //day of week = 3 if choice = 3 and opendayofweek = 3 then tradeon = 1 endif   //day of week = 4 if choice = 4 and opendayofweek = 4 then tradeon = 1 endif   //day of week = 5 if choice = 5 and opendayofweek = 5 then tradeon = 1 endif   //day of week = 6 if choice = 6 and opendayofweek = 6 then tradeon = 1 endif endif

   

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/09/2019 at 6:44 PM #106992
  Vonasi

  I was still concerned that my ‘Robustness Tester – Daily’ still suffered from being vulnerable to daily seasonality and my ‘Robustness Tester – Lite’ didn’t really run enough tests to give meaningful analysis. I wanted something that could run lots of tests and was almost random. So welcome to the ultimate robustness tester – which I call ‘Robustness Tester – Random’!

  It isn’t truly random but it can create many different possible entry patterns so it almost seems like it is. It works by checking the remainder of the barindex number divided by a ‘qty’ number. If the remainder is zero then it allows a trade. It also then compares the remainder to a value j that cycles between 1 and ‘qty’ and if the remainder matches j then it also allows a trade. This cycling is interrupted by a further variable ‘random’ which causes the cycling to pause or skip values which gives us our randomness or if not actual randomness then at least a lot of very different varying combinations of trading patterns.

  As with all the other robustness testers add the code to the top of your strategy (but under any DEFPARAMS) and put the condition TRADEON in your entry conditions and then set optimising variables ‘Qty’ from 2 to whatever you want and ‘Random’ from 1 to whatever you want (maximum = maximum qty value).

  You can also set a ‘startdate’ to only test from that date. If you set this to zero it tests on all of history.

  Another benefit of this tester is that it can be run on weekly chart too as it is barindex based and not day of the week based.

  In the image I have graphed TRADEON and you can see the sort of variety of trading pattern created to test with. The second image shows the results for a simple strategy tested using qty = 10 and random = 5. All the win rates are very consistent and all tests were profitable – not all the same but I think that if six identical versions of this strategy were set to run each only allowed to open trades on one day of the week then the combined equity curve should be satisfactory.

  In my opinion this version of robustness tester is now a very powerful tool. It is incredible how many strategies that can look great but if you take a random sample of the trades or even just change the start date by one day they can fall apart. This tester now lets us really test that. I think this tester might actually be the most important thing that I have coded so far! It will be fun to try it on some of the strategies posted in the forums….

  //Robustness Tester - Random //By Vonasi //20190909 //StartDate = 20000101 //Qty = 10 //Random = 3 once j = 0 once flag = 1 if flag = 1 then j = j + 1 if j > qty then flag = -1 j = j - 1 endif endif if flag = -1 then j = j - 1 if j = 0 then j = j + random flag = 1 endif endif tradeon = 0 if opendate >= startdate then if barindex mod qty = 0 or barindex mod qty = j then tradeon = 1 endif endif

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

  //Robustness Tester - Random //By Vonasi //20190909   //StartDate = 20000101 //Qty = 10 //Random = 3   once j = 0 once flag = 1   if flag = 1 then j = j + 1 if j > qty then flag = -1 j = j - 1 endif endif   if flag = -1 then j = j - 1 if j = 0 then j = j + random flag = 1 endif endif   tradeon = 0 if opendate >= startdate then if barindex mod qty = 0 or barindex mod qty = j then tradeon = 1 endif endif

   

   

  10 users thanked author for this post.

  Brad, Elio_58, TempusFugit, Kovit, Gauvel, Bard, Marlon, PastaPesto, GraHal, Nicolas

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/10/2019 at 6:46 PM #107093
  Vonasi

  I thought of a good way to analyse all the test results from the various robustness testers that I have created.

  • Run a test
  • Organise the results in the optimizer window by ‘% of winners’ from low to high.
  • Drag the optimiser window results into Excel.
  • Create a line graph of the % of winners.
  • Re-organise the results in the optimiser window by ‘Avg gain per trade’ from low to high.
  • Drag the results again into Excel
  • Create a line graph of the ‘Avg gain per trade’

   

  If the lines on the charts you created are completely flat and level with no major spikes then you have a robust strategy. We should expect some inclination and maybe some ripples but we are looking for as flat a line as possible.

  It might be a good idea to create an Excel sheet that you can quickly call up and that already has the charts created. You then can just drop your results into two areas on the sheet and get automatic charts. This means that you can also fix the y axis max and min values so as to ensure that when you look at any line you are comparing it like for like with previous lines that you have looked at.

  Attached are two Robustness Test Random charts of a strategy that I have been working on. Settings were qty = 10 and random = 5 giving 45 tests. Not too bad – pretty flat lines but the avg gain per trade is slightly steep. Still pretty robust and profitable on all tests.

  1 user thanked author for this post.

  Gauvel

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/10/2019 at 9:22 PM #107127
  Magifina

  Hello Vonasi,
  I’m not sure to have understood all your robustness code. If the line is flat the strategy is robust. Did you mean that the curve and the values must have a constant and regular evolution?

  On the left excel graph the curve is steeper under 65% of winning trade. Does that mean that our strategy will be robust if we choose variables that return more than 65% of winning trades?

  It could help us if you can show us a good graphic and a bad one in order for us to eliminate robustless strategies.

  

  Magifina

  Participant

  Topics: 2

  Replies: 41

  Been thanked: 12 times
  09/10/2019 at 10:11 PM #107129
  Vonasi

  We are testing with fixed variables – we test our final strategy with the fixed variable values that we think work best after optimising.

  To understand how this tests the robustness you have to consider that when we create an equity curve it has a fixed starting point and we start looking for trades from that starting point. If after 3 bars we find a trade and open a position and our strategy keeps it open for 10 bars and does not allow any new trades to open during that time even if there are opportunities then how do we know what the equity curve would have looked like if we’d started at bar 4 and our first trade had been one of those missed opportunities?

  All the robustness test does is randomly choose points to allow trades to be started if our strategy wants to. By running many tests of random trades we can compare each test and if we have a 100% robust strategy then it will have exactly the same win% and exactly the same avg gain per trade in every random test. This is highly unlikely but that is the perfect strategy robustness test result.

  So when you organise the avg gain and % win from low to high and plot it in Excel you can see how similar all the test results are at a glance. If the win rate on the left is 0% and on the right 100% and even worse the line is very wobbly between the two then that is a complete fail. However if it is 65% on the left and 70% on the right then that is a pretty good strategy. The same with the average gain. If our average gain is zero on the left and 100 on the right then that is a complete and utter fail. If it is 3 on the left and 5 on the right or even 86 on the left and 90 on the right then that is pretty good and it is a robust strategy.

  Attached is an image of what I would class as a fail. A massive difference in average gain and in percentage of winners. Plus a bit wobbly and some tests even made a loss.

  Checking each equity curve is also useful as they can tell you a lot. A good strong strategy would allow you to turn it off and then back on again randomly and still make money. When you look at the equity curves you have to imagine that that is what your results would look like if you randomly traded your strategy and ask yourself if you would be happy with all the equity curves. I’ve attached the best and the worst test result equity curves. Depending upon when you happen to trade you could get either of them or anything in between.

  As with all strategy testing we should also keep in mind that we want a lot of trades in our tests and so if we use settings in the tester that mean some tests open hardly any trades then those results don’t really tell us anything so we should keep one eye on trade quantity in the results.

  2 users thanked author for this post.

  GraHal, Magifina

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/11/2019 at 1:11 PM #107190
  GraHal

  Vonasi wrote:

  Run a test

  Organise the results in the optimizer window by ‘% of winners’ from low to high.

  Forgive the daft question, but aren’t you setting up excel with the results from all sets of variable values?  Most of which I likely would not consider going forward with in a System.

  I scan read sections of this Thread … hence my question may be off the mark a bit?

  I was under the impression that – backtested trade results from the final selected set of variable values – would be what we need to be putting under scrutiny of robustness??

  

  GraHal

  Participant

  Topics: 211

  Replies: 8760

  Been thanked: 2176 times
  09/11/2019 at 1:35 PM #107192
  Vonasi

  For the Robustness Tester Random this is the procedure:

  • First of all you create your strategy and then you optimise the life out of it until you have found a set of variables that you think work. You then fix these values in the code or the optimiser.
  • You then add the robustness tester to your code and add the condition TRADEON to when positions are opened (not accumulating positions but just the first entry or any reversal entries).
  • You then add two variables to your optimiser ‘QTY’ and ‘RANDOM’. QTY should be 2 to whatever and RANDOM 1 to whatever but not more than QTY. The optimiser will tell you how many tests this will run.
  • You then back test your strategy and the optimiser results window will show you the results of all the tests.
  • Check the trade quantites as if they are low then you should reduce the number of tests by adjusting QTY and RANDOM
  • Click on each test to compare the equity curves – all similar is good.
  • Click on the avg gain per trade tab in the optimiser window to organise the results in low to high and drag the results into Excel.
  • Click on the % of winners tab in the optimiser window to organise the results in low to high and also drag the results into Excel.
  • Create two line graphs in Excel of the avg gain per trade (of the first dragged data) and % of winning trades (of the second dragged data).
  • Level straight lines with not much change in value from end to end are what you are looking for.

   

  As with all analysis the final decision as to whether you are happy with the results or not is entirely down to you…. and stop scan reading GraHal!

  7 users thanked author for this post.

  Kovit, TempusFugit, Francesco, Wilko, Mike, O-jay8, GraHal

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  09/11/2019 at 2:03 PM #107196
  GraHal

  Great! thank you Vonasi … above is just what we needed, an idiots guide / check list … even I can follow this now! 🙂

  

  GraHal

  Participant

  Topics: 211

  Replies: 8760

  Been thanked: 2176 times
  09/11/2019 at 2:35 PM #107201
  GraHal

  If every single combo comes back the same result have I done something wrong or is it a superlative System or an oddball System?? 🙂

  We enter TRADEON as below … Yes?

  If Condition and TRADEON Then Buy at Market Endif

  1 2 3

  If Condition and TRADEON Then Buy at Market Endif

  EDIT/PS

  Schoolboy error … I’d not entered Qty as Qty and RANDOM as RANDOMS in the code! 🙂

  That’s the trouble with idiot’s guides … for the guidance of wisemen and the slavish adherence of idiots!!?? 🙂

  I redeemed myself just in time!!!!! 🙂

  

  GraHal

  Participant

  Topics: 211

  Replies: 8760

  Been thanked: 2176 times
  09/11/2019 at 2:44 PM #107203
  Vonasi

  Yes that is how you use TRADEON.

  Maybe post the whole code so I can see if there is a mistake? Time frame and market would be useful too.

  You are stress testing the robustness tester GraHal!

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
• Author
  Posts