Array/Bubble sort in strategy doesn’t work, but works as an indicator

• 09/09/2024 at 9:54 AM #237343
  robertogozzi

  @Finning

  I think it’s easier for me to code it differently, will you please post exactly what you want to achieve in your strategy?

   

  1 user thanked author for this post.

  Finning

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
  09/10/2024 at 3:56 PM #237468
  Finning

  Hi Rob, yes, thank you, will do. Will get it all together and will post. Might be a day or so as I’m a bit busy, but will try and get done ASAP. Many thanks!

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  09/19/2024 at 11:58 AM #237815
  Finning

  Hi Rob,

  please find below.

  This code won’t work because of the stated problem, but it gives the idea of what it is I’m trying to achieve.

  It works as an indicator, just not with backtesting, because the backtest doesn’t like the summing of the array from what I can see.

  I’m trying to find which period Bollinger gives the most number of tradecandles, then trade that period.

  I would eventually like to make it so it interrogates a set number of candles in for a lookback (eg – 300), rather than finding the tradecandle period over all of the data.

  defparam cumulateorders = false ////////////////////////////////////////////////////////////////// // Clearing the temp arrays so they give the correct answer every bar unset ($datemp) unset ($latemp) ////////////////////////////////////////////////////////////////// // fill up the array with a sum count of tradecandles ($da[k]) and a corresponding Bollinger period ($la[k]) for k = 1 to 40 averageline = average[k](close) a = Average[k](Close) StdDeviation = STD[k](Close) Bsup = a + 2 * StdDeviation Binf = a - 2 * StdDeviation tradecandle = low>(a+((Bsup-a)*0.75)) if tradecandle then newcount = 1 else newcount=0 endif once $da[k]=0 $da[k] = newcount+$da[k] $la[k] = k next MaxElements = lastset($da) ////////////////////////////////////////////////////////////////// // (Temp arrays to be created for use for Bubble sort, so original summation arrays are not damaged) for k = 1 to 40 $datemp[k] = $da[k] $latemp[k] =$la[k] next ////////////////////////////////////////////////////////////////// // (Bubble Sort of temp arrays) FOR i = 0 TO MaxElements -1 FOR j = 0 TO MaxElements - i IF $datemp[j] > $datemp[j + 1] THEN // swap data temp = $datemp[j] $datemp[j] = $datemp[j + 1] $datemp[j + 1] = temp // swap labels temp = $latemp[j] $latemp[j] = $latemp[j + 1] $latemp[j + 1] = temp ENDIF NEXT NEXT ////////////////////////////////////////////////////////////////// // remove duplicates in temp arrays by comparing the current element to the next one (creating 2 new arrays) NewMaxElements = 0 FOR i = 0 TO MaxElements IF ($datemp[i] <> $datemp[i + 1]) OR (i = MaxElements) THEN $dx[NewMaxElements] = $datemp[i] //save datum to new array, when different $lx[NewMaxElements] = $latemp[i] //save label, too if (i = MaxElements) then break endif NewMaxElements = NewMaxElements + 1 ENDIF NEXT ////////////////////////////////////////////////////////////////// // This is the value of the period that gives the max number of tradecandles from ALL of the data yyy = $lx[NewMaxElements] ////////////////////////////////////////////////////////////////// //Using the found best value yyy that gives the most number of tradecandles for use in trading, re-calc the Bollinger variables to find tradecandle conditions. av = Average[yyy](Close) StdDeviation = STD[yyy](Close) Bup = av + 2 * StdDeviation Bdown = av - 2 * StdDeviation tradecandle2 = low>(av+((Bup-av)*0.75)) //If there a tradecandle based on the right period yyy, and other conditions, then buy If not longonmarket and tradecandle2 and av>av[1] and Bup>Bup[1] and Bdown>Bdown[1] then buy 1 contract at market endif //Just a simple stop for an exit if longonmarket then sell at av stop endif graph yyy as "newmaxelements"

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110

  defparam cumulateorders = false     ////////////////////////////////////////////////////////////////// // Clearing the temp arrays so they give the correct answer every bar unset ($datemp) unset ($latemp)   ////////////////////////////////////////////////////////////////// // fill up the array with a sum count of tradecandles ($da[k]) and a corresponding Bollinger period ($la[k])   for k = 1 to 40   averageline = average[k](close)   a = Average[k](Close) StdDeviation = STD[k](Close) Bsup = a + 2 * StdDeviation Binf = a - 2 * StdDeviation tradecandle = low>(a+((Bsup-a)*0.75)) if tradecandle then   newcount = 1 else newcount=0   endif   once $da[k]=0   $da[k] = newcount+$da[k]   $la[k] = k   next   MaxElements = lastset($da)   ////////////////////////////////////////////////////////////////// // (Temp arrays to be created for use for Bubble sort, so original summation arrays are not damaged)   for k = 1 to 40 $datemp[k] = $da[k] $latemp[k] =$la[k] next   ////////////////////////////////////////////////////////////////// // (Bubble Sort of temp arrays) FOR i = 0 TO MaxElements -1 FOR j = 0 TO MaxElements - i IF $datemp[j] > $datemp[j + 1] THEN // swap data temp   = $datemp[j] $datemp[j] = $datemp[j + 1] $datemp[j + 1] = temp // swap labels temp   = $latemp[j] $latemp[j] = $latemp[j + 1] $latemp[j + 1] = temp ENDIF NEXT NEXT   ////////////////////////////////////////////////////////////////// // remove duplicates in temp arrays by comparing the current element to the next one (creating 2 new arrays) NewMaxElements = 0 FOR i = 0 TO MaxElements IF ($datemp[i] <> $datemp[i + 1]) OR (i = MaxElements) THEN $dx[NewMaxElements] = $datemp[i]          //save datum to new array, when different $lx[NewMaxElements] = $latemp[i]          //save label, too if (i = MaxElements) then break endif NewMaxElements      = NewMaxElements + 1 ENDIF NEXT ////////////////////////////////////////////////////////////////// // This is the value of the period that gives the max number of tradecandles from ALL of the data   yyy = $lx[NewMaxElements]   ////////////////////////////////////////////////////////////////// //Using the found best value yyy that gives the most number of tradecandles for use in trading, re-calc the Bollinger variables to find tradecandle conditions.   av = Average[yyy](Close) StdDeviation = STD[yyy](Close) Bup = av + 2 * StdDeviation Bdown = av - 2 * StdDeviation tradecandle2 = low>(av+((Bup-av)*0.75))   //If there a tradecandle based on the right period yyy, and other conditions, then buy If not longonmarket and tradecandle2 and av>av[1] and Bup>Bup[1] and Bdown>Bdown[1] then buy 1 contract at market endif   //Just a simple stop for an exit if longonmarket then sell at av stop endif               graph yyy as "newmaxelements"

  Thanks,

  Finning

   

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  09/20/2024 at 4:27 PM #237890
  robertogozzi

  As to “It works as an indicator, just not with backtesting, because the backtest doesn’t like the summing of the array from what I can see.” I’d say it’s the other way round, as I stated in one of my posts, as indicators will sum up the same condition multiple times when using arrays, while the backtest only does that only once per candle (as it doesn’t run live)  at the closing of each bar, when the condition is tallied  if it has occurred.

  In any case, can you explain exactly what you want to tally?

   

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
  09/21/2024 at 11:27 AM #237902
  Finning

  Hi Rob,

  For within set bounds of potential period [k] values, say 1 to 40, I want to find which period gives the most occurrences of the term tradecandle, if tradecandle is defined by the below:

  a = Average[k](Close) StdDeviation = STD[k](Close) Bsup = a + 2 * StdDeviation Binf = a - 2 * StdDeviation tradecandle = low>(a+((Bsup-a)*0.75))

  1 2 3 4 5 6

  a = Average[k](Close) StdDeviation = STD[k](Close) Bsup = a + 2 * StdDeviation Binf = a - 2 * StdDeviation tradecandle = low>(a+((Bsup-a)*0.75))

  And I need to do this in a backtest/strategy/trading, that is my key problem. I already have this working as an indicator, as shown previously.

  Which period [k], from 1 to 40, gives the highest number (sum) of the term tradecandle, over a barindex/total lookback period, or a defined rolling lookback period, if that is possible.

  Many thanks

   

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  09/24/2024 at 3:00 PM #237972
  druby

  Hi Finning…

  After looking at the Array/tick problem and following your attempt to dynamically optimise the period in the code, I thought, what does optimisation look like.

  I set about making an indicator to visualise the process and try to make the code work for indicator and back-test so some comparison could be achieved.

  Though you could hard code a load of variables, arrays do make it easier.

  The drawing tools for indicator are good, but are limited by the use of graph in back-test.

  Additionally Roberto’s comment above about back-test not being live, meaning the tick problem, isn’t a problem in back-test, writing a back-test specific code which won’t work as a indicator seams to rub me up the wrong way. What I see in a indicator,  I expect to see in a back test etc,  or else I want to know why so I can make a  judgment.

  Further, if you have an indicator, which match as a back-test version then you have a template to write a back-test specific version where you can compare results with.

   

  Though the code is still a work-in-progress, the image shows both indicator (top)and back-test (bottom) versions, with a simulated entry/stop and the position frame in the middle.

  Looking at the indicator plot…

  The white lines show when the tradeCandle condition were true for each bar and sample period and are labelled with the yellow number list.

  The two red vertical lines, bookend the current lookback range ‘n’ and the blue numbers are the count of tradeCandle within it.

  The optimised period is denoted with an red arrow pointing to one of the ‘blue’ count numbers.

  The optimised period is the highest count with lowest period found in the range of sample periods over the lookback range.

  The blue dotted line is the historic path of the optimised period value and the pink was its count

  At the left edge are two green vertical lines, this represent the range of bars that first met the lookback distance, labelled START.

  The green and red line represent a simulated entry and stop, and the below, the position panel some what agree.

  Not yet sure where the difference comes from yet.

  Below that the back-test version, follows the same colour convention with graph etc.

  Both plots appear to show similar data. yeh!

  I did notice that if the there was not  enough bars in the chart and/ or pre-loaded-bars used, the plots could deviate early on in  the plot, but eventually aligned.

  To get some alignment with preload bars, it’s better to compare  an indicator plot with equivalent bars with the back-test plot, this needs a copy chart with more bars.

   

  In the code …

  ‘n’ is the lookback and ‘k’ s the upper period range.

  ‘kk’  alters the lower period of the range. I put this in so the range could be manually alter if the optimise period was at the top of the range.

  However I realised that if you did that, on the way up, if tradeCandles dramatically dropped, the the optimised value would be from the current settings

  and not from the bottom of original range. This part needs some work.

  anyway, see what you make of it.

  druby

   

  1 user thanked author for this post.

  PeterSt

  

  druby

  Participant

  Topics: 1

  Replies: 355

  Been thanked: 269 times
  09/26/2024 at 4:46 PM #238086
  robertogozzi

  This is the indicator:

  DEFPARAM DrawOnLastBarOnly = true // check 6 bars, from bar 10 to 15 ONCE N1 = 10 ONCE N2 = 15 ONCE Diff = (N2 - N1) + 1 // once x = 0 // iniztialize the array on the very first bar IF BarIndex = 0 THEN FOR j = N1 TO N2 $a[j] = 0 NEXT ENDIF // tally all due occurrences and store them in the array and print it FOR k = N1 TO N2 Multi = (k/1) Offset = highest[2](high) + ((range / 2) * Multi) a = Average[k](Close) sDev = STD[k](Close) Bsup = a + 2 * sDev Binf = a - 2 * sDev candle = low > (a+((Bsup-a)*0.75)) $a[k] = $a[k] + candle y = $a[k] DrawText("$a[#k#]=#y#",BarIndex,Offset) NEXT return

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

  DEFPARAM DrawOnLastBarOnly = true // check 6 bars, from bar 10 to 15 ONCE N1   = 10 ONCE N2   = 15 ONCE Diff = (N2 - N1) + 1 // once x    = 0 // iniztialize the array on the very first bar IF BarIndex = 0 THEN    FOR j = N1 TO N2       $a[j] = 0    NEXT ENDIF // tally all due occurrences and store them in the array and print it FOR k = N1 TO N2    Multi  = (k/1)    Offset = highest[2](high) + ((range / 2) * Multi)    a      = Average[k](Close)    sDev   = STD[k](Close)    Bsup   = a + 2 * sDev    Binf   = a - 2 * sDev    candle = low  > (a+((Bsup-a)*0.75))    $a[k]  = $a[k] + candle    y      = $a[k]    DrawText("$a[#k#]=#y#",BarIndex,Offset) NEXT return

  this is the strategy to backtest:

  DEFPARAM PreLoadBars = 0 ONCE N1 = 10 ONCE N2 = 15 ONCE Diff = (N2 - N1) + 1 once x = 0 IF BarIndex = 0 THEN FOR j = N1 TO N2 $a[j] = 0 NEXT ENDIF FOR k = N1 TO N2 Multi = (k/1) Offset = highest[2](high) + ((range / 2) * Multi) a = Average[k](Close) sDev = STD[k](Close) Bsup = a + 2 * sDev Binf = a - 2 * sDev candle = low > (a+((Bsup-a)*0.75)) $a[k] = $a[k] + candle y = $a[k] NEXT buy at -close limit //at least one BUY instruction is required (evan doing nothing) graph $a[10] graph $a[11] graph $a[12] graph $a[13] graph $a[14] graph $a[15]

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

  DEFPARAM PreLoadBars = 0 ONCE N1   = 10 ONCE N2   = 15 ONCE Diff = (N2 - N1) + 1 once x    = 0 IF BarIndex = 0 THEN    FOR j = N1 TO N2       $a[j] = 0    NEXT ENDIF FOR k = N1 TO N2    Multi  = (k/1)    Offset = highest[2](high) + ((range / 2) * Multi)    a      = Average[k](Close)    sDev   = STD[k](Close)    Bsup   = a + 2 * sDev    Binf   = a - 2 * sDev    candle = low > (a+((Bsup-a)*0.75))    $a[k]  = $a[k] + candle    y      = $a[k] NEXT buy at -close limit  //at least one BUY instruction is required (evan doing nothing) graph $a[10] graph $a[11] graph $a[12] graph $a[13] graph $a[14] graph $a[15]

  both retain the same data.

   

   

  1 user thanked author for this post.

  druby

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
  09/27/2024 at 11:32 AM #238138
  druby

  Hello Roberto, thankyou for posting the code however there is an issue with the indicator version.

  When I tested the code, the values were indeed correct when the chart was built, but when ‘ticks’ started coming in, the count incremented on every tick the ‘candle’ condition was true.

  On line 23 the array elements appears on both sides of the ‘=’ sign, so the array elements are going to be updated on every tick and use the updated value from previous tick’ed result, till bar end.

  As far as I’m aware, this was the original issue, and the count was for, counting the bars, like in the back-test version, but the tick updates corrupted the count.

   

  From your earlier comment, regarding that the back-test does not run live, I take that to mean, it only uses the closed bar data and hence is always a bar behind.

  So, I suppose the simplest way to avoid the array tick update scenario and align with a back-test version is to only use fixed data from previous closed bar(s) that will not change.

  However, I have not tried that and somehow, I don’t think it will be that simple.

   

  From your back-test version… it count the total true ‘candle’ conditions,  for the sample range of periods N1 to N2, over the full range of bars.

  I have been following Finning’s posts and in edition to that, he was working towards finding the,  highest condition count, with the lower period value, over an ‘n’ range of bars.

  Effectively to dynamically find the optimised value of the ‘k’ periods in relation to the true condition count over ‘n’ bars.

   

  Regarding orders and ticks with respect to arrays, if lets say, indicators run live, and back-test do not, then what does the Automatic Trading system do or follow.

  Regards

   

  

  druby

  Participant

  Topics: 1

  Replies: 355

  Been thanked: 269 times
  09/27/2024 at 2:38 PM #238149
  robertogozzi

  Yes, you can skip the current bar, which is barely the same as the following code that only updates at the closing of each bar, ignoring incoming ticks after the first one.

  This version uses a 2-element array to tell ticks belonging to different candles (see also the attached pic):

  DEFPARAM DrawOnLastBarOnly = true // check 6 bars, from bar 0 to 5 ONCE N1 = 0 ONCE N2 = 5 ONCE Diff = (N2 - N1) + 1 // iniztialize the array on the very first bar IF LastSet($a) < 0 THEN FOR j = N1 TO N2 $a[j] = 0 NEXT ENDIF // ONCE $myTime[0] = 0 // use this 2-element array to tell when ticks belong to different candles ONCE $myTime[1] = 0 // use this 2-element array to tell when ticks belong to different candles // set element [0] = current OpenTime $myTime[0] = OpenTime // // esecute this block only on the very first tick of each new candle IF $myTime[1] <> $myTime[0] THEN // tally all due occurrences and store them in the array and print it FOR k = N1 TO N2 a = Average[k](Close) sDev = STD[k](Close) Bsup = a + 2 * sDev Binf = a - 2 * sDev candle = close crosses over average[3] //low > (a+((Bsup-a)*0.75)) $a[k] = $a[k] + candle NEXT // set element [1] = element [0] so that this block is skipped until a new candle opens $myTime[1] = $myTime[0] ENDIF // plot data on the chart every tick FOR k = N1 TO N2 Multi = (k/1) Offset = highest[2](high) + ((range / 2) * Multi) y = $a[k] DrawText("$a[#k#]=#y#",BarIndex,Offset) NEXT return

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

  DEFPARAM DrawOnLastBarOnly = true // check 6 bars, from bar 0 to 5 ONCE N1   = 0 ONCE N2   = 5 ONCE Diff = (N2 - N1) + 1 // iniztialize the array on the very first bar IF LastSet($a) < 0 THEN    FOR j = N1 TO N2       $a[j] = 0    NEXT ENDIF // ONCE $myTime[0] = 0    // use this 2-element array to tell when ticks belong to different candles ONCE $myTime[1] = 0    // use this 2-element array to tell when ticks belong to different candles // set element [0] = current OpenTime $myTime[0] = OpenTime // // esecute this block only on the very first tick of each new candle IF $myTime[1] <> $myTime[0] THEN    // tally all due occurrences and store them in the array and print it    FOR k = N1 TO N2       a      = Average[k](Close)       sDev   = STD[k](Close)       Bsup   = a + 2 * sDev       Binf   = a - 2 * sDev       candle = close crosses over average[3] //low  > (a+((Bsup-a)*0.75))       $a[k]  = $a[k] + candle    NEXT    // set element [1] = element [0] so that this block is skipped until a new candle opens    $myTime[1] = $myTime[0] ENDIF // plot data on the chart every tick FOR k = N1 TO N2    Multi  = (k/1)    Offset = highest[2](high) + ((range / 2) * Multi)    y      = $a[k]    DrawText("$a[#k#]=#y#",BarIndex,Offset) NEXT return

  As I said above, It’s not much different than using a signal from the last closed candle, identified by index [1]. I just wanted to find a way not to let arrays be updated each tick in indicators

   

   

  2 users thanked author for this post.

  Finning, druby

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
  09/29/2024 at 1:19 PM #238234
  druby

  Hi Roberto, thanks for your reply.

  When I tested the code, the array counts were different to the working back-test version, if fact the values were all the same.
  After a while I realised that Line 26 had a different condition statement than originally coded. I expect the alien statement was part of a test. When reverted back to original, the counts resembled the back-test.

  The $mytime[ ] additional code appeared to do its job and isolate the code block from multiple ticks.
  However, after a while the counts started deviating from the back-test. I put this down mainly to the ‘Candle’ condition being captured at the first bar tick and not at bar end. Additionally, the values of Average, STD, Close and Low would theoretically be off with respects to values at the bar end.

  If the condition is true or false at first tick and this is mirrored at bar end, (0=0, 1=1), then not much harm done.
  However, its when there both different between first tick and bar end, (0=1, 1=0) that appears to cause the issue. This could results in the count being incremented when it shouldn’t and not incremented when it should, with respect to an end of bar value.

  In the images, Image1 shows results which fall within the chart build phase, and Image2 in the following live tick data. Image3 is the same area as image2, but after a re-build and when it falls in the chart build area.

  Bubble v12.0 represent the true/false condition relative to the period value in violet. The 6 horizontal lines period values match the y scale. The period+0.75 just represents when their true .
  MyIndicator(197) is the latest version of your indicator . Bottom up [10,11,13,12,14,15]
  MySystem(57) : Variables, is your original back-test version. Bottom up[10,11,13,12,14,15]

  Comparing the results of the indicator and back-test, in (Image2) I see addition increments in the count when they shouldn’t, see red and green arrows. These can seem to happen before and after the true condition. The green arrows represent when the bar condition is false and the first tick made it true. And the red arrows represent when the bar condition is false but the first tick made it true.

  Image1,  appears to represent correctly, probably due to the closed OHLC data used.
  Image2, the counts deviate, see y scale, between new indicator and back-test versions. This I think because of the first tick collection point as explained above.
  Image3, shows how the Image2 displays when it falls in the area of the build phase. The counts have changed and align, and aligned with the true conditions. Again, this due to the closed OHLC data used in the build.

  Over to you, I wonder what your next move is!
  Regards

  1 user thanked author for this post.

  Finning

  

  druby

  Participant

  Topics: 1

  Replies: 355

  Been thanked: 269 times
  09/29/2024 at 1:25 PM #238236
  druby

  For some reason the, platform dumped Image 1 and 2 when posted or when using Edit function.

  

  druby

  Participant

  Topics: 1

  Replies: 355

  Been thanked: 269 times
  09/29/2024 at 1:28 PM #238238
  druby

  oops….

  

  druby

  Participant

  Topics: 1

  Replies: 355

  Been thanked: 269 times
  09/29/2024 at 3:35 PM #238244
  robertogozzi

  druby However, after a while the counts started deviating from the back-test

  It is probably due to the fact that the backtest once it’s finished doesn’t update its data any longer, while the indicator still does!

   

  1 user thanked author for this post.

  Finning

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
  09/30/2024 at 11:03 AM #238286
  Finning

  Hi all, sorry for my absence, something I need to get better at, despite my schedule.

  Roberto and DRuby, thanks for the great ideas.

  One question with all of this, could the above tick problem be bypassed if we were just looking at the last closed bar datas [1]?

  I think I mentioned the [1] idea before to avoid the tick data problem, and it would well and truly do what I need, if it makes things any easier?

  I am downloading and fully digesting the codes above now.

  Finning

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  09/30/2024 at 11:15 AM #238288
  robertogozzi

  Yes, you can surely refer the prior closed bar, i.e. [1], which is almost the same as my last code to prevent arrays from repeatdly being updated each new incoming tick. Indeed I only allow array elements to be updated on the very first tick of each bar, which it’s almost the same as using the prior bar.

   

  

  robertogozzi

  Moderator

  Topics: 69

  Replies: 17142

  Been thanked: 4032 times
• Author
  Posts