I think it’s due to the different timing strategies are run, compared to indicators and screeners.
Strategies are NOT run at any incoming tick, but only at the closing of a bar.
Because when live, arrays are updated each tick, so when, say, on a 1-hour TF a condition (such as a crossover) occurs multiple times an array (differently from variables) are updated multiple times. A common variable may change temporarily while the candle is forming then it’s definitly set according to the condition on the last tick, while an array is updated each time the condition occurs, so a crossover may occur many times within trhe same 1-hour or 1-minute candle.
Since strategies are only run when a bar closes, there’s not diference between variables and arrays as to historicization.
Try this indicator on a 1-minute timeframe, you will see the variable being updated only once on a crossover, while the array is updated multiple times whenver there’s a crossover, then a pullback below, then agin a crossover, etc…
But if I tried the above in a strategy it bugged out again, and DID NOT give the right answer. I thought this would have beat the problem of the tick updates as the last bar should have been finalised and closed?
Hi Rob, no worries, thanks for still looking at this.
I’ve been playing around with it, and I haven’t had much luck either.
There does seem to be a problem though, where the array sum doesn’t seem to work, and it returns nothing.
For example, the following code works in a strategy/backtest:
1
2
3
4
5
6
7
8
9
Fork=0todayofweek
once$da[k]=0
$da[k]=k
next
graph$da[k]astest
But if you try and sum this up,
1
2
3
4
5
6
7
8
9
Fork=0todayofweek
once$da[k]=0
$da[k]=k+$da[k]
next
graph$da[k]astest
nothing is returned, just a blank graph screen.
If you run the above 2nd sum code as an indicator, it works and returns a result. So I don’t know why the above doesn’t work in a strategy/backtest then??
I run this code as an indicator which prints the right number of expected lines, 7, since its sat today, 0 – 6.
However the value returned from the return line is 7, maybe its a count of the loop iterations.
To me your two sentences don’t read consistently. This could be a language issue (always on my side 🙂 ).
So just to be sure : i should return as 7 here when it is Saturday; this is because the variable always increased first and then exits the loop. Thus, for i = o to 6 and i will be 7 after the loop.
For example, the following code works in a strategy/backtest:
1
2
3
4
5
6
7
8
9
For k =0todayofweek
once $da[k]=0
$da[k]= k
next
graph $da[k]as test
But if you try and sum this up,
1
2
3
4
5
6
7
8
9
For k =0todayofweek
once $da[k]=0
$da[k]= k+$da[k]
next
graph $da[k]as test
nothing is returned, just a blank graph screen.
If you run the above 2nd sum code as an indicator, it works and returns a result. So I don’t know why the above doesn’t work in a strategy/backtest then??
This problem certainly isn’t helping.
Thanks,
Finning.
Already in the first snippet I would deem the Once in the loop an erroneous line of code. This is already so because of the Once being in the loop which is “not done” anyway, plus that your approach to $da[k] = k on line 5 will always overrule your line 3. In other words, with this code you can’t prove anything, because the code is wrong.
And what you can’t do in your first snippet, you also can’t do in your seconds snippet – there too you use the Once in a loop. That this does not work out on your line 5 in that snippet, looks completely normal to me.
I may combine this all with using Once in the Indicator code in the first place, which I again would never do because the logic of that is wrong (or at least feels wrong to me). This is because an indictor can be called from within strategy code (with CALL) and any variable declared with Once in the strategy code, can also be used in the Indicator. Try to see through here that the strategy code is first and the called indicator is second. Thus, the strategy code initiates (initializes) and the Indicator may use. Now, what you imply here (never mind you don’t use it) is the other way around : that the indicator (which is thus in order later than the virtually calling strategy) declares the variable which next the strategy can use ? … No. I would not even begin to think of this or try it.
Lastly on this subject, an array element (like $da[0] or $da[1]) can not be declared with once, because arrays and their elements are persistent anyway. This is harder to explain (it requires upside down reasoning), but PRT would just not support it, although you won’t notice that. Thus, what can be done with a normal variable, can not be done with an array element and this is because normal variables require special treatment internally, because they are all contained in an array itself (often called a “data series”) so that you can address them with [0] (last bar), [1] (one but last bar) and [2], etc. With array elements this can’t be done hence their storage is completely different.
Part of this required upside down reasoning (which I clearly can’t do) is that a not “declared” variable, will still have an initial value of 0 as long as they are given a value in the code somewhere. With an array elements this clearly does not exist and not-addressed (written to) elements remain undefined.
Side note (or not) : the way PRT deals with not-declared variables is so wrong, that I personally can not dedicate logic to it and explain. Also, the way PRT deals with arrays is so extraordinary odd, that here too no explanations can be given to any seen behavior, because it is again wrong what PRT does. Many people may think arrays are comfortable to use, but I still did not find even one application for them, because I can not apply my normal coding logic to them (there is always something working out differently than what I expect, hence I always will apply errors in the logic).
I know, this post won’t be much useful. The only thing useful should be that what you apply for examples will never lead to decent conclusions because the examples are (both) wrong. So try to make that right and then continue ?
(I forgot to check my last post for typo’s – sorry for that)
Something else, and time to say it after the existence of this topic which I try to follow somewhat :
I would never ever begin to apply code with arrays to both strategy code and indicator code equally. Why ? well, because indicator code deals totally different with arrays at the data level. In itself this was already mentioned in the thread somewhere (or at least it was implied) but the data level in the indicator is at “tick level” while in the strategy it is at “the bar level”. This, while the indicator seemingly returns at the fixed bar level in the Return command. Notice the seemingly, because for the data this is not true at all. Thus for what is returned, Yes, but for what it “is”, No – not at all.
Try to envision :
You sort the array in the indicator. This is done per tick. Thus, each tick adds new data and in each tick you sort the array as well (the indicator is called per tick – your indicator code is called per tick). So notice please that per second 10000 ticks can be passed at volatile times for e.q. Nasdaq. This will already go wrong technically, because nothing will be able to sort the array 10000 times per second, while the array also grows with 10000 elements per second (assuming you might do this). Please try to grasp this carefully. Thus, your indicator code may be called more often than that it has time to process all what you give it to process. This will make the result in the data (the array itself) just rubbish (read : corrupt).
Then you sort the array in strategy code. Well, to begin with, you won’t be able to let the array contain the tick data, unless you add that in the Indicator code. This latter is doable, and the sorting would be fine too, as long as you realize that you now sort per each second (if the TF is 1 second) or each minute (TF is 1 minute) etc.
The both above are so much apples and oranges that (once again) I would not even start to sort out (pun intended)) why the behavior of both is different, or worse : to make it behave equally. It can’t.
Add to this that you can’t even visualize anything properly, because the indicator again outputs (per Return) at the bar (-time) level.
What I would do in the situation that I want something to get going with the sort, is making either work for you in an acceptable way, but forget about making both indicator and strategy being equal. All you could IMO attempt in the indicator, is apply the sort only once per your TF of the strategy. This means under the regime “once per bar” only (I forgot the command for that). You may run into anomalies of being one indicator-bar behind (your (sorting) math is applied at the start of the indicator call, while under way the array receives new tick data elements) – and anyway it will be messy.
But why apply it in the indicator which is not what your goal is anyway ? thus, make it to work in the strategy and always keep in mind what happens in the indicator at the same time, *if* you still call the indicator. This will be more than messy. And thus : don’t run the indicator and now you won’t have tick data, which may be fine for you (I did not follow the subject to the letter).
The moral of the latter is for me all over obvious from of your post #1 : the indicator will mangle with your array data which you try to deal with in the strategy – and it does that behind your back. Thus :
Whatever you do in the strategy – say it is called each minute – right under your nose / behind your back the indicator will add/change elements 10s of 1000s of times. This will happen completely in parallel to your strategy code being executed this one time in the minute. I could also say : if your strategy code takes 10ms to run, then 100 ticks could have passed in the indicator, hence your indicator code has been called 100 times and changed the same array 100 times. And there will be no way I know of to freeze that. Of course, we could attest that the indicator – called per CALL and thus called once per minute, would make the array a “state” which you can deal with in the strategy code, but it makes no sense because you will be looking at 10ms of data while 10000 ticks per second times 60 for one minute occurred for real.
See ? this is again so much apples and oranges that nothing makes sense and once again I can not explain it. And so here too : I would not even start to think of making such a thing (like your sort). Unless … unless all bits and pieces are deeply understood. … Someone else may be able to do that, and then explain how to set up your sorting code so that it becomes reliable. Thus, this requires reasoning first and then of a type which is decent (which is not mine, really not).
On this occasion its probably my dumb writing style and working backwards from an hypothetical error situation, ‘undefined array element’.
My results, PRINT(k) from a backtest and RETURN k from a indicator, point to the loop iterator incrementing beyond the DAYOFWEEK range, however it could be misleading with the TO in the statement.
It appears that no additional loop code is executed after this last value of k is reached and hence no array element for that index.
After the loop, k is still in scope, and in this situation, K is +1 the range of the loop.
Your comment backs up this concept with,
variable always increased first and then exits the loop.
I wonder if the comment under this link was an indirectly reference to this situation.
Hi druby – The first day of the week is normally settable – I think I have seen this in PRT somewhere too. Or not, because I can not find it today. Fact is that this is not the same in all countries of the world. Sunday, Monday or even Thursday of what I recall. https://commons.wikimedia.org/wiki/File:First_Day_of_Week_World_Map.svg
No Thursday to be seen. Bure sure 4 different days.
Hi DRuby, thanks for having a look and investigating, and hi PeterSt, thanks for your detailed/thoughtful/insightful responses.
A big thing that I have left out about all of this, is why am I asking these questions, and what am I trying to achieve with all of this? I should have maybe started with that, but I didn’t want to clog the internet with a lot of words (no pun intended PeterSt, I appreciate your detailed responses in which you have expressed your thoughts clearly, from which I will now try and do the same), and to be expedient, I was just chasing the answer that I wanted to solve the problem that I have. Maybe the detailed explanation will help with the solving of the problem, as it appears I might have left out a couple of crucial ideas about what I am trying to achieve. I don’t have anything to hide, just wanted to cut to the chase, and didn’t want to put people in TLDR territory.
If you want the TLDR version – I’m trying to build the functionality of a single feature backtest engine that dynamically updates a single value in a strategy to help promote strategy longevity and performance.
Otherwise:
First of all, to clear the air, in the first post of this thread, I used crosses over of a moving average as my target of what I was trying to count, to show how it would work as an indicator, but not in a strategy. The 2nd example I did with the dayofweek, which was another example to show how I could get the indicator to work, but again I could not get it to work in a strategy, even if the dayofweek example I used wasn’t a very good one.
What is a trade to me?
A long trade to me is when the low of a bar, is higher than 1.5 standard deviations from the middle of a 2 standard deviation Bollinger band.
To me, this pattern/occurrence is what I would call a “trade candle”, and this is what I am trying to then trade when I see it. When these occur, with a couple of other small tests, there is a very good chance that I want to be long, lets just say.
A “trade candle” is a specific condition that I want to count, and the prevalence of this specific condition is modified by the changing of its associated period variable. The aim is to find the best value of the period variable to give the maximum number of observable “trade candles” in the lookback period I am investigating.
I am trying to find the best period that gives the highest number of my wanted “trade candles”.
Bear in mind, when I say period in relation to a function, for example, this is like saying:
1
average[period](close)
From this, over a given range of Bollinger bands – say from periods 1 to 40 – which period Bollinger band returns to me the highest number of “trade candles”, with a “trade candle” being when the low of a bar, is higher than 1.5 standard deviations from the middle of a 2 standard deviation Bollinger band.
So in the last 100 bars lets just say, a 5 period Bollinger band might give me 7 “trade candles”, a 15 period might give me 25 “trade candles”, and a 33 period might give me 35 “trade candles”, and a 37 period might give me 20 “trade candles”.
Therefore, I know that the trade (of buying when the low of a bar, is higher than 1.5 standard deviations from the middle of a 2 standard deviation Bollinger band) is best traded on a 33 period Bollinger band.
In essence, I know what I am trying to trade (“trade candle”), I am just trying to find where the trade is.
The idea being that Bollinger band period with the most “trade candles” in the past will have the best chance of having more “trade candles” seen in the future.
Yes, of course, this is a form of curve fitting. But it’s the place I’d like to start from. And why I want to make this process dynamic, so it will keep fitting the curve for me.
Ok, so with that explanation of what I am trying to achieve out of the road:
What exactly am I trying to do programming wise:
First of all, I need to start with the period range that I want to investigate.
K – is the period range that you want to investigate – so if K is 1 to 40, you at least have an inclination that the best period for the Bollinger band is somewhere in that range. Since computing/time is limited, you at least need to have an idea of where the value you are trying to find is, to maximize efficiency.
So, for each bar, I am running a loop – and the intent is, for the last bar just gone, I want to see, for Bollinger periods 1 to 40, was the trade candle condition present in the last bar for each Bollinger period tested.
From here, for periods 1 to 40, if the trade candle was present, its count is kept in the array, $da[k], with the index intended to be the period of the Bollinger band test.
Not only is the test result kept in $da[k], it is added up, so that over the test period (I say 100 bars, but I don’t actually have that machinery in place to limit that yet – which I need – and it runs/sums up the full length of a backtest instead) – it counts the total number of trade candles seen each bar, for the length of the lookback.
So $da[k], the index is the Bollinger band period (from 1 to 40 – so should only have 40 index places, 41 including 0), and the value for each index should be the sum of the occurrence of trade candles over the lookback period.
PeterSt, the reason there is “once $da[k]=0” stems from one of our recent previous conversations, in which you pointed out that to be able to sum an array, it first has to be set/initialised for it to work, because an array sum in PRT won’t work unless it is initially set to a value.
The reason there is an $la[k], is that there is no way to get the index number of $da if it was sorted, so $la[k] = k is used to record the value of the Bollinger band period, which is the key bit of information that we are trying to get out of all of this.
That’s what the bubble sort/duplicate removal is then needed for, to give the maximum number of $da, the highest number of trade candles, and $la, the corresponding period of the Bollinger band that gave the highest number of trade candles, the main piece of information that we are trying to find.
So without the bubble sort/duplication, the above statement would look like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
fork=1to40
a=Average[k](Close)
StdDeviation=STD[k](Close)
Bsup=a+2*StdDeviation
Binf=a-2*StdDeviation
tradecandle=low>(a+((Bsup-a)*0.75))
iftradecandlethen
newcount=1
else
newcount=0
endif
once$da[k]=0
$da[k]=newcount+$da[k]
$la[k]=k
next
And from the above, the bubble sort/duplicate removal tools could be added on if you wanted to sort to get the final result.
PeterSt, I read what you were saying about array updates, and ticks, and all of that. I would like to totally get away from all of that, as I fully agree if you are dealing with those sorts of technicalities, it is an absolute mess, of which you won’t gain any value, let alone meaningful data/understanding.
I also, absolutely, do not want to CALL the above, I want the above and bubble sort in the strategy, for proper integration, so the chance of problems is greatly reduced/eliminated.
What I want could easily work with bar old information ([1]), and still provide a great result. I was hoping that I could just do that, to get past all of the tick problems, and just update with fixed old values ([1]) from the end of the last bar, but it didn’t seem to work.
What I need essentially is to be able to run the above, but without all of the ticks. I only need *simple* end of bar information to be used and then stored in the array. I’m not trying to do HFT or anything of the sort with ticks, I’m looking at daily bars, and hourly at absolute best, and the information that is important to me is end of bar data.
The above works fine as an indicator, and it gives me the information that I want to use. I just need the information it makes to be made available for use in trading, without using CALL.
As mentioned, as far as I have narrowed it down, the problem seems to be with the array summation when used in a backtest. I’m still playing around with it, but that’s where the problem seems to be.
I am using a daily bar demo account as well if that makes a difference, and there is no option to turn on/off tick data.
I hope my explanation helps with why I am going to the fuss of all of this. I can see it being very useful, if I can get it to work.
And no, I don’t want to just manually run backtests, stop systems, enter new values into them, and restart them again…
To help us continually offer you the best experience on ProRealCode, we use cookies. By clicking on "Continue" you are agreeing to our use of them. You can also check our "privacy policy" page for more information.Continue
.
