Back testing error as you chack longer periods
Forums › ProRealTime English forum › ProOrder support › Back testing error as you chack longer periods
- This topic has 9 replies, 3 voices, and was last updated 1 year ago by JS.
-
-
08/05/2023 at 12:57 AM #218611
Hi All
I may be seeing somthing that I am causing but I think there is an error in the long term data. The way I got to this is I am trying to find the right settings for an indicator to cover the biggest possible scenarios.
So the way I found this was setting up an indicator on the 1 min time frame this 10 K units. Initial settings I was using as the start piont showed pretty much what I thought it would. So I tried longer periods and that is where the problem started, at 15 units all of a sudden no profit at all. Not somthing I did not expect but looking into it a bit more I found that although I expected the time frame outside the initial 10k settings to show a diferant set of winning to losing, I did expect to see somthing of a similar profile to the 10k profile. in other words at the end of the 15K data I should see the same thing in terms of winning and loosing for the last 6 days.
I then looked more closely at the last 6 days. So even if the time prior to the 6 days (about 10k units) at higher time frames the last 6 days should still present the same outcome on all the charts. The higher time frame should just be more data points but the last 6 days should all be the same on all time frames.
Its not and it gets worse the longer the time frame is. My scenarios for anyone wanting to repeat it is trading only from 8:30 – 10:30 Smallish number of trades and an easily identifiable pattern. Then add longer time frames. I Used my local country index. In my case the South African top 40, same as DAX or FTSE 100
To see what I was dealing with I decided to leave the test open on the chart and found exactly what I expected if I let the chart run for 2 weeks on live data. So now I had a pretty good idea of what the live data produced in terms of trades and winning and loosing, that equates to 20k units. Then going to 15 K units I should be able to see the same outcome as I got on the live data. Not even close.
Does anyone in the group have an idea of how I can get data that is accurate, or am I doing somthing wrong in the back testing phase. Are there any commands that could be included into the script to get more accurate data.
Grant Murray
08/05/2023 at 3:59 AM #218612Hi there Grant,
This is not something I recognize, although I should be doing sufficient backtesting to run into your issue if it was really there globally. It is only that I don’t have access to FTSE and for DAX I don’t have consistent results anyway, so let’s say that I am not exactly experienced in the instruments you use.
Am I right that you are working with PRT-IBKR (and not PRT-IG) ? And if so, can you try the same backtests on V12 vs V11 ?I like to add that your post is not so easy to read for me as you start out with talking about the 1 minute timeframe (which is a phenomenon) while you proceed with that same term “time frame” and mean longer backtesting periods. Or ? But if you really mean “timeframe” as such (as in 1 min, 1 hr, 1 day) then all is moot (for understanding) because then it lacks the concrete examples.
If you can post your code (as text and as .itf) it is 100 times easier to help out (we could try the very same as you do). If you rather not show your code, this is understandable.
Groeten,
Peter08/05/2023 at 11:15 AM #218621ok thanks for the reply so start on one minute time frame. Run the test with the number of units at 10 K units. Save the test results with a screenshot. Then leave the time frame as 1 min but change the number of units to 25000 units. Run the test again with all the same settings. You are not interested in any of the test results except the last 7 days. Compare the test from the 10K with the test of the 25k but limit yourself to only looking at the last 7 days of the test.
Then extend the number of units out to the maximum.
If the historic data is correct then the output of the last 7 days of each test should be the same. So if we look at this from the end as day 7 made 100 points on the 10 K test, it should have also made 100 pionts on the 25K test and on the longest 195,000 units as well.
If you look at the trades on the last 7 days they are all diferant in terms of the number of points won or lost in these trades.This indicator was not written by me it was one that I found that was written by <span class=”bbp-author-name”>Nicolas</span> all I have done is changed and cleaned out the code of anything that is not necessary for the trading system.
This will work on any index, this is not an automated trade in this form as it gives out huge numbers for incorrect signals when the market goes flat or sideways. I use this for manual trades and it works very well I was just looking to fine tune the settings and found this error with the historic data. Its a type of Supertrend using STD Deviation as the source for the changes.
Try it on an index and see what you get on your trading system
my feed is PRT / IGGrant Murray
DEFPARAM FLATAFTER = 103000
DEFPARAM FLATBEFORE = 083000
// — settings
SignalPeriod = 8
ArrowPeriod = 2.3// — end of settings
bbup = average[signalperiod]+std[signalperiod]*arrowperiod
bbdn = average[signalperiod]-std[signalperiod]*arrowperiod
//>if ts=0 then
if close crosses over bbup then
ts=bbdn
trend=1
elsif close crosses under bbdn then
ts=bbup
trend=-1
endif
endifif trend=1 then
ts=max(ts,bbdn)
elsif trend=-1 then
ts=min(ts,bbup)
endifif trend=1 and close crosses under ts then //and verify < trig then
trend=-1
ts=bbup
r=255
g=0
//drawarrowdown(barindex,ts) coloured(“red”)
//drawsegment(startbar,startts,barindex,ts) style(dottedline2) coloured(“blue”)
startbar=barindex
startts=ts
endifif trend=-1 and close crosses over ts then //and verify > -trig then
trend=1
ts=bbdn
r=0
g=255
//drawarrowup(barindex,ts) coloured(“lime”)
//drawsegment(startbar,startts,barindex,ts) style(dottedline2) coloured(“blue”)
startbar=barindex
startts=ts
endif//return ts coloured(r,g,0) style(dottedline,2)
// Conditions to enter long positions
IF NOT LongOnMarket AND trend = 1 and close crosses over ts THEN
BUY 1 CONTRACTS AT MARKET
ENDIF// Conditions to exit long positions
If LongOnMarket AND trend= -1 and close crosses under ts then
SELL AT MARKET
ENDIF// Conditions to enter short positions
IF NOT ShortOnMarket AND trend= -1 and close crosses under ts then
SELLSHORT 1 CONTRACTS AT MARKET
ENDIF// Conditions to exit short positions
IF NOT LongOnMarket AND trend = 1 and close crosses over ts THEN
EXITSHORT AT MARKET
ENDIF
SET TARGET $PROFIT 500008/05/2023 at 11:33 AM #218622Nah …
De mistake you seem to make is that you think that the “flow” of all the orders will remain the same in each backtest you are performing. But – a bit depending on the time in the market – this hardly ever is so. For example, if your position stays in for more than one day and you add one day to the previous backtest of 7 days – now becoming 8 days – then when a position is taken at this first new day, it will run into the second day and an original position on that 2nd day (7 days back) can not be taken because one is running already. And all will be different.
See the example below and envision that the Long taken there is at the beginning of your -8 day (with one added day). Also envision that the -7th day starts at half of the chart you see (so look to the right for half of the graph) – now it is obvious that what previously started on that half/midpoint, can not start because it already has started (a position is already there).08/05/2023 at 12:48 PM #218624Hi
Yes I understand that and I would see that as normal for longer trades that dont have any form of time limit. But if you check the code the trading times are strictly limited to 8:30 till 10:30 in the Morning on the same day so there are absolutely no overruns, every trade is killed off whether in profit or not at 10:30 on the same day.
All this test is doing is verifying that the data that is part of the set you are testing is exactly the same for every unit count and from what I can see its not. This is not necessarily a bad thing it just an observation that I wanted others to test to see if this is in fact the case or if this is limited to somthing in my particular instrument that I am Trading.
This changes nothing in the way you trade and the way you use the indicator all it does tell you is that using a back test to get a better set of setting for the indicator is not a 100 percent true reflection of what the history of the market instrument did. So historic testing even if the partial day is eliminated, in this case it is, has a percentage of error in the data.
Grant
08/05/2023 at 2:13 PM #218626Left is 10k bars of 1 minute. Right is 15k bars of 1 minute.
First attachment : Left Starts where it starts regarding the 10k bars. Right starts at the same date/time (I scrolled it to tat position) and each trade looks the same to me.
Second attachment : The end of both of them (last Friday (yesterday)). Here too all looks the same to me.You can show yours too ? (this is MicroDAX)
08/05/2023 at 2:49 PM #218629Hi Grant,
What are you changing…?
When you change the time frame (1 min, 5 min, 1 hour…) you can never compare the results… (the trades will be different)
When you increase the number of units (10k units, 15k units, 100k units…), you only extend the period you are trading over and this does not change the trades that were (already) executed…
08/05/2023 at 3:34 PM #21863008/05/2023 at 3:40 PM #21863108/05/2023 at 4:08 PM #218632 -
AuthorPosts
Find exclusive trading pro-tools on