Why is backtesting so unreliable?
Forums › ProRealTime English forum › ProOrder support › Why is backtesting so unreliable?
- This topic has 20 replies, 3 voices, and was last updated 2 years ago by Nicolas.
Tagged: backtest, backtesting, oos, renko, wf
-
-
10/22/2019 at 2:57 PM #110841
I’ve developed a few codes that appear to be successful in backtesting, but whenever I run them in demo nothing comes close to that kind of profit.
I adapted the following from the Pure Renko Strategy posted by Doctrading.
This version is designed for the DJI 5min chart on UK time and shows incredible results – 75% wins over 17 months with only 3 losing days. Well, you know what they say about things that sound too good to be true. Run it live and it loses money every day. Same story with other seemingly successful codes I’ve developed. I’d be very curious to know why there should be such a huge discrepancy.
Thanks in advance for any enlightenment.
123456789101112131415161718192021222324// Definition of code parametersDEFPARAM CumulateOrders = False // Cumulating positions deactivated// The system will cancel all pending orders and close all positions at 0:00. No new ones will be allowed until after the "FLATBEFORE" time.DEFPARAM FLATBEFORE = 143000// Cancel all pending orders and close all positions at the "FLATAFTER" timeDEFPARAM FLATAFTER = 210000boxSize = 13once renkoMax = ROUND(close / boxSize) * boxSizeonce renkoMin = renkoMax - boxSizeIF high > renkoMax + boxSize THENWHILE high > renkoMax + boxSizerenkoMax = renkoMax + boxSizerenkoMin = renkoMin + boxSizeWENDELSIF low < renkoMin - boxSize THENWHILE low < renkoMin - boxSizerenkoMax = renkoMax - boxSizerenkoMin = renkoMin - boxSizeWENDENDIFbuy 1 CONTRACT at renkoMax + boxSize stopsellshort 1 CONTRACT at renkoMin - boxSize stop// Stops and targetsSET STOP $TRAILING 1210/22/2019 at 3:20 PM #110847nonetheless – Welcome to the forums. When posting code you are expected to use the ‘Insert PRT Code’ button to make the code more readable to others. I have tidied up your post. Please use the button in future posts. 🙂
1 user thanked author for this post.
10/22/2019 at 4:09 PM #110863Same story over and over again 🙂
These are the things you should be aware of:
- try to not over optimize your settings, it will fails if you don’t validate an OOS period (overfitting)
- spread in backtests, did you set it?
- broker slippage: no real way to simulate that in backtests
- renko bricks construction: only dependent to price moves, so results will be completely different if you start a backtest with a different time period, because price will start moving at first bar of history
- set stop trailing instruction: doesn’t respect the IG trailing stop step, don’t use it
- always make backtests with tick by tick box checked
1 user thanked author for this post.
10/22/2019 at 4:59 PM #110876You should never do tests with 5 minute time frames, 17 months are very few.
Normally you only have profits on paper but then you lose money.10/22/2019 at 5:18 PM #110883Thanks for the advice – I think I’ll stick to manual trading, at least I know how that works.
Best of luck to all!
10/22/2019 at 6:04 PM #110887Just confirm that you had the tick by tick Mode box selected in your backtest @nonetheless … as without using the tick by tick mode then bt results look like the holy grail!!
Just tying to save you getting too disillusioned with Auto-Trading 🙂
10/22/2019 at 6:16 PM #110889Is attached something similar to what you got @nonetheless … because attached is without tick by tick as that is all I could get due to too many instances of … well, there’s a lot to it and you be better to read up elsewhere on this website.
Suffice it to say that Real Live is guaranteed to be nothing like attached due to TP and SL being met in the same bar … in short your Trailing Stop is way to low / close to Trade Price.
10/22/2019 at 7:20 PM #110893The problem is not the tick by tick, but the reduced time of the test ….. And even as it is done, with the computer it is easy to get back exceptional results.
1 user thanked author for this post.
10/22/2019 at 7:37 PM #110897Below looks like it might be more realistic … see how it is Short during a down period and vice versa?
I’m putting below on Forward Test on Demo for a while … never had a Renko System before! 🙂
Nonetheless Thank You! 🙂 🙂
EDIT / PS
I’m sure below could be improved more, e.g. by adding one of the many coded Trailing Stops available on here.
1234567891011121314151617181920212223242526//https://www.prorealcode.com/topic/why-is-backtesting-so-unreliable/#post-110889// Definition of code parametersDEFPARAM CumulateOrders = False // Cumulating positions deactivated// The system will cancel all pending orders and close all positions at 0:00. No new ones will be allowed until after the "FLATBEFORE" time.//DEFPARAM FLATBEFORE = 143000// Cancel all pending orders and close all positions at the "FLATAFTER" time//DEFPARAM FLATAFTER = 210000boxSize = 110 //50once renkoMax = ROUND(close / boxSize) * boxSizeonce renkoMin = renkoMax - boxSizeIF high > renkoMax + boxSize THENWHILE high > renkoMax + boxSizerenkoMax = renkoMax + boxSizerenkoMin = renkoMin + boxSizeWENDELSIF low < renkoMin - boxSize THENWHILE low < renkoMin - boxSizerenkoMax = renkoMax - boxSizerenkoMin = renkoMin - boxSizeWENDENDIFbuy 1 CONTRACT at renkoMax + boxSize stopsellshort 1 CONTRACT at renkoMin - boxSize stop// Stops and targetsSET STOP PLOSS 75SET TARGET PPROFIT 1501 user thanked author for this post.
02/24/2020 at 11:25 PM #120355The above code (or a fork of) was doing very well for me on Demo Forward Test but for the last several days it gets Rejected / Stopped every time the code is read.
I get the Rejected with the error message … System Error.
Can anybody see anything in the code that ProOrder or IG may not like??
Is While, Wend approved terms etc ?? Thee must be some reason surely??
I know IG have had big Issues today, but lots of my other Systems taking trades just fine today so why is this System getting Rejected for System Error??
02/24/2020 at 11:42 PM #12035602/24/2020 at 11:51 PM #12035702/24/2020 at 11:59 PM #12035902/25/2020 at 12:32 AM #12036102/25/2020 at 12:47 AM #120365I’ve had stuff getting rejected all day, esp on the DOW and DAX, NAS systems working fine.
Rang IG about it but got a recorded message acknowledging platform problems and to stay on the line only if I wanted to close a position. Presumably they’re inundated with complaints.
BTW @Grahal glad to hear that Renko thing was actually working for you! Maybe I should take another look at it…
-
AuthorPosts
Find exclusive trading pro-tools on