Worst Performer – Optimise vs Real Time?

• 11/14/2018 at 11:41 AM #84830
  GraHal

  Just a topic for discussion / opinion, hope you join in?

  What you reckon is the Indicator / set up that gives the biggest variance between results under Backtest / optimisation  compared to Forward Test?

  I’ve been having a clearout of old Systems under Forward Test and I noticed amazing results under Backtest optimisation (BT) for MA Cross but then very poor results when the same variable values are put on Forward Test (FT).

  Seems the more individual sets of MA Cross then the bigger the variation (BT to FT). Does the Optimisation engine find combinations of MA Crosses that do not occur again at the same price action points under Forward Test!?  It’s hard to believe though when the MA crosses under BT gave smooth equity curves (not a few big spikey gains) over 100k bars etc.

  I know there is loads on the internet re why MA Cross are not a good basis for an Auto-System?  Have you read up on this and / or have more intelligence to add re MA Crosses etc please?

  Please add to the discussion re Indicators / set ups with inherent big variance between BT & FT … this may prevent newbies getting discouraged / losing hard-earned cash and also focus the minds of seasoned traders!?

  

  GraHal

  Participant

  Topics: 214

  Replies: 8828

  Been thanked: 2200 times
  11/14/2018 at 1:16 PM #84847
  Vonasi

  It sounds like you have just curve fitted your MA crosses to your historical data. From your ohther posts it seems that you seem to favour faster time frames so I am guessing that the actual amount of historical data in your 100k of bars was actually not very much actual real time to base your curve fitting/optimization on.

  The trouble with MA’s is that their behaviour is very different for trending markets or sideways markets so with no filter to identify what market they are in they will either perform great or terrible.

  The other thing to consider is what other possible curve fitted elements are in your strategy. Fixed take profit and stop loss levels could be the real curve fit culprits as it is generally easier to curve fit an exit than it is to curve fit an entry.

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  11/14/2018 at 2:24 PM #84854
  GraHal

  Vonasi wrote:

  actual amount of historical data in your 100k of bars was actually not very much actual real time

  If 100k bars of (for example)  5 min TF shows as plenty of rises / falls / sideways and trends / no trends etc AND the BT results contains 100’s of trades.

  Surely the statistical signifance of results of above BT  would be comparable to results of Daily TF over 100k bars … bearing in mind, for example, a MA(5) on 5 min TF is changing it’s value relative to the last 25 mins whereas a MA(5)  on 1 Day TF is changing it’s value relative to the last 5 days.

  

  GraHal

  Participant

  Topics: 214

  Replies: 8828

  Been thanked: 2200 times
  11/14/2018 at 2:57 PM #84860
  GraHal

  To clarify … a Trend on a 5 min TF may last 2 hours (24 bars) whereas a comparable trend on a Daily TF would be > 1 calendar month (24 trading days).

  Anyway your younger than me Vonasi … I’ve not got enough time left to wait for Daily bars! 🙂

  

  GraHal

  Participant

  Topics: 214

  Replies: 8828

  Been thanked: 2200 times
  11/14/2018 at 3:43 PM #84870
  Vonasi

  You always have to consider that there is a bigger picture. Your 100k bars at 5 minute is just a snap shot of time. You could be carrying out your optimization on a sideways market and then put it live just as the market turns to a trending market. You just see lots of ups and downs in your back test period with your rapidly changing candles but you have to zoom out and see what the bigger picture is. Testing on the last two weeks does not tell us how a strategy would work in say something like the 2008 recession. You may have just launched your strategy as we re-enter one of those but you have never tested it in one (and unfortunately cannot with PRT). The next two weeks could very easily see completely different market behaviour to the snapshot of data that you optimized/curve fitted on.

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  11/28/2018 at 8:30 AM #85885
  jebus89

  I think for me, moving averages has to be the worst indicator for moving forward. Unless used as just a filter i think its very scary to optimize moving averages and using them/it as a trigger for entering a trade. Also i have noticed that 2018 has been an EXTREMLY shitty year for moving averages and macds and everything in that area. There has been some huge selloff-dips. Ive had many algos taking trades because they think the market is hitting a bottom, just to plummet further down.. Many false signals and false breakouts and instant down 5% the next candle lol. So i wouldnt be too quick to judge a system as a bad system if its just been acting up lately. But yea moving averages as triggers for entry/exit is not deemed safe in my playbook.

  Also ive found that if a system looks great in backtest but going forward in demo it just looks like instant shit => That is 9/10 times me curvefitting my system. The last 1/10 is a good system that is just having a really rough time when u activate it.. I managed to activate 10 systems the day b4 the october crash lol. Did not look good going 1-2 weeks into that one..

  

  jebus89

  Participant

  Topics: 118

  Replies: 586

  Been thanked: 238 times
  11/28/2018 at 10:12 AM #85891
  Vonasi

  jebus89 wrote:

  Also i have noticed that 2018 has been an EXTREMLY shitty year

  2018 has been a year where our  curve fitted to historical data rule book that we all have to work with has been severely tested. It is as if the markets have forgotten everything that has ever happened before. Which they do everyday – it is only us humans who remember! Markets do not move themselves – humans move them and these changes in market structure and the ignoring of the past just prove that our curve fitted indicators and our curve fitted interpretation of them is only a tiny fraction of what moves the markets. Fear and greed are the main market movers. At the moment the greedy have decided that everything has gone high enough but they’re not 100% sure so they keep buying a bit of value and then getting out again – and the fearful are just getting out and moving their money to safer options. We have to remember that any major markets value is based on that of individual companies trading throughout the world and the value of forex depending on the currency those companies trade in – so it is really a value based on the world economy and the world economy is based on how much spare cash people have to throw around. At the moment there is a lot of cash but people are a bit worried about the future and so hesitant to spend it – hence the choppy sideways markets coming off of all time highs. No moving average is going to be able to follow all that fear and greed.

  I don’t think that I have a single MA in any of my strategies although I have just spent a few days playing with the generalised DEMA Nicolas posted in the library. It seems like a reasonable filter but like all indicators has days when it fits the market better than others. As soon as I have to optimize a period for an indicator I have a little alarm bell in my head that rings ‘curve fit’!

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
• Author
  Posts