How does backtesting work?

• 06/13/2016 at 7:08 PM #9343
  markouk

  Call me slow but 1.5 months into using PRT and I have only just today really observed how misleading the Back test function is. I have looked into the forum and can see some threads on this but nothing has really explained it in full. Can anyone explain exactly how it works. There is always a huge contrast between back test strategies and then actual real life situations.

  The attached screenshot demonstrates this.

  • On the left you have real money trading this morning.
  • On the right is a back test I just performed.

  Obviously I am trying understand the floors in the strategy but am finding if difficult when the back test is not at all similar to real LIVE trading. My LIVE trading was mirrored in Demo accounts all making losses for the day. But when in back test I only ever get profits.

  This is not the first instance of such contrast and I know my strategy is nothing amazing, but I now recognise that I really need to understand the back test before I can create strategies with minimum risk (i.e. no testing on real money).

  Are there any Gurus out there who can explain how this works please.

  

  markouk

  Participant

  Topics: 8

  Replies: 22

  Been thanked: 0 times
  06/14/2016 at 11:58 AM #9380
  K12AN

  Backtesting uses static data i.e. the candles/bars which have already been formed. So you can already see the high, low, open and close in your timeframe. The problem is, this snapshot does not show exactly how this candle was formed and what actually happened during this timeframe. Back testing using static data give the best case scenario. So your profit target will be achieved always before your stop loss is triggered. This is obviously not always true – as you are seeing in demo/live. To backtest to see if your stop loss would be triggered before your take profit, you need tick data for back testing.  This can be quite expensive.

  The good news is that PRT are working on making backtesting more realistic, bad news is there is no date as to when this will be available.

  Using stop losses, and trailing stops etc will really flatter your strategy performance in backtesting and make you believe that you will be a billionaire within months 🙂 Unfortunately, this is not the case.

   

   

   

  

  K12AN

  Participant

  Topics: 10

  Replies: 13

  Been thanked: 1 time
  06/14/2016 at 12:37 PM #9382
  markouk

  K12AN, thank you for taking the time to explain which you have done so perfectly for me to understand. For a while there I had thought Millionaire was possible quite quickly, let alone Billionaire 🙂

  The data space size required to record every tick would be huge and then the larger internet stream for each user, so I would understand why this would be expensive.

  If I understand your email well, building a strategy that shows a profit without the use of Stops is probably going to be a good one. Obviously stops can be put in place later when trading for real.

  

  markouk

  Participant

  Topics: 8

  Replies: 22

  Been thanked: 0 times
  06/14/2016 at 2:43 PM #9390
  grizzly

  Mark,

  Any backtest result that has a entry and exit position worth 0 bars is completely unreliable.

  I suggest using this condition to ensure all entries and exits are at least held open for 3 candles (for backtests and live account):

  Stayonmarket = (barindex-tradeindex>=3)

  1	Stayonmarket = (barindex-tradeindex>=3)

   

  2 users thanked author for this post.

  Asura, markouk

  

  grizzly

  Participant

  Topics: 5

  Replies: 57

  Been thanked: 16 times
  06/14/2016 at 4:48 PM #9396
  K12AN

  Hi Mark. If you don’t put a stop, but do put a target profit, your profit will be achieved (most of the time)…however, once you implement your stop loss in real life it is possible/likely that your Stop could be hit before your profit target – defeats the purpose of back testing.

  I have had suggestions of testing strategies on 1 minute timeframes etc, but for me this wouldn’t work, as my strategy is for swing trading and only takes 1 position per day and would like to close the position at 17:00.

  In back testing this flatters my results, as I open and close a position within the same (daily) bar – so my profit may be hit before my SL.

  I have tested my strategy to not have a target profit or a Stop loss and just accept the outcome of the trade at close – but this is a risky approach and the drawdowns and returns are not impressive. Far from what I saw if I include a stop loss or target profit (which would positively skew my backtesting).

  A lot depends on what time frame you intend to trade. If you’re able to open a trade, but not close it until the next bar or later (and not have a SL) then your system may be a good one. However, I may be corrected on this as I’m fairly new to this as well 😉

  Also don’t forget to include spreads and account for slippage in your back testing.

  I remember seeing the initial back test results for my strategy and thought I’d won the lottery! Tested it in demo mode and it was completely different which lead me to research the shortcomings in backtesting. If only they were true 🙂 Still not giving up on automated trading, but am still yet to find a strategy which has been forward tested in demo/live trading and consistently makes good returns.  Has anyone got evidence of such a strategy? I don’t want to know the workings of it, but just would like to know that it IS possible 🙂

  1 user thanked author for this post.

  markouk

  

  K12AN

  Participant

  Topics: 10

  Replies: 13

  Been thanked: 1 time
  06/15/2016 at 12:48 PM #9412
  markouk

  I am sure this is a step that everyone takes in their journey into auto trading and now it’s my turn. So I have decided the best way forward is to create strategies in the way you think is a winner BUT then test in real time on a demo account. For me back testing is proving way too unreliable and massively time consuming once you start introducing variable optimizations etc. Testing every strategy in realtime may be slower but at least I will come to understand if it works or not within say….a day or two testing in ProOrder.

  Thanks again for everyone’s comments and input.

  

  markouk

  Participant

  Topics: 8

  Replies: 22

  Been thanked: 0 times
  06/15/2016 at 1:00 PM #9413
  grizzly

  One of the pitfalls I’ve found with Optimising variables is that you creep dangerously into curve-fitting the backtest results.

  It’s best to think of Optimising as really a “Stress test”, rather than a “What numerical value do I use for SMA to maximise my statistical results?”.

  E.g. What is the outcome of my TS if I entered the market 1 candle later ? What happens if I choose to trade only on Tuesday to Friday ?

  

  grizzly

  Participant

  Topics: 5

  Replies: 57

  Been thanked: 16 times
• Author
  Posts