How important is spread in back tests?

• 03/11/2021 at 12:35 AM #163764
  Vonasi

  I’ve posted topics on the subject of spread before but tonight a strategy got me thinking about it again.

  I created a mega simple price action daily strategy that performed well across all of history on the DJI but when I added a spread of 4 it fails in the early years and then becomes very profitable from 2000 onwards. I have to assume that if we had been able to spread bet from 1970 to 2000 then we most likely would not have had a spread of 4 as it would have been a much higher percentage of price than it is today. Also volatility has changed over the years and been higher at some points and lower at others and as we all know volatility effects spread.

  So my point for discussion is…. how important is spread in back tests in reality?

  On my daily strategy it could fool me into thinking that in the early years my strategy didn’t work very well when in reality spread might have been tiny and the strategy actually worked well. In later years spread fixed at a certain value in back tests lulls me in to a false sense of security as in the volatile times my wins would be smaller and my losses would be bigger.

  The first image is my simple strategy with no spread and second image is spread of 4.

  On a daily time frame back test I think that back testing with spread probably tells us as many lies as it tells us truths. However on a much faster time frame it is far more important to add it to the calculation of profits and losses. When creating strategies we often moan about the lack of historical data that we can back test on in faster time frames but in reality being able to test a 1 second strategy in 1970 using a fixed 4 point spread value which is relevant only today would tell us nothing about whether our strategy wins or loses.

  Your thoughts on spread are welcome!

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  03/11/2021 at 4:06 AM #163773
  Finning

  Hi Vonasi,

  yes, totally agree with you that spread cannot be disregarded. Change your spread from 1 to 5 pips and see what happens.

  Spread-Betting – if you can’t analyse the spread – then you are just betting.

  In my opinion, Bid and Offer data not being provided by a spread betting broker is criminal on their behalf. And us as the consumers, we have no choice but to prevail, or to disengage.

  As you have said, and I tried to allude to above, if you don’t know what your spread is, how do you know if it’s a good idea to bet?

  A mid price is just that – a mid price. Half way inbetween two prices, the bid and the offer. Both of which must then be known, to find the mid.

  That these prices are not known, can not be true. And it is not true.

  Indeed, if you are using the IG Excel API, you can get the Bid and Offer data.

  I could post a link but it will be deleted; there are external software companies – I am sure that there are more than the one I found – that put a nice front end on the IG Excel API and make it a bit more usable, if  you want to get your data into Excel. I have attached a photo showing an example of some of their data that they can provide, and, low and behold, there are the bid and the offer prices.

  The spread data is there for manual traders on your screen – the bid and the offer. But as soon as you try and do something with software/automation – this data – that is on your screen – is somehow not provided for you for computational use.

  Maybe that’s a massive clue as to how important it is?

  Especially because the spreads are artificial – they are set – for an over the counter product. They are there – how could they not be there? The spread is the crux of an OTC brokers business.

  Sorry Vonasi, not having a rant – at you.  But I think that it is criminal that spread betters (the trader), can’t get access to spread information for automated trading – only spread information for manual trading.

  Simple as that.

  Because in my opinion, it can have a huge influence on whether you should even be trading, or not.

  Finning.

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  03/11/2021 at 9:23 AM #163791
  Vonasi

  Yes all your points are valid but spread is available for auto trading in exactly the same way as it is for manual trading – live and in the future only. I can’t look at a trade on some candles in 1992 and pretend to manually trade it and know what the spread would have been exactly the same as I can’t know it if I auto traded it in a back test.

  Spread betting only started in 1974 long before the computer was invented and so a tick by tick record of spread is unlikely to have been recorded back then so even if IG added historical spread I imagine it wouldn’t go back very far.

  IG could easily start recording tick by tick spread and provide that feed to PRT from now until the future but it would take a long time before any useful database was ever compiled for back testing on a large sample.

  So I still think that in slower time frame strategies worrying about spread in back tests is pretty pointless. Usually the movements between the price you buy at and sell at are pretty big and 1 pip or 4 pips is not likely to be the difference between a winning strategy and a losing one. However if at volatile times spread went into three figures then it could do! In faster time frame strategies where we might be snatching at a few easy pips or the odd small loss spread becomes very important. This just proves why it is very important to forward test live in demo with live spreads before going really live with a strategy – not only to test for curve fit but to test for effect of fees and spread.

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
  03/11/2021 at 10:17 AM #163808
  jebus89

  Hey Vonasi, i have come to same conclusion. Unless your daily-algos are looking for 1-2 pips, spread is not that important to look at. And obviously a 4 pip spread in a 30.000 market is very different then a 4 pip spread when that same market was 3.000 20 years ago or whatnot.

  I would like to add another thought to this. I have looked into my own trading comparing PRT results vs real results and there is a clear difference. I now value the avg $ per trade vs spread alot higher than before. I dont have exact numbers and im very sure this varies some from market to market, but i would say that you have to expect a more negative overall result in your live results vs your backtest/PRT results!

  For instance i have several trades (a very small % vs all trades) that have been stopped out in live results, but not backtest (even with tick by tick, because PRT dosnt know the buy/sell price, it guesstimates) + a few trades that have not hit target even tho it hit target in backtest/PRT, again for the same guesstimating issues that PRT dosnt know the exact prices. Altough this only affect a few trades in the overall picture, those few trades could be HUGE difference makers when youre trying to find the big trends. We all know that missing 1-3 of your best trades makes a big difference for trend traders..

  Another thing regarding this issue is that its very hard to track individual real results from each algo unless you track each trade, at least for me having done thousands of trades, this is not possible.. Would need to hire someone to constantly keep track and records of each algos live results..

  So i would say expect even less of your algos then what you see, make sure your edge is bigger than “just barely profitable” in backtest, because that will break you. A great example are 2 of my old forex algos, where the edge was just 1-2$ per trade. These algos still look decent in backtest even with spread, but the real life results was not profitable and they just bled money from my account..

  All of this is of course only my own experience. But i would love to hear if someone is facing similar problems, and if anyone has any sort of solution or numbers to go on.

   

  Edit: This is also very important, in my view, to think about when youre using machine learning to optimize algos! Your “perfect” targets and stop losses and trailing stop losses might be flawed. maybe 1-20% of the trades might not higt targets or might get hit by stop loss, or your entry might be worse than the backtest making the stop loss a bit higher. This is just the spread and the filling of orders. PRT dosnt know when or how you actually get your order filled, it just guesstimates that you get filled, but theres no guarantee until you go into real money.. (unless you use limit orders! making your entries guaranteed! Been very eager to discuss this topic but dosnt seem that many people use these types of orders – if you got experience with this please let me know)

  

  jebus89

  Participant

  Topics: 118

  Replies: 586

  Been thanked: 238 times
  03/11/2021 at 11:27 AM #163821
  Finning

  Hi Vonasi,

  not trying to pick one here – just respectfully asking a question – but how is spread information available for auto-trading the same as it is for manual trading? With autotrading, your algo can’t see the bid and the offer – the spread isn’t visible – but you can see it with manual trading – the two prices that come up on your screen. Yes, you can put in estimates of spread into your auto trading from what you see of the two prices on the screen, but that’s it. You might be able to see it with your eyes, but your program can’t “see” it.

  If a single price – the mid – is recorded pip by pip – you would only have to double that information per pip to have bid and offer data, from which you can get the spread. They mightn’t have bid and offer data from 1992, but if they have pip by pip data from August 2010, then that’s a good start. IG wouldn’t have to start recording bid/offer data now – it is already there – it is available through the Excel API. They just need to make that same data available to PRT!!!

  Yes, I am off track with your initial question about spread. Me complaining about this deliberate lack of information isn’t going to change things, of course.

  How does spread affect trading?

  All you can do, and I agree with Jebus89, is that you have to have as big a per-trade profit possible – all the while  minimizing your hold time due to overnight fees. You don’t want your per trade profit to be borderline with the spread amount. So in that regard, yes, spread isn’t important. But only if you have trades that win big, in a short period of time!

  Unless your pants really are on fire, you don’t want to be in the 1-2 pip profit target area. I personally don’t operate in the 1 minute timeframe trying to pick off pips.

  But it would be good to have a handle on spread with regards to spread volatility – so if the spread is above a certain level, you can just turn off your system because it’s just not worth trading.

  I’ve seen some pretty high spreads at times… say for instance it goes to 20 pips. I know the volatility of the market has obviously gone up as well, but if your algo knew that the spread is that high, you might decide that it’s just not worth the risk to take the trade.

  

  Finning

  Participant

  Topics: 18

  Replies: 130

  Been thanked: 9 times
  03/11/2021 at 12:32 PM #163827
  Vonasi

  Some good points made…. but…. when manually trading you can see the spread but when you click to buy you have no idea at what price your order will be opened as a micro flash of wider spread could see you getting a very different price to the one you thought that you clicked on. That is exactly the same for an auto strategy if it could see the spread. It will see the spread at the close of a candle and decide it is OK and place an order to buy at the open of the next candle… do you see where I am going with this?

  As for overnight fees – I did a test a long while back to compare holding over the weekend or over night or not and I found that the fees made no difference to profitability. In fact it turned out more profitable to just hold and pay the fees in my test. Paying the spread to get in and out of a trade and missing beneficial weekend and overnight gaps wiped out any benefit of avoiding tiny fees. I guess this is very market dependent though.

  jebus89 – your point about simulated trading is very valid and one that I have tried to point out in threads about those subjects. It is very easy for a simulated equity curve to bear no relation to a real equity curve due to spread, fees and slippage. So using a simulated equity curve to base decisions on can lead to a lot of disappointment in real life.

  

  Vonasi

  Moderator

  Topics: 192

  Replies: 5088

  Been thanked: 1811 times
• Author
  Posts