Technique help to reduce over-fitting
Forums › ProRealTime English forum › ProOrder support › Technique help to reduce over-fitting
- This topic has 63 replies, 12 voices, and was last updated 3 years ago by Paul.
Tagged: overfit, overfitting, robustness
-
-
09/02/2020 at 7:04 PM #143234
Always Keep It Simple (In code and strategy). 3 Variables only, 100 000 units.
Strategy without overfitting (or as little as possible) based on Stochastics I develop today, but no matter for the strategy on backtest. Need to test on OOS now
Have a nice evening
09/02/2020 at 7:27 PM #14323809/02/2020 at 7:45 PM #143239Your new strategy looks nice, @zilliq, thanks for the tips too.
Btw, recently I picked up a topic thanks to @vonasi, I think it is helpful on reducing the optimization time as well (which I believe everyone is so eager).
Control of yearly/monthly/weekly profit/loss => https://www.prorealcode.com/topic/annual-percentage-returns/#post-93664
The article is originally for yearly, but you can transform it to monthly/weekly (below I share example for monthly). The better is even set a range of highest profit and lowest loss allowed.
Since we added a quit once the bad month happened, thus the back test is faster, it will not continue to test for the rest of the months.
You can even add a condition check to not have consecutive month/weeks (even days) of loss, e.g. do a summation[3](monthlyreturn) <= 0 for no 3 months consecutive loss.
12345678910111213141516capital = 5000once lastcapital = capitalrunningperc = ((strategyprofit - laststrategyprofit) / lastcapital) * 100if month <> month[1] thenmonthlyreturn = runningperc[1]laststrategyprofit = strategyprofitlastcapital = capital + strategyprofittotal = total + monthlyreturncount = count + 1//monthlyaverage = total/countendifif monthlyreturn < -0.5 thenQUITendifSince the optimization timing is faster (thanks to skip the bad parameters), you can have a bigger range of optimization.
09/02/2020 at 7:47 PM #143240I guess you are teasing us and you are not going to share your excellent looking Strategy??
@GraHal, you can subscribe when he put in market place, haha 🙂 But I’m sure you also have such strategy, right? Perhaps same for many people here as well…
09/02/2020 at 8:18 PM #143242No loosers?
Of course there is losers;- 87 % winners
100 % winners on 100 000 units = Overfitting for me@dow jones LOL not even.
I consider to sell an algo is a scam because it need always some work and retesting
09/02/2020 at 8:28 PM #14324309/02/2020 at 9:55 PM #14325309/03/2020 at 8:31 AM #143260Of course there is losers;- 87 % winners
Good! Any chance to share the same screenshot with position size window below the equity curve?
09/03/2020 at 9:14 AM #14326509/03/2020 at 9:19 AM #14326809/04/2020 at 11:05 AM #143365To continue to minimize the overfitting, today I begin an OOS demo test with a strategy on SAR to have only 2 variables
58,99 % winners on the backtest but on 556 trades ! 100 000 units EUR/USD 1 mn Sharpe Ratio 1.1.
We will see if it was a good idea 🙂
Have a nice day
Zilliq
09/04/2020 at 11:09 AM #14336709/04/2020 at 12:20 PM #143374Control of yearly/monthly/weekly profit/loss
An update to this one, below is an example to control on the consecutive monthly and weekly loss.
You can probably extend it to maximum win for monthly/weekly too (I didn’t do it as it is already difficult to achieve no monthly and weekly loss over long period).
I kind of think that a stable strategy should have stable income, so no huge spike or huge drawdown on certain period that resulted good total gain only.
Adding this means the back test will stop for the combination if you will anyway don’t like it, e.g. if you won’t accept strategy that loss 2 consecutive months, then no point to continue the back test for whole period.
So making the back test a bit faster. Hope you guys like it and suggestion welcome.
1234567891011121314151617181920212223242526272829303132333435363738394041//1. Check monthly not lower than expected profit and no consecutive month losscapital = 5000monthlymingain = -10maxconsecutivemonthlyloss = 1maxconsecutiveweeklyloss = 2once lastmonthlycapital = capitalonce lastweeklycapital = capitalonce consecutiveweeklyloss = 0once consecutivemonthlyloss = 0runningmonthlyperc = ((strategyprofit - lastmonthstrategyprofit) / lastmonthlycapital) * 100runningweeklyperc = ((strategyprofit - lastweekstrategyprofit) / lastweeklycapital) * 100//Ensure none of the month less than expected gainif month <> month[1] thenmonthlyreturn = runningmonthlyperc[1]lastmonthstrategyprofit = strategyprofitlastmonthlycapital = capital + strategyprofitif monthlyreturn < 0 thenconsecutivemonthlyloss = consecutivemonthlyloss + 1elseconsecutivemonthlyloss = 0endifendifif monthlyreturn < monthlymingain or consecutivemonthlyloss > maxconsecutivemonthlyloss thenQUITendif//2. Check consecutive week of loss is acceptableif opendayofweek < opendayofweek[1] thenweeklyreturn = runningweeklyperc[1]lastweekstrategyprofit = strategyprofitlastweeklycapital = capital + strategyprofitif weeklyreturn < 0 thenconsecutiveweeklyloss = consecutiveweeklyloss + 1elseconsecutiveweeklyloss = 0endifendifif consecutiveweeklyloss > maxconsecutiveweeklyloss thenQUITendif1 user thanked author for this post.
09/04/2020 at 1:35 PM #143380Very good idea and thank you for that great snippet DowJones. But, while it should improve backtest time, you will also introduce a form of over-fitting by avoiding bad months, yeah i know nothing’s perfect 😉
1 user thanked author for this post.
09/04/2020 at 1:56 PM #143386you will also introduce a form of over-fitting by avoiding bad months, yeah i know nothing’s perfect 😉
Nicolas is right (as always!) Also remember that a strategy cares nothing about things like when the name/number of the current month changes.
You could have a bad second half of a month and a bad first half of the next month and that back test is included in the results because both months are average results whereas if they same results had just happened a couple of weeks earlier and occurred all in the same month then that test would be excluded. Surely it is better just to have a rolling period or rolling quantity of trades and if the results are below what you would be happy to trade then quit the back test to save optimising time?
1 user thanked author for this post.
-
AuthorPosts
Find exclusive trading pro-tools on