Strategy Robustness Tester

Vonasi · Sun, 01 Sep 2019 12:34:14 +0000

Following on from a discussion elsewhere regarding walk forward testing I had the idea to create a bit of code that you can add to any strategy and see how it would have performed if you had traded on only odd days or only on even days, or odd or even months or odd and even years or any combination of the three. By comparing the output of all 23 tests we can get an idea of how consistently a strategy makes profit. The gain figure in the optimisation results should be ignored as the number of trades for each test will be very different. The average gain per trade, win rate and equity curve for each test can tell us a lot about the strategies robustness though. You just add the code to the start of your strategy code and then insert the extra condition ‘and tradeon’ in the conditions whenever you send buy or sellshort orders to the market. Then just optimise with a variable ‘choice’ with values 0 to 22 step 1. Then just click on each result in the optimisation report to see the equity curve for each test. Here is the code to add to the start of your strategy in case it is of use to anyone: //Day Month and Year Robustness Tester //By Vonasi //20190901 //0 = all days //1 = odd days only //2 = even days only //3 = odd months only //4 = even months only //5 = even years only //6 = odd years only //7 = odd days in odd months only //8 = even days in even months only //9 = even days in odd months only //10 = odd days in even months only //11 = even months in even years only //12 = odd months in odd years only //13 = even months in odd years only //14 = odd months in even years only //15 = odd days in odd months in odd years only //16 = even days in even months in even years only //17 = even days in odd months in odd years only //18 = odd days in even months in even years only //19 = odd days in even months in odd years only //20 = even days in odd months in even years only //21 = even days in even months in odd years only //22 = odd days in odd months in even years only oddday = opendayofweek mod 2 = 1 evenday = opendayofweek mod 2 = 0 oddmonth = openmonth mod 2 = 1 evenmonth = openmonth mod 2 = 0 oddyear = openyear mod 2 = 1 evenyear = openyear mod 2 = 0 tradeon = 0 if choice = 0 then tradeon = 1 endif //odd days only if choice = 1 and oddday then tradeon = 1 endif //even days only if choice = 2 and evenday then tradeon = 1 endif //odd months only if choice = 3 and oddmonth then tradeon = 1 endif //even months only if choice = 4 and evenmonth then tradeon = 1 endif //even years only if choice = 5 and evenyear then tradeon = 1 endif //odd years only if choice = 6 and oddyear then tradeon = 1 endif //odd days in odd months only if choice = 7 and oddday and oddmonth then tradeon = 1 endif //even days in even months only if choice = 8 and evenday and evenmonth then tradeon = 1 endif //even days in odd months only if choice = 9 and evenday and oddmonth then tradeon = 1 endif //odd days in even months only if choice = 10 and oddday and evenmonth then tradeon = 1 endif //even months in even years only if choice = 11 and evenmonth and evenyear then tradeon = 1 endif //odd months in odd years only if choice = 12 and oddmonth and oddyear then tradeon = 1 endif //even months in odd years only if choice = 13 and evenmonth and oddyear then tradeon = 1 endif //odd months in even years only if choice = 14 and oddmonth and evenyear then tradeon = 1 endif //odd days in odd months in odd years only if choice = 15 and oddday and oddmonth and oddyear then tradeon = 1 endif //even days in even months in even years only if choice = 16 and evenday and evenmonth and evenyear then tradeon = 1 endif //even days in odd months in odd years only if choice = 17 and evenday and oddmonth and oddyear then tradeon = 1 endif //odd days in even months in even years only if choice = 18 and oddday and evenmonth and evenyear then tradeon = 1 endif //odd days in even months in odd years only if choice = 19 and oddday and evenmonth and oddyear then tradeon = 1 endif //even days in odd months in even years only if choice = 20 and evenday and oddmonth and evenyear then tradeon = 1 endif //even days in even months in odd years only if choice = 21 and evenday and evenmonth and oddyear then tradeon = 1 endif //odd days in odd months in even years only if choice = 22 and oddday and oddmonth and evenyear then tradeon = 1 endif Then alter your strategy code to use the TRADEON variable so that trades are only opened if the day, month and year meet the test criteria. if (your long entry conditions) and tradeon then buy 1 contract at market endif if (your short entry conditions) and tradeon then sellshort 1 contract at market endif

This topic has 161 replies, 20 voices, and was last updated 3 years ago by Roger.

Tagged: casino, robustness, walk forward

Viewing 15 posts - 31 through 45 (of 162 total)

← 1 2 3 4 5 6 7 8 9 10 11 →

09/11/2019 at 9:06 PM #107276

Vonasi

Ooops! – QTY should be 2 to 10 (not 1 to 10) and RANDOM 1 to 5. both steps of 1.

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/11/2019 at 9:17 PM #107279

GraHal

Vonasi wrote:

QTY should be 2 to 10

Yeah I did – 2 to 10 for Qty – as I was very deliberately following the idiots guide at that point as I didn’t know what I was doing! 🙂

So yeah all my results are good! I even just double checked! 🙂

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/11/2019 at 9:30 PM #107283

Vonasi

Your image shows 2 to 20 and 1 to 10 not 2 to 10 and 1 to 5!

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/11/2019 at 9:47 PM #107285

GraHal

Yeah but that is okay?

I didn’t know what effect anything would have at that point so I went for a bigger range than you had said.

GraHal wrote:

I take it you mean Qty 1 to 10 and RANDOM 1 to 5 … both in steps of 1?

I was just double checking as you said Qty 10 and RANDOM 5

GraHal wrote:

Yeah I did – 2 to 10 for Qty –

I forgot what I used until I checked,

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/11/2019 at 10:21 PM #107290

Vonasi

I only commented on the settings because you were creating a Google sheet for 10 5 and if you dragged 20 10 data in then all the charts would be wrong.

1 user thanked author for this post.

GraHal

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/12/2019 at 2:45 PM #107350

GraHal

I’m addicted now! 🙂

It’s good as it kinda confirms what I suspected – but all the same never fully trusted – that certain of my running Systems are okay to leave running and most others benefit from my intervention / exit if they show a healthy profit on a particular trade … bird in hand worth two in the bush!

Attached are VRT results from a System I’ve had running for over a year and which has steadily been adding to profit (up and down, but nothing alarming) and I can’t recall ever stepping in and been horrified at a mounting loss on any particular trade.

I would say the VRT stats while not fantastic bear out above previous thoughts on this System?

I am going to add the VRT score number ( 47.7 / 48 in this case ) to my System title in future so I know if any System is good, bad or mid!

Thank you Vonasi for sharing this valuable tool with us all and thank her ashore for leaving you in peace aboard your yacht over the last few days. It’s a pity you weren’t a bit nearer I could have returned the favour by coming round with my paint brushes or scraped a few barnacles off! 🙂

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/12/2019 at 3:23 PM #107355

Vonasi

I’m glad you finally got your head around the tester and created your analysis graphs!

Perhaps you should share your Google version of the data analyser on here to save others having to fight the same war again?

Regarding that analysis graph – I would ignore the trades where the number of trades is quite low as they are a poor data sample for a test – in fact I have taken to clearing those rows from the worksheet which results in a broken line but gives a more accurate score if they are at either end of the graphed line.

On the point of barnacles – I never let another man touch my barnacles!

1 user thanked author for this post.

GraHal

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/12/2019 at 3:54 PM #107363

O-jay8

Well done Vonasi.

Nice analysis tool

1 user thanked author for this post.

Vonasi

O-jay8
Participant

Topics: 5
Replies: 207
Been thanked: 285 times

09/12/2019 at 6:40 PM #107378

GraHal

Re share … good idea!

Below is a link to a google sheets version of the Vonasi Robustness Tester (we can name it what you want?).

VRT Share

Google sheets does not allow drag and drop of data, but today I found an easy way to get data into sheets … no more drag to notebook then import as csv as I have been doing! 🙁

All we need to do is select the top row in PRT optimiser results then ctrl + c, wait 2 or 3 seconds then ctrl + v into a google sheet!

I guess the few seconds wait is to allow select all … but you can’t see any signs that the whole sheet is selected.

So with the above copy and paste and VTR Share Template you can complete the Analysis in a few minutes … after you get into a routine!

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/12/2019 at 6:52 PM #107384

GraHal

Vonasi I had a few thoughts … is there any point doing more than 100 combinations as when we sort by low to high then the best av per trade and % winning trades disappear below the fold (as they say) due to PRT Optimser results limited to 100 rows?

Not only is there no point, > 100 combos would give false results surely??

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/12/2019 at 8:09 PM #107385

GraHal

So whats the gig with this one??

Seems weird that the randomiser should result in loads of trades in 9 periods and markedly few trades in the other 36 periods??

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/12/2019 at 8:29 PM #107388

GraHal

hahaha solved it … I had a Quit if strategyprofit < 500.

Ill run it again with the Quit rem’d out! 🙂

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times

09/12/2019 at 9:27 PM #107391

Vonasi

GraHal wrote:

is there any point doing more than 100 combinations

Probably not as 100 is already an awful lot of tests – far more than the one that we usually base our decision on!

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/12/2019 at 9:59 PM #107393

Vonasi

GraHal – when I click on your link to access the Google sheet it says I need to request access. Could you unlock it for everybody?

Vonasi
Moderator

Topics: 192
Replies: 5088
Been thanked: 1811 times

09/12/2019 at 10:15 PM #107394

GraHal

Done … try it now (I thought I had unlocked it!).

Btw I agree with you … better / easier to have a separate sheet(s) for different Qty / Random ranges.

GraHal
Participant

Topics: 211
Replies: 8760
Been thanked: 2176 times
Author

Posts

Viewing 15 posts - 31 through 45 (of 162 total)

← 1 2 3 4 5 6 7 8 9 10 11 →

Find exclusive trading pro-tools on

Strategy Robustness Tester

1 user thanked author for this post.

1 user thanked author for this post.

1 user thanked author for this post.