#051: HOW MANY OPTIMIZATION INPUTS IS STILL ACCEPTABLE?
In today’s article, I would like to share with you one interesting study, which will answer to you one important question – how many optimization inputs are acceptable for a strategy and what are the limits we should not cross? Let’s take a look at it.
The study is based on an excellent work of my colleague in our hedge fund, who takes care of an automated strategy development process, the database and performing of analytical tasks in this database. Let’s first take a look at the methodology how was this study performed.
In our hedge fund, there is currently over 700 strategies, most of them on futures markets – intraday and swing ones. All of these strategies have already passed our tests, so they all meet our minimum quality requirements and are usable for live trading.
This is where all the work doesn’t end, but where it all begins. All our strategies are continuously monitored and updated, so we receive new important information about all strategies and their performance. So we have available not only the 3-month period of additional out of sample, but even the real out of sample – i.e. data that haven’t even existed when the strategy was developed. This gives us unique possibility to monitor the real OOS performance and compare it to the previous performance.
Further on, as part of our workflow we have created an index, that monitors the real OOS performance to all previous data we had available (thanks to fantastic work of my colleague) and this index (which is unfortunately private and I won’t share any more information about it) helps us to analyze what factors impact the real OOS performance. Today we analyze really big quantity of different aspects and components (plus we use Python Jupiter).
One of these studies was performed in order to show us what is the relationship between the OOS performance and the number of optimization inputs.
And this is the study I would like to share with you today.
The results are really simple for interpretation – simply, the higher avgDhidx value, the better is the performance of strategies with given number of parameters. We have compared 1-7 input parameters (none of our strategies have more than that) and here are the results:
Even though the study is not perfect as yet, as we don’t have enough samples for certain variants (we are developing new strategies every day, so the sample size is continuously growing), it is possible to state some general results:
It seems that having more than 6 optimization inputs is dangerous.
It is the only level in our test that has negative index value.
The range from 2 to 6 seems reasonable (number 4 is a certain anomaly, which we need to investigate further later on).
Having just one optimization seems not sufficient, which is understandable when looking at the complexity of markets in these days.
If you want to keep the number of optimization inputs low (for example due to the number of iterations), it is better to choose 2-3 optimization inputs.
For me personally, the most important is to see and to confirm, whether a strategy with fewer optimization inputs, is more robust. Our study hasn’t confirmed that. You can have even 5-6 optimization inputs and as long as the strategy passes all our robustness tests, it can be as robust as the strategy which has 2-3 optimization inputs.
Happy Trading!
Tomas
