Halving GridSearch

The next best alternative to the exhaustive cousins Random & GridSearch

hey there, how things are holding? i hope its all fine. Hey today i bring you something new and very useful indeed called Halving Grid Search for hyperparameters tuning.

Halving Grid Search, a new class of successive Halving, where training is performed on the subsets of data, rather than on all the data. The worst performing data are filtered out by training them on a small subset of data. After N number iterations select the best data/candidates leading to a faster evaluation time.

In General, the Halving GridSearch performs 11x times faster than our regular GridSearch.

According to sklearn,

HalvingGridSearch: The search strategy starts evaluating all the candidates with a small amount of resources and iteratively selects the best candidates, using more and more resources.

Parameter list:~

class sklearn.model_selection.HalvingGridSearchCV(estimator, param_grid, *, factor=3, resource='n_samples',                                                max_resources='auto', min_resources='exhaust', aggressive_elimination=False, cv=5, scoring=None, refit=True, error_score=nan, return_train_score=True, random_state=None, 
n_jobs=None, verbose=0)

Let’s understand on how to apply Halving GridSEarch with a sample.

Halving GridSEarch with a sample.

Out[9]: {‘max_depth’: None, ‘min_samples_split’: 5, ‘n_estimators’: 9}

Here we are the output, the optimized hyperparameter.

Let’s also explore another variant of Halving GridSearch

HalvingRandomSearchCV

“The candidates are sampled at random from the parameter space and the number of sampled candidates is determined by n_candidates.”

HalvingRandomSearchCV

Out[10]: {‘max_depth’: 3, ‘min_samples_split’: 3, ‘n_estimators’: 9}

So that's it…………i hope you enjoyed this short and useful article.

Next, we will look into PowerTransformers to handle highly skewed data.

Thanks again, for your time, if you enjoyed this short article there are tons of topics in advanced analytics, data science, and machine learning available in my medium repo. https://medium.com/@bobrupakroy

Some of my alternative internet presences Facebook, Instagram, Udemy, Blogger, Issuu, Slideshare, Scribd and more.

Also available on Quora @ https://www.quora.com/profile/Rupak-Bob-Roy

Let me know if you need anything. Talk Soon.