Bootstrap selection of ridge regularization parameter: a comparative study via a simulation study

Özkale M. R., Altuner H.

COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION, 2021 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2021
  • Doi Numarası: 10.1080/03610918.2021.1948574
  • Dergi Adı: COMMUNICATIONS IN STATISTICS-SIMULATION AND COMPUTATION
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Applied Science & Technology Source, Business Source Elite, Business Source Premier, CAB Abstracts, Compendex, Computer & Applied Sciences, Veterinary Science Database, zbMATH, Civil Engineering Abstracts
  • Anahtar Kelimeler: Bootstrap, Ridge regression, Regularization parameter, Signal-to-noise ratio, Mean square error, MONTE-CARLO, BIASED-ESTIMATORS, CROSS-VALIDATION, REGRESSION, PERFORMANCE
  • Çukurova Üniversitesi Adresli: Evet

Özet

In multiple linear regressions, it is known that least-squares estimates of the parameters are likely to be too large in absolute value and possibly of wrong sign, if explanatory variables are correlated. To reduce the undesirable effects of collinearity, the ridge estimator has been proposed as an alternative method to the least squares estimator. The biggest debate with the ridge estimator is the selection of the regularization parameter. Several methods for the selection of the regularization parameter have been discussed. Although most of these methods are based on minimizing the mean square error of the ridge estimator, bootstrap resampling method is also proposed to provide an optimal regularization parameter which gives an estimate of mean square error of prediction. The purpose of this paper is to provide a comprehensive study of regularization parameter selection methods which consider bootstrap approach as well as estimation and prediction approaches. The paper concludes with application of these procedures to simulation studies and real data.