Pertanika Journal of Science & Technology
Pertanika Journal Home Facebook
Pertanika · Universiti Putra Malaysia Press

Pertanika Journal of Science & Technology

Official journal of Universiti Putra Malaysia for scholarly work across science, engineering and related technologies.

e-ISSN 2231-8526 ISSN 0128-7680
Browse latest issue Author guidelines Submit Here
Research article

Penalized LAD-SCAD Estimator Based on Robust Wrapped Correlation Screening Method for High Dimensional Models

Ishaq Abdullahi Baba, Habshah Midi, Leong Wah June and Gafurjan Ibragimove

Volume 29, Issue 2 April 2021 Published 30 April 2021
https://doi.org/10.47836/pjst.29.2.19
KeywordsLAD-SCAD estimators, robust screening, ultrahigh dimensional data, variable selection
Article content

Abstract

The widely used least absolute deviation (LAD) estimator with the smoothly clipped absolute deviation (SCAD) penalty function (abbreviated as LAD-SCAD) is known to produce corrupt estimates in the presence of outlying observations. The problem becomes more complicated when the number of predictors diverges. To overcome these problems, the LAD-SCAD based on sure independence screening (SIS) technique is put forward. The SIS method uses the rank correlation screening (RCS) algorithm in the pre-screening step and the traditional Pathwise coordinate descent algorithm for computing the sequence of the regularization parameters in the post screening step for onward model selection. It is now evident that the rank correlation is less robust against outliers. Motivated by these inadequacies, we propose to improvise the LAD-SCAD estimator using robust wrapped correlation screening (WCS) method by replacing the rank correlation in the SIS method with robust wrapped correlation. The proposed estimator is denoted as WCS+LAD-SCAD and will be employed for variable selection. The simulation study and real-life data examples show that the proposed procedure produces more efficient results compared to the existing methods.
Supporting literature

References

  1. Ahmed, T., & Bajwa, W. U. (2019). ExSIS: Extended sure independence screening for ultrahigh-dimensional linear models. Signal Processing, 159, 33-48. https://doi.org/10.1016/j.sigpro.2019.01.018
  2. Arslan, O. (2012). Weighted LAD-LASSO method for robust parameter estimation and variable selection in regression. Computational Statistics & Data Analysis, 56(6), 1952-1965. https://doi.org/10.1016/j.csda.2011.11.022
  3. Bai, Z. D., & Wu, Y. (1997). General M-estimation. Journal of Multivariate Analysis, 63(1), 119-135. https://doi.org/10.1006/jmva.1997.1694
  4. Brown, P. J., Fearn, T., & Vannucci, M. (2001). Bayesian wavelet regression on curves with application to a spectroscopic calibration problem. Journal of the American Statistical Association, 96(454), 398-408. https://doi.org/10.1198/016214501753168118
  5. Candes, E., & Tao, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n. The annals of Statistics, 35(6), 2313-2351. https://doi.org/10.1214/009053606000001523
  6. Chang, L., Roberts, S., & Welsh, A. (2018). Robust Lasso Regression Using Tukey’s Biweight Criterion. Technometrics, 60(1), 36-47. https://doi.org/10.1080/00401706.2017.1305299
  7. Croux, C., & Dehon, C. (2010). Influence functions of the Spearman and Kendall correlation measures. Statistical Methods & Applications, 19(4), 497-515. https://doi.org/10.1007/s10260-010-0142-z
  8. Desboulets, L. D. D. (2018). A review on variable selection in regression analysis. Econometrics, 6(4), Article 45. https://doi.org/10.3390/econometrics6040045
  9. Dhhan, W., Rana, S., & Midi, H. (2017). A high breakdown, high efficiency and bounded influence modified GM estimator based on support vector regression. Journal of Applied Statistics, 44(4), 700-714. https://doi.org/10.1080/02664763.2016.1182133
  10. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348-1360. https://doi.org/10.1198/016214501753382273
  11. Fan, J., & Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5), 849-911. https://doi.org/10.1111/j.1467-9868.2008.00674.x
  12. Fan, J., & Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. The Annals of Statistics, 32(3), 928-961. https://doi.org/10.1214/009053604000000256
  13. Fan, J., & Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. The Annals of Statistics, 38(6), 3567-3604.
  14. Fan, J., Samworth, R., & Wu, Y. (2009). Ultrahigh dimensional feature selection: beyond the linear model. The Journal of Machine Learning Research, 10, 2013-2038.
  15. Frank, L. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109-135.
  16. Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1), 1.
  17. Gao, X., & Huang, J. (2010). Asymptotic analysis of high-dimensional LAD regression with LASSO. Statistica Sinica, 1485-1506.
  18. George, E. I. (2000). The variable selection problem. Journal of the American Statistical Association, 95(452), 1304-1308.
  19. Ghaoui, L. E., Viallon, V., & Rabbani, T. (2010). Safe feature elimination for the lasso and sparse supervised learning problems. Machine Learning, 2000, 1-31.
  20. Heinze, G., Wallisch, C., & Dunkler, D. (2018). Variable selection–a review and recommendations for the practicing statistician. Biometrical Journal, 60(3), 431-449. https://doi.org/10.1002/bimj.201700067
  21. Huang, J., & Xie, H. (2007). Asymptotic oracle properties of SCAD-penalized least squares estimators. In Asymptotics: Particles, Processes and Inverse Problems (pp. 149-166). Institute of Mathematical Statistics. https://doi.org/10.1214/074921707000000337
  22. Hubert, M., & Van der Veeken, S. (2008). Outlier detection for skewed data. Journal of Chemometrics: A Journal of the Chemometrics Society, 22(3‐4), 235-246. https://doi.org/10.1002/cem.1123
  23. Hubert, M., Rousseeuw, P. J., & Branden, K. V. (2005). ROBPCA: a new approach to robust principal component analysis. Technometrics, 47(1), 64-79. https://doi.org/10.1198/004017004000000563
  24. Leng, C., Lin, Y., & Wahba, G. (2006). A note on the lasso and related procedures in model selection. Statistica Sinica, 1273-1284.
  25. Li, G., Peng, H., & Zhu, L. (2011). Nonconcave penalized M-estimation with a diverging number of parameters. Statistica Sinica, 391-419.
  26. Li, R., Zhong, W., & Zhu, L. (2012). Feature screening via distance correlation learning. Journal of the American Statistical Association, 107(499), 1129-1139. https://doi.org/10.1080/01621459.2012.695654
  27. Liebmann, B., Friedl, A., & Varmuza, K. (2009). Determination of glucose and ethanol in bioethanol production by near infrared spectroscopy and chemometrics. Analytica Chimica Acta, 642(1-2), 171-178. https://doi.org/10.1016/j.aca.2008.10.069
  28. Liu, J., Wang, Y., Fu, C., Guo, J., & Yu, Q. (2016). A robust regression based on weighted LSSVM and penalized trimmed squares. Chaos, Solitons & Fractals, 89, 328-334. https://doi.org/10.1016/j.chaos.2015.12.012
  29. Maronna, R. A., Martin, R. D., & Yohai, V. J. (2006). Robust statistics: Theory and methods (with R). John Wiley & Sons.
  30. Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3), 1436-1462. https://doi.org/10.1214/009053606000000281
  31. Raymaekers, J., & Rousseeuw, P. J. (2019). Fast robust correlation for high-dimensional data. Technometrics, 1-15. https://doi.org/10.1080/00401706.2019.1677270
  32. Rousseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier detection. Wiley.
  33. Saldana, D. F., & Feng, Y. (2018). SIS: An R package for sure independence screening in ultrahigh dimensional statistical models. Journal of Statistical Software, 83(2), 1-25. https://doi.org/10.18637/jss.v083.i02
  34. Shevlyakov, G., & Smirnov, P. (2011). Robust estimation of the correlation coefficient: An attempt of survey. Austrian Journal of Statistics, 40(1&2), 147-156. https://doi.org/10.17713/ajs.v40i1&2.206
  35. Stuart, C. (2011). Robust regression. Durham University.
  36. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
  37. Tibshirani, R., Bien, J., Friedman, J., Hastie, T., Simon, N., Taylor, J., & Tibshirani, R. J. (2012). Strong rules for discarding predictors in lasso‐type problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266. https://doi.org/10.1111/j.1467-9868.2011.01004.x
  38. Uraibi, H. S., Midi, H., & Rana, S. (2017). Selective overview of forward selection in terms of robust correlations. Communications in Statistics: Simulation and Computation, 46(7), 5479-5503. https://doi.org/10.1080/03610918.2016.1164862
  39. Wang, H., Li, G., & Jiang, G. (2007). Robust regression shrinkage and consistent variable selection through the LAD-Lasso. Journal of Business & Economic Statistics, 25(3), 347-355. https://doi.org/10.1198/073500106000000251
  40. Wang, M., Song, L., & Tian, G. L. (2015). SCAD-penalized least absolute deviation regression in high-dimensional models. Communications in Statistics-Theory and Methods, 44(12), 2452-2472. https://doi.org/10.1080/03610926.2013.781643
  41. Wang, T., & Zhu, L. (2011). Consistent tuning parameter selection in high dimensional sparse linear regression. Journal of Multivariate Analysis, 102(7), 1141-1151. https://doi.org/10.1016/j.jmva.2011.03.007
  42. Whittingham, M. J., Stephens, P. A., Bradbury, R. B., & Freckleton, R. P. (2006). Why do we still use stepwise modelling in ecology and behaviour? Journal of Animal Ecology, 75(5), 1182-1189. https://doi.org/10.1111/j.1365-2656.2006.01141.x
  43. Wu, Y., & Liu, Y. (2009). Variable selection in quantile regression. Statistica Sinica, 19(2), 801-817.
  44. Xiang, Z. J., & Ramadge, P. J. (2012). Fast lasso screening tests based on correlations. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 2137-2140). IEEE Conference Publication. https://doi.org/10.1109/ICASSP.2012.6288334
  45. Xie, H., & Huang, J. (2009). SCAD-penalized regression in high-dimensional partially linear models. The Annals of Statistics, 37(2), 673-696. https://doi.org/10.1214/07-AOS580
  46. Zhang, Y., Li, R., & Tsai, C. L. (2010). Regularization parameter selections via generalized information criterion. Journal of the American Statistical Association, 105(489), 312-323. https://doi.org/10.1198/jasa.2009.tm08013
  47. Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418-1429. https://doi.org/10.1198/016214506000000735
  48. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (statistical methodology), 67(2), 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License .