ℓ1-regularized generalized least squares

S. Nobari, Kaveh and Gibberd, Alex (2026) ℓ1-regularized generalized least squares. Electronic Journal of Statistics, 20 (1). ISSN 1935-7524

Abstract

We study a simple ℓ1-regularized generalized least-squares (GLS) estimator for high-dimensional regressions with autocorrelated errors. The estimation procedure consists of three steps: performing a LASSO regression, fitting an autoregressive model to the realized residuals, and then running a second-stage LASSO regression on the rotated (whitened) data. We examine the theoretical performance of the method in a sub-Gaussian random-design setting, in particular assessing the impact of the rotation on the design matrix and how this impacts the estimation error of the procedure. We show that the GLS (and a feasible variant) maintains a smaller estimation error than an unadjusted LASSO regression when the errors are driven by an autoregressive process. A simulation study verifies the performance of the proposed method, demonstrating that the penalized (feasible) GLS-LASSO estimator performs on par with the LASSO in the case of white noise errors, whilst outperforming when the errors exhibit significant autocorrelation.