Estimation Of (near) Low-rank Matrices With Noise And High-dimensional Scaling

Abstract

We study an instance of high-dimensional statistical inference inwhich the goal is to use $N$ noisy observations to estimate a matrix$\Theta^* \in \real^{k \times p}$ that is assumed to be either exactlylow rank, or "near" low-rank, meaning that it can bewell-approximated by a matrix with low rank. We consider an$M$-estimator based on regularization by the trace or nuclear normover matrices, and analyze its performance under high-dimensionalscaling. We provide non-asymptotic bounds on the Frobenius norm errorthat hold for a general class of noisy observation models, and applyto both exactly low-rank and approximately low-rank matrices. We thenillustrate their consequences for a number of specific learningmodels, including low-rank multivariate or multi-task regression,system identification in vector autoregressive processes, and recoveryof low-rank matrices from random projections. Simulations showexcellent agreement with the high-dimensional scaling of the errorpredicted by our theory.