Overview of reduced-rank regression with dense and sparse coefficients, and a new estimation procedure

The reduced-rank regression (RRR) is a specific multivariate (linear) regression problem when the coefficient matrix is required to be rank deficient, i.e. . The available RRR methods factorize C in a number of ways in order to impose certain rank on C. The main such methods are reviewed in the first part of the paper. In addition, a new factorization-free RRR method is proposed based on the null-space of C. This approach is both computationally efficient, and clarifies the RRR geometry very well. The second part of the paper reviews the most popular RRR methods producing sparse regression coefficients C. They all achieve this goal by tuning parameters. The paper investigates an alternative of achieving sparse C by imposing cardinality constraints on its columns and/or rows. This tuning-free approach is computationally very simple and fast, and achieves good level of sparseness on the expense of little loss of fit. The proposed new approach for sparse C is illustrated and compared to existing methods on simulated and real data sets.