Constraint principal components for linear discrimination

In many modern data, the number of variables is much higher than the number of observations and the within-group scatter matrix is singular. Then, the Fisher's linear discriminant analysis (LDA) cannot be applied. The work considers a way to circumvent this problem by doing principal component analysis (PCA) enhanced with additional discriminating features. Two approaches are proposed: the original PCs are rotated to maximize the Fisher's LDA criterion, and second, penalized PCs are produced to achieve simultaneous dimension reduction and maximization of the Fisher's LDA criterion. Both approaches are illustrated and compared to other existing methods on several well known data sets.