Multi-Class Discriminant Analysis Based on SVM Ensembles for Ranking Principal Components
Tiene Filisbino1; Gilson Giraldi1; Carlos Thomaz2; Diego Leite1
1 National Laboratory Scientific Computing; 2 Centro Educacional - FEI
doi:10.20906/CPS/CILAMCE2015-0375
Resumo
The problem of ranking components obtained by subspace learning techniques depends on the definition of a criterium to quantify the importance of a given feature. This problem is very known in the context of principal component analysis (PCA). In this case, it was observed that, since PCA explains the covariance structure of all the data its most expressive components, that is, the first principal components with the largest eigenvalues, do not necessarily represent the most important discriminant directions to separate sample groups [1]. This observation motivates the application and development of other techniques, like the discriminant principal components analysis (DPCA), to identify the most important linear directions for recognition tasks rather than PCA. Instead of sorting the principal components in decreasing order of the corresponding eigenvalues, the DPCA uses the discriminant weights given by separating hyperplanes to select among the principal components the most discriminant ones. Originally, the DPCA is a two-class discriminant feature technique. In this paper we extend the DPCA for multi-class problems. The multi-class DPCA pipeline consists of the following steps: (a) apply PCA technique for dimension- ality reduction in order to eliminate redundancy. (b) Compute an support vector machine (SVM) ensemble, based on the "one-against-all" SVM multi-class approach. (c) Combine the discriminate weights computed through the separating SVM hyperplanes in order to determine the discriminant contribution of each feature. So, given a N-class database, the step (b) builds N SVM machines in the PCA space. We implement the step (c) by adapting ensemble techniques [2] to yield a global discriminant vector to sort PCA components in decreasing order of the corresponding discriminant weights. The method is not restricted to any particular probability density function of the sample groups because it can be based on either a parametric or non- parametric separating hyperplane approach. In
Palavras-chave: Discriminant Analysis; Principal Components Analysis; Support Vector Machine; Ensemble Methods; AdaBoost