Feature Selection For Ensembles

Abstract

The traditional motivation behind feature selection algorithms is to find the best subset of features for a task using one particular learning algorithm. Given the recent success of ensembles, however, we investigate the notion of ensemble feature selection. This task is harder than traditional feature selection in that one not only needs to find features germane to the learning task and learning algorithm, but one also needs to find a set of feature subsets that will promote disagreement among the ensemble’s classifiers. In this paper, we present an ensemble feature selection approach that is based on genetic algorithms. Though conceptually simple, our algorithm shows improved performance over the popular and powerful ensemble approaches of Bagging and Ada-boosting and demonstrates the utility of ensemble feature selection.