Feature Clustering for PSO-Based Feature Construction on High-Dimensional Data
DOI:
https://doi.org/10.32890/jict2019.18.4.12Keywords:
Particle swarm optimisation, feature construction, genetic programming, classification, high-dimensional dataAbstract
Feature construction (FC) refers to a process that uses the original features to construct new features with better discrimination ability. Particle Swarm Optimisation (PSO) is an effective search technique that has been successfully utilised in FC. However, the application of PSO for feature construction using high dimensional data has been a challenge due to its large search space and high computational cost. Moreover, unnecessary features that were irrelevant, redundant and contained noise were constructed when PSO was applied to the whole feature. The feature clustering methods were used to aggregate similar features into clusters, whereby the dimensionality of the data was lowered by choosing representative features from every cluster to form the final feature subset. The clustering of each features are proven to be accurate in feature selection (FS), however, only one study investigated its application in FC for classification. The study identified some limitations, such as the implementation of only two binary classes and the decreasing accuracy of the data. This paper proposes a cluster based PSO feature construction approach called ClusPSOFC. The Redundancy-Based Feature Clustering (RFC) algorithm was applied to choose the most informative features from the original data, while PSO was used to construct new features from those selected by RFC. Experimental results were obtained by using six UCI data sets and six high-dimensional data to demonstrate the efficiency of the proposed method when compared to the original full features, other PSO based FC methods, and standard genetic programming based feature construction (GPFC). Hence, the ClusPSOFC method is effective for feature construction in the classification of high dimensional data.
References
Ahmed, S., Zhang, M., Peng, L., & Xue, B. (2014). Multiple feature construction for effective biomarker identification and classification using genetic programming. Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, 249-256. doi:10.1145/2576768.2598292
Ahmed, S., Zhang, M., Peng, L., & Xue, B. (2016). A Multi-objective genetic programming biomarker detection approach in mass spectrometry data. In European Conference on the Applications of Evolutionary Computation, 106-122. doi.org/10.1007/978-3-319-31204-0_8
Banka, H., & Dara, S. (2015). A Hamming distance based binary particle swarm optimization (HDBPSO) algorithm for high dimensional feature selection, classification and validation. Pattern Recognition Letters, 52, 94-100. doi.org/10.1016/j.patrec.2014.10.007 Journal of ICT, 18, No. 4 (October) 2019, pp: 439-
Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16-28. doi. org/10.1016/j.compeleceng.2013.11.024
Chen, Q., Zhang, M., & Xue, B. (2017). Genetic programming with embedded feature construction for high-dimensional symbolic regression. Intelligent and Evolutionary Systems, 87-102. doi.org/10.1007/978-3-319-49049-6_7
Dai, Y., Xue, B., & Zhang, M. (2014). New representations in PSO for feature construction in classification. In European Conference on the Applications of Evolutionary Computation, 476-488. doi. org/10.1007/978-3-662-45523-4_39
Dash, M., & Liu, H. (1997). Feature selection for classification. Intelligent data analysis, 1(3), 131-156. doi.org/10.3233/IDA-1997-1302
Ding, C., & Peng, H. (2005). Minimum redundancy feature selection from microarray gene expression data. Journal of bioinformatics and computational biology, 3(02), 185-205. doi.org/10.1142/ S0219720005001004
Elola, A., Del Ser, J., Bilbao, M. N., Perfecto, C., Alexandre, E., & Salcedo-Sanz, S. (2017). Hybridizing cartesian genetic programming and harmony search for adaptive feature construction in supervised learning problems. Applied Soft Computing, 52, 760-770. doi.org/10.1016/j. asoc.2016.09.049
Fayyad, U., & Irani, K. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In 13th International Joint Conference on Artificial Intelligence, 2, 1022-1027. doi.org/hdl.handle. net/2014/35171
Frank, A., & Asuncion, A. (2010). UCI machine learning repository.
García, D., González, A., & Pérez, R. (2011). A two-step approach of feature construction for a genetic learning algorithm. In 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011), 1255-1262. doi: 10.1109/FUZZY.2011.6007576
Gunasundari, S., Janakiraman, S., & Meenambal, S. (2016). Velocity bounded boolean particle swarm optimization for improved feature selection in liver and kidney disease diagnosis. Expert Systems with Applications, 56, 28-47. doi.org/10.1016/j.eswa.2016.02.042
Gupta, A., Gupta, A., & Sharma, K. (2016, March). Clustering based feature selection methods from fMRI data for classification of cognitive states of the human brain. In 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), 3581-3584.
Hall, M. A., & Smith, L. A. (1998). Practical feature subset selection for machine learning. In Computer science’98 proceedings of the 21st Australasian computer science conference ACSC, 98, 181-191. doi.org/hdl.handle.net/10289/1512 Journal of ICT, 18, No. 4 (October) 2019, pp: 439-
Hsu, H. H., & Hsieh, C. W. (2010). Feature Selection via Correlation Coefficient Clustering. JSW, 5(12), 1371-1377. doi.org/10.4304/ jsw.5.12.1371-1377
Hu, L., Gao, W., Zhao, K., Zhang, P., & Wang, F. (2018). Feature selection considering two types of feature relevancy and feature interdependency. Expert Systems with Applications, 93, 423-434. doi. org/10.1016/j.eswa.2017.10.016
Jabbar, A. M., Ku-Mahamud, K. R., & Sagban, R. (2018). Ant-based sorting and ACO-based clustering approaches: A review. In 2018 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), 217-223. dio:10.1109/ISCAIE.2018.8405473
Jain, I., Jain, V. K., & Jain, R. (2018). Correlation feature selection based improved-binary particle swarm optimization for gene selection and cancer classification. Applied Soft Computing, 62, 203-215. doi. org/10.1016/j.asoc.2017.09.038
Jaskowiak, P. A., & Campello, R. J. (2015). A cluster-based hybrid feature selection approach. In 2015 Brazilian Conference on Intelligent Systems (BRACIS), 43-48. doi.org/10.1109/BRACIS.2015.14
Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In IEEE International Conference on Neural Networks, 4, 1942–1948. doi. org/10.1007/978-0-387-30164-8_630
Kennedy, J., & Eberhart, R. C. (1997). A discrete binary version of the particle swarm algorithm. In 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, 5, 4104-4108. doi.org/10.1109/ICSMC.1997.637339
Kumari, P., Rajeswari, K., & Vaithiyanathan, V. (2015). Correlation and clustering based efficient feature subset selection. Journal of Engineering Technology, 3, 135-144.
Mahanipour, A., & Nezamabadi-pour, H. (2017). Improved PSO-based feature construction algorithm using feature selection methods. In 2017 2nd Conference on Swarm Intelligence and Evolutionary Computation (CSIEC),1-5. dio : 10.1109/CSIEC.2017.7940173
Mahanipour, A., Nezamabadi-pour, H., & Nikpour, B. (2018, March). Using fuzzy-rough set feature selection for feature construction based on genetic programming. In 2018 3rd Conference on Swarm Intelligence and Evolutionary Computation (CSIEC), 1-6. dio: 10.1109/ CSIEC.2018.8405407
Moradi, P., & Rostami, M. (2015). Integration of graph clustering with ant colony optimization for feature selection. Knowledge-Based Systems, 84, 144-161. doi.org/10.1016/j.knosys.2015.04.007 Journal of ICT, 18, No. 4 (October) 2019, pp: 439-
Muharram, M. A., & Smith, G. D. (2004). Evolutionary feature construction using information gain and gini index. In European Conference on Genetic Programming (pp. 379-388). Springer, Berlin, Heidelberg. doi. org/10.1007/978-3-540-24650-3_36
Neshatian, K., Zhang, M., & Andreae, P. (2012). A filter approach to multiple feature construction for symbolic learning classifiers using genetic programming. IEEE Transactions on Evolutionary Computation, 16(5), 645-661. doi.org/10.1109/TEVC.2011.2166158
Nguyen, H. B., Xue, B., Liu, I., & Zhang, M. (2014). PSO and statistical clustering for feature selection: A new representation. In Asia-Pacific Conference on Simulated Evolution and Learning (pp. 569-581). Springer, Cham. doi.org/10.1007/978-3-319-13563-2_48
Roth, V., & Lange, T. (2004). Feature selection in clustering problems. In Advances in neural information processing systems (pp. 473-480).
Rutkowski, L. (2008). Computational intelligence: Methods and techniques. Springer Science & Business Media. dio: 10.1007/978-3-540-76288-1
Sahu, B., & Mishra, D. (2012). A novel feature selection algorithm using particle swarm optimization for cancer microarray data. Procedia Engineering, 38, 27-31. doi.org/10.1016/j.proeng.2012.06.005
Sardana, M., Agrawal, R. K., & Kaur, B. (2016). A hybrid of clustering and quantum genetic algorithm for relevant genes selection for cancer microarray data. International Journal of Knowledge-based and Intelligent Engineering Systems, 20(3), 161-173. dio: 10.3233/KES-160341
Setiono, R., & Liu, H. (1998). Fragmentation problem and automated feature construction. Proceedings of 10th International Conference on Tools with Artificial Intelligence (ICTA), 208-215. IEEE. doi.org/10.1109/ TAI.1998.744845
Song, Q., Ni, J., & Wang, G. (2013). A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Transactions on Knowledge and Data Engineering, 25(1), 1-14. doi.org/10.1109/ TKDE.2011.181
Swesi, I. M., & Bakar, A. A. (2017). An enhanced binary particle swarm optimization (EBPSO) algorithm based a v-shaped transfer function for feature selection in high dimensional data. International Journal of Advances in Soft Computing & Its Applications, 9(3), 217-238.
Tariq, H., Eldridge, E., & Welch, I. (2018). An efficient approach for feature construction of high-dimensional microarray data by random projections. PloS one, 13(4), e0196385. doi.org/10.1371/journal. pone.0196385 Journal of ICT, 18, No. 4 (October) 2019, pp: 439-
Tran, B., Xue, B., & Zhang, M. (2016). Genetic programming for feature construction and selection in classification on high-dimensional data. Memetic Computing, 8(1), 3-15. doi.org/10.1007/s12293-015-0173-y
Tran, B., Xue, B., & Zhang, M. (2017). Using feature clustering for GP-based feature construction on high-dimensional data. In European Conference on Genetic Programming, 210-226. doi.org/10.1007/978-3-319-55696-3_14
Tran, B., Zhang, M., & Xue, B. (2016). Multiple feature construction in classification on high-dimensional data using GP. In SSCI, 1-8. doi. org/10.1109/SSCI.2016.7850130
Vafaie, H., & De Jong, K. (1998). Feature space transformation using genetic algorithms. IEEE Intelligent Systems and their Applications, 13(2), 57-65. doi.org/10.1109/5254.671093
Wong, K. C. (2015). A short survey on data clustering algorithms. In 2015 Second International Conference on Soft Computing and Machine Intelligence (ISCMI), 64-68. doi.org/10.1109/ISCMI.2015.10
Xu, D., & Tian, Y. (2015). A comprehensive survey of clustering algorithms. Annals of Data Science, 2(2), 165-193. doi.org/10.1007/ s40745-015-0040-1
Xue, B., Zhang, M., Dai, Y., & Browne, W. N. (2013). PSO for feature construction and binary classification. Proceedings of the 15th annual conference on Genetic and evolutionary computation, 137-144. ACM. doi:10.1145/2463372.2463376
Xue, B., Zhang, M., & Browne, W. N. (2014). Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Applied Soft Computing, 18, 261-276. doi.org/10.1016/j. asoc.2013.09.018
Yang, Z. M., He, J. Y., & Shao, Y. H. (2013). Feature selection based on linear twin support vector machines. Procedia Computer Science, 17, 1039-1046.
Yazdani, S., Shanbehzadeh, J., & Hadavandi, E. (2017). MBCGP-FE: A modified balanced cartesian genetic programming feature extractor. Knowledge-Based Systems, 135, 89-98. doi.org/10.1016/j. knosys.2017.08.005
Zhang, Y., Gong, D. W., Sun, X. Y., & Guo, Y. N. (2017). A PSO-based multi-objective multi-label feature selection method in classification. Scientific reports, 7(1), 376. doi.org/10.1038/s41598-017-00416-0
Published
Issue
Section
How to Cite
Research impact
Harvested 2026-10-10Counts differ between services because each indexes a different body of literature. None of them is the whole picture.
- Semantic Scholar 0 not indexed View →
- OpenCitations 0 View →
- Scopus 0 not indexed View →
- Google Scholar no free count Search →
- Dimensions no free count Search →
2002 - 2020
