Hybrid NSGA-II Optimization for Improving the Three-Term BP Network for Multiclass Classification Problems

Authors

  • Ashraf Osman Ibrahim Faculty of Computer and Technology, Alzaiem Alazhari University,Khartoum North, Sudan
  • Siti Mariyam Shamsuddin UTM Big Data Centre, Universiti Teknologi Malaysia, Malaysia
  • Sultan Noman Qasem College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Saudi Arabia

DOI:

https://doi.org/10.32890/jict2015.14.2

Keywords:

Artificial Neural Network, hybridization technique, genetic algorithm, NSGA-II, multiobjective optimization

Abstract

Recently, hybrid algorithms have received considerable attention from a number of researchers. This paper presents a hybrid of the multiobjective evolutionary algorithm to gain a better accuracy of the fi nal solutions. The aim of using the hybrid algorithm is to improve the multiobjective evolutionary algorithm performance in terms of the enhancement of all the individuals in the population and increase the quality of the Pareto optimal solutions. The multiobjective evolutionary algorithm used in this study is a nondominated sorting genetic algorithm-II (NSGA-II) together with its hybrid, the backpropagation algorithm (BP), which is used as a local search algorithm to optimize the accuracy and complexity of the three-term backpropagation (TBP) network. The outcome positively demonstrates that the hybrid algorithm is able to improve the classification performance with a smaller number of hidden nodes and is effective in multiclass classifi cation problems. Furthermore, the results indicate that the proposed hybrid method is a potentially useful classifi er for enhancing the classification process ability when compared with the multiobjective genetic algorithm based on the TBP network (MOGATBP) and certain other methods found in the literature.

 

References

Abbass, H. A. (2002). An evolutionary artificial neural networks approach for breast cancer diagnosis. Artificial Intelligence in Medicine, 25(3), 265-281.

Abbass, H. A. (2003). Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Computation, 15(11), 2705-2726.

Adam, A., Chew, L. C., Shapiai, M. I., Jau, L. W., Ibrahim, Z., & Khalid, M. (2011). A hybrid artificial neural network-naive bayes for solving imbalanced dataset problems in semiconductor manufacturing test process. In Hybrid Intelligent Systems (HIS), 2011 11th International Conference on (pp. 133-138). IEEE.

Ahmad, F., Isa, N. A. M., Hussain, Z., & Sulaiman, S. N. (2012). A genetic algorithm-based multi-objective optimization of an artificial neural network classifier for breast cancer diagnosis. Neural Computing and Applications, 23(5), 1427-1435. Journal of ICT, 14, 2015, pp: 21–

Ak, R., Li, Y., Vitelli, V., Zio, E., Droguett, E. L., & Jacinto, C. M. C. (2012). NSGA-II-Trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Expert Systems with Applications,40(4), 1205-1212.

Almeida, L. M., & Ludermir, T. B. (2010). A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks. Neurocomputing, 73(7), 1438-1450.

Asuncion, Arthur, & Newman, David. (2007). UCI machine learning repository.

Bonissone, P. P., Chen, Y. T., Goebel, K., & Khedkar, P. S. (1999). Hybrid soft computing systems: industrial and commercial applications. Proceedings of the IEEE, 87(9), 1641-1667.

Collins, M. (2002). Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1–8.

Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., & Singer, Y. (2006). Online passive-aggressive algorithms. The Journal of Machine Learning Research (JMLR), 7, 551–585.

Cruz-Ramı, M., Hervás-Martı, C., Fernández, J. C., Briceño, J., & de la Mata, M. (2012). Multiobjective evolutionary algorithm for donor-recipient decision system in liver transplants. European Journal of Operational Research, 222(2), 317-327.

Dekel, O., Keshet, J., & Singer, Y. (2004). Large margin hierarchical classification. In Proceedings of the 21st International Conference on Machine Learning (ICML), 27.

Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., & Weischedel, R. (2004). The automatic content extraction (ACE) program– tasks, data, and evaluation. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), 837–840.

Finkel, J., Grenager, T., & Manning, C. (2005). Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL), 363–370.

Fleischman, M. (2001). Automated subcategorization of named entities. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (ACL), 25–30.

Fleischman, M., & Hovy, E. (2002). Fine-grained classification of named entities. In Proceedings of the 19th international Conference on Computational Linguistics (COLING), 1–7. Journal of ICT, 14, 2015, pp: 21–

Florian, R., Ittycheriah, A., Jing, H., & Zhang, T. (2003). Named entity recognition through classifier combination. In Proceedings of the 7th Conference on Natural Language Learning (CoNLL-2003), 168–171.

Freund, Y., & Schapire, R. (1999). Large margin classification using the perceptron algorithm. Machine learning, 37(3),277–296.

Grishman, R., & Sundheim, B. (1996). Message understanding conference-6: a brief history. In Proceedings of the 16th International Conference on Computational Linguistics (COLING), 466–471.

Isozaki, H., & Kazawa, H. (2002). Efficient support vector classifiers for named entity recognition. In Proceedings of the 19th International Conference on Computational Linguistics, 1–7.

Johansson, R.< & Nugues, P. (2008). Dependency-based semantic role labeling of propbank. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP), 69–78.

Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. A. M. T. (2002). A fast and elitist multiobjective genetic algorithm: NSGA-II. Evolutionary Computation, IEEE Transactions on, 6(2), 182-197.

Delgado, M., Cuéllar, M. P., & Pegalajar, M. C. (2008). Multiobjective hybrid optimization and training of recurrent neural networks. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 38(2), 381-403.

Egrioglu, E., Aladag, C. H., & Yolcu, U. (2013). Fuzzy time series forecasting with a novel hybrid approach combining fuzzy c-means and neural networks. Expert Systems with Applications, 40(3), 854-857.

Fan, C. Y., Chang, P. C., Lin, J. J., & Hsieh, J. C. (2011). A hybrid model combining case-based reasoning and fuzzy decision tree for medical data classification. Applied Soft Computing, 11(1), 632-644.

Fernandez, C., J. C., Martínez, F. J., Hervás, C., & Gutiérrez, P. A. (2010). Sensitivity versus accuracy in multiclass problems using memetic Pareto evolutionary neural networks. Neural Networks, IEEE Transactions on, 21(5), 750-770.

Fernández, J. C., Hervás, C., Martínez-Estudillo, F. J., & Gutiérrez, P. A. (2011). Memetic Pareto Evolutionary Artificial Neural Networks to determine growth/no-growth in predictive microbiology. Applied Soft Computing, 11(1), 534-550.

Garcıa-Pedrajas, N., Ortiz-Boyer, D., & Hervás-Martınez, C. (2004). Cooperative coevolution of generalized multi-layer perceptrons. Neurocomputing, 56, 257-283.

García-Pedrajas, N., Hervás-Martínez, C., & Ortiz-Boyer, D. (2005). Cooperative coevolution of artificial neural network ensembles for pattern classification. Evolutionary Computation, IEEE Transactions on, 9(3), 271-302. Journal of ICT, 14, 2015, pp: 21–

Goh, C. K., Teoh, E. J., & Chen Tan, K. (2008). Hybrid multiobjective evolutionary design for artificial neural networks. Neural Networks, IEEE Transactions on, 19(9), 1531-1548.

Hasan, S., Quo, T. S., & Shamsuddin, S. M. (2012). Artificial fish swarm optmization for multilayer network learning in classification problems. Journal of Information & Communication Technology, 11, 37-53.

Ibrahim, A. O., Hasan, S., & Noman, S. (2014). Memetic Elitist Pareto evolutionary algorithm of three-term backpropagation network for classification problems. Int. J. Advance Soft Compu. Appl, 6(3).

Ibrahim, A. O., Shamsuddin, S. M., Ahmad, N. B., & Qasem, S. N. (2013). Three-Term Backpropagation Network based on elitist multiobjective genetic algorithm for medical diseases diagnosis classification. Life Science Journal, 10(4), 1815-1822.

Ibrahim, A. O., Shamsuddin, S. M., & Bahiah, N. (2013). Multi objective genetic algorithm for training three term backpropagation network. 32-38.

Idris, I., Selamat, A., & Omatu, S. (2014). Hybrid email spam detection model with negative selection algorithm and differential evolution. Engineering Applications of Artificial Intelligence, 28, 97-110.

Isa, N. A. M., Mashor, M. Y., Othman, N. H., & Zamli, K. Z. (2005). Application of articial neural networks in the classification of cervical cells based on the Bethesda System. Journal of Information & Communication Technology, 4, 77-97.

Ishibuchi, H., Hitotsuyanagi, Y., Tsukamoto, N., & Nojima, Y. (2009). Use of biased neighborhood structures in multiobjective memetic algorithms. Soft Computing, 13(8-9), 795-810.

Jin, Y., Sendhoff, B., & Körner, E (2006). Simultaneous generation of accurate and interpretable neural network classifiers. In Multi-Objective Machine Learning (pp. 291-312): Springer Berlin Heidelberg.

Karegowda, A. G., Manjunath, A. S., & Jayaram, M. A. (2011). Application of genetic algorithm optimized neural network connection weights for medical diagnosis of pima Indians diabetes. International Journal on Soft Computing, 2(2), 15-23.

Khashei, M., Hamadani, A. Z., & Bijari, M. (2012). A novel hybrid classification model of artificial neural networks and multiple linear regression models. Expert Systems with Applications, 39(3), 2606-2620.

Lara, A., Sanchez, G., Coello Coello, C. A., & Schutze, O. (2010). HCS: A new local search strategy for memetic multiobjective evolutionary algorithms. Evolutionary Computation, IEEE Transactions on, 14(1), 112-132.

Neri, F., & Cotta, C. (2012). Memetic algorithms and memetic computing optimization: A literature review. Swarm and Evolutionary Computation, 2, 1-14. Journal of ICT, 14, 2015, pp: 21–

Pettersson, F., Chakraborti, N., & Saxén, H. (2007). A genetic algorithms based multi-objective neural net applied to noisy blast furnace data. Applied Soft Computing, 7(1), 387-397.

Qasem, S. N., & Shamsuddin, S. M. (2011). Radial basis function network based on time variant multi-objective particle swarm optimization for medical diseases diagnosis. Applied Soft Computing, 11(1), 1427-1438.

Qasem, S. N., Shamsuddin, S. M., Hashim, S. Z. M., Darus, M., & Al-Shammari, E. (2013). Memetic multiobjective particle swarm optimization-based radial basis function network for classification problems. Information Sciences, 239, 165-190.

Qasem, S. N., Shamsuddin, S. M., & Zain, A. M. (2012). Multi-objective hybrid evolutionary algorithms for radial basis function neural network design. Knowledge-Based Systems, 27, 475-497.

Qiongshuai, L., & Shiqing, W. (2011). A hybrid model of neural network and classification in wine. In Computer Research and Development (ICCRD), 2011 3rd International Conference on (Vol. 3, pp. 58-61). IEEE.

Ramesh, S., Kannan, S., & Baskar, S. (2011). Application of modified NSGA-II algorithm to multi-objective reactive power planning. Applied Soft Computing, 12(2), 741-753.

Srinivas, N., & Deb, K. (1994). Muiltiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary computation, 2(3), 221-248.

Tang, X. (2009). Hybrid Hidden Markov Model and artificial neural network for automatic speech recognition. Paper presented at the Circuits, In Circuits, Communications and Systems, 2009. PACCS’09. Pacific-Asia Conference on (pp. 682-685). IEEE.

Wiegand, S., Igel, C., & Handmann, U. (2004). Evolutionary multi-objective optimisation of neural networks for face detection. International Journal of Computational Intelligence and Applications, 4(03), 237-253.

Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423-1447.

Zhou, A., Qu, B. Y., Li, H., Zhao, S. Z., Suganthan, P. N., & Zhang, Q. (2011). Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm and Evolutionary Computation, 1(1), 32-49.

Zweiri, Y. H., Whidborne, J. F., & Seneviratne, L. D. (2003). A three-term backpropagation algorithm. Neurocomputing, 50, 305-318.

Downloads

Published

28-04-2015

Issue

Vol. 14 (2015): Journal of Information and Communication Technology (JICT) Vol. 14, 2015

Section

Articles

How to Cite

Ibrahim, A. O., Shamsuddin, S. M., & Qasem, S. N. (2015). Hybrid NSGA-II Optimization for Improving the Three-Term BP Network for Multiclass Classification Problems. Journal of Information and Communication Technology, 14, 21-38. https://doi.org/10.32890/jict2015.14.2

Research impact

Harvested 2026-09-22
7 citations, from OpenAlex — the highest of the sources checked

Counts differ between services because each indexes a different body of literature. None of them is the whole picture.

Identifiers DOI 10.32890/jict2015.14.2 OpenAlex W4242180789 Scopus 85182135505

Most read articles by the same author(s)