Evaluating the Effectiveness of Digital Product Advertisement Type using Machine Learning and Shapley Additive Explanations Analysis

Authors

  • Khinsa Fairuz Zahirah School of Information System, Universitas Ciputra Surabaya, Indonesia
  • Winarto Poernomo School of Business and Management, Universitas Ciputra Surabaya, Indonesia
  • Eddy Yusuf School of Information Technology, Universitas Ciputra Surabaya, Indonesia
  • Denny Bernardus School of Business and Management, Universitas Ciputra Surabaya, Indonesia

DOI:

https://doi.org/10.32890/jict2026.25.1.5

Keywords:

Ad type, digital advertising, machine learning, SHAP analysis

Abstract

Digital advertising continues to grow rapidly, yet advertisers face persistent uncertainty in selecting the most effective ad format for different campaign objectives and audience segments. Prior studies often rely on limited metrics or lack interpretability, making it difficult to explain why certain formats perform better. This study addresses this gap by evaluating ad format effectiveness using explainable machine learning. This study evaluates the effectiveness of image and video advertisements (ads) across five key performance indicators (KPIs): reach, impressions, link clicks, cost per click (CPC), and cost per thousand impressions (CPM). A dataset of 4,526 campaigns from Meta Ads Manager (July 2021–August 2024) was analysed using machine learning models integrated with Shapley Additive Explanations (SHAP). Model performance was assessed using Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and R-squared (R²). The results showed that video ads achieved higher reach and impressions with lower CPM. Meanwhile, image ads delivered lower CPCs and stronger click performance, particularly among users aged 13–17 and 55+. Aside from confirming established format patterns, this study made certain contributions by applying explainable machine learning in a large-scale, non-Western context and by clarifying the mechanisms behind ad performance. The findings offered actionable guidance; for example, video ads were optimal for awareness and visibility objectives. Additionally, image ads were preferable for engagement-driven campaigns.

Author Biographies

  • Winarto Poernomo, School of Business and Management, Universitas Ciputra Surabaya

    Winarto Poernomo, S.E., M.M. is the Campus Director at Universitas Ciputra Jakarta with over 10 years of experience in marketing and management education. He previously served as Vice Chairman at UC Makassar, where he led the campus to reach BEP within three years. With a Master's from UC Surabaya and currently pursuing a Doctorate, he brings academic depth and strategic insight. His focus areas include digital marketing, consumer behavior, and SME agility, which shape both his teaching and research.

  • Eddy Yusuf, School of Information Technology, Universitas Ciputra Surabaya

    Eddy Yusuf, Ph.D. is the Academic Director at Universitas Ciputra Jakarta, spearheading the integration of artificial intelligence and informatics into higher education curricula. He earned his Ph.D. in Physics from Florida State University and brings a scientific lens to digital transformation in education. A recipient of international grants and a former Vice Rector for internationalization, Dr. Yusuf is now focused on developing scalable AI-powered learning ecosystems and fostering data-driven innovation in academic programs.

  • Denny Bernardus, School of Business and Management, Universitas Ciputra Surabaya

    Prof. Dr. Ir. Denny Bernardus, M.M. is a professor of entrepreneurship and management at Universitas Ciputra Surabaya, with a strong interest in digital transformation within startups and family businesses. Holding a Doctorate in Management from Unika Widya Mandala, he bridges academia and industry through research on entrepreneurial education, digital business models, and corporate innovation. As a director within the Ciputra Group, his work emphasizes the role of technology and strategic marketing in driving sustainable enterprise growth.

References

Abu Bakar et al. (2025). AI-driven influencer and market analysis: A social network approach to measure e-commerce relationships. Journal of Information and Communication Technology, 24(3), 1–21. https://doi.org/10.32890/jict2025. 24.3.1

Al-Khafaji, A., & Karan, O. (2024). Explainable AI for predicting user behaviour in digital advertising. In Lecture Notes in Networks and Systems, (pp. 520–531). Springer. https://doi.org/10.1007/ 978-3-031-56728-5_43

Arboleda-Florez, M., & Castro-Zuluaga, C. (2023). Interpreting direct sales’ demand forecasts using SHAP values. Production, 33. https://doi.org/10.1590/0103-6513.20220035

Asdemir, K., Kumar, N., & Jacob, V. S. (2012). Pricing models for online advertising: CPM vs. CPC. Information Systems Research, 23(3), 804–822. https://doi.org/10.1287/isre.1110.0391

Balcan, M., & Sharma, D. (2024). Learning accurate and interpretable decision trees. Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research, 244, 288–307. https://proceedings.mlr.press/v244/balcan24a.html

Booth, D. (2019). Marketing analytics in the age of machine learning. Applied Marketing Analytics, 4(3), 214. https://doi.org/10.69554/krmg2945

Braun, M., de Langhe, B., Puntoni, S., & Schwartz, E. M. (2023). Leveraging digital advertising platforms for consumer research. SSRN Electronic Journal. https://doi.org/10.2139/ssrn. 4589567

Chavda, V., & Patel, S. (2024). Review on boosting online advertising ROI: Using machine learning. International Journal of Advanced Networking and Applications, 15(4), 69–73. https://www. researchgate.net/publication/378190219_Review_on_Boosting_Online_Advertising_ROI_Usi ng_Machine_Learning

Chicco, D., Warrens, M. J., & Jurman, G. (2021). The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression-analysis evaluation. PeerJ Computer Science, 7(5), e623. https://doi.org/10.7717/peerj-cs.623

Chumwatana, T. (2018). Comment analysis for product and service satisfaction from Thai customers’ review in social network. Journal of Information and Communication Technology, 17(2), 271–289. https://doi.org/10.32890/jict2018.17.2.8254

Cunha, B. M., & Barbosa, S. D. J. (2024). Evaluating the effectiveness of visual representations of SHAP values toward explainable artificial intelligence. Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (pp. 1-11). ACM. https://doi.org/10. 1145/3702038.3702093

Das, L. B., George, S. N., & Aprem, A. (2023). Artificial intelligence and machine learning: Theory and practice. IK International Pvt. Limited. https://books.google.co.id/books?id=K9oFEQAA QBAJ

de Carvalho, M. S., & da Silva, G. L. (2021). Inside the black box: Using explainable AI to improve evidence-based policies. 2021 IEEE 23rd Conference on Business Informatics (CBI) (pp. 57–64). IEEE. https://doi.org/10.1109/CBI52690.2021.10055

Farahat, A., & Shanahan, J. (2013). Econometric analysis and digital marketing. Proceedings of the Sixth ACM International Conference on Web Search and Data Mining (WSDM ’13). Association for Computing Machinery. https://doi.org/10.1145/2433396.2433502

Félegyházi, Z. (2024). Facebook hirdetések vizuális hatékonyságának növelése és helyzete az online hirdetések piacán. Acta Carolus Robertus, 14(1), 65–77. https://doi.org/10.33032/acr.4870

Grewal, R., Gupta, S., & Hamilton, R. (2021). Marketing insights from multimedia data: Text, image, audio, and video. Journal of Marketing Research, 58(6), 1025–1033. https://doi.org/10.1177/ 00222437211054601

Hamilton, R. I., & Papadopoulos, P. N. (2023). Using SHAP values and machine learning to understand trends in the transient-stability limit. IEEE Transactions on Power Systems, Advance online publication. https://doi.org/10.1109/TPWRS.2023.3248941

Hancock, J. T., & Khoshgoftaar, T. M. (2020). Survey on categorical data for neural networks. Journal of Big Data, 7(1). https://doi.org/10.1186/s40537-020-00305-w

Huang, S., Aral, S., Hu, Y. J., & Brynjolfsson, E. (2020). Social advertising effectiveness across products: A large-scale field experiment. Marketing Science, 39(6), 1142–1165. https://doi.org/10.1287/mksc.2020.1240

Ismail, N., & Yusof, U. K.. (2022). Recent trends of machine learning predictions using open data: A systematic review. Journal of Information and Communication Technology, 21(3), 337–381. https://doi.org/10.32890/jict2022.21.3.3

Jha, A., Jain, H., Sharma, P., Sharma, Y., & Tiwari, K. (2024). Optimising real-time bidding strategies: An experimental analysis of reinforcement-learning and machine-learning techniques. Procedia Computer Science, 235, 2017–2026. https://doi.org/10.1016/j.procs.2024.04.191

Ji, H., Xu, X., Su, G., Qang, J., & Wang, Y. (2024). Utilising machine learning for precise audience targeting in data science and targeted advertising. Academic Journal of Science and Technology, 9(2), 215–220. https://doi.org/10.54097/r7gek671

Johnson, J. M., & Khoshgoftaar, T. M. (2019). Survey on deep learning with class imbalance. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0192-5

Kedar, M. M. (2024). Exploring the effectiveness of SHAP over other explainable-AI methods. International Journal of Scientific Research in Engineering and Management, 8(6), 1–5. https://doi.org/10.55041/ijsrem35556

Kedi, W. E., Ejimuda, C., Idemudia, C., & Ijomah, T. I. (2024). Machine-learning software for optimising SME social-media marketing campaigns. Computer Science & IT Research Journal, 5(7), 1634–1647. https://doi.org/10.51594/csitrj.v5i7.1349

Kosaraju, N., Sankepally, S. R., & Rao, K. M. (2023). Categorical data: Need, encoding, selection of encoding method and its emergence in machine-learning models—A practical review study on heart-disease prediction dataset using Pearson correlation. In Lecture Notes in Networks and Systems, 551 (pp. 369–382). Springer. https://doi.org/10.1007/978-981-19-6631-6_26

Kovalerchuk, B., Dunn, A., Worland, A., & Wagle, S. (2024). Interactive decision tree creation and enhancement with complete visualisation for explainable modeling. In Artificial Intelligence and Visualization: Advancing Visual Knowledge Discovery (Studies in Computational Intelligence, Vol. 1126, pp. 3–40). Springer. https://doi.org/10.1007/978-3-031-46549-9_1

Lakshmanarao, A., Ushanag, S., & Leela, B. S. (2021). Ad prediction using click-through rate and machine learning with reinforcement learning. 2021 Fourth International Conference on Electrical, Computer and Communication Technologies (ICECCT). https://ieeexplore.ieee.org/ document/9616653

Liu, Z., Yao, Y., Liu, C., & Sun, Z. (2021). Handling inter-class and intra-class imbalance in classimbalanced learning. arXiv. https://doi.org/10.48550/arxiv.2111.12791

Lundberg, S., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems (Vol. 30, pp. 4765–4774). Curran Associates. https://proceedings.neurips.cc/paper_files/paper/2017/file/8a20a8621978632d76c43dfd28b67 767-Paper.pdf

Lusito, S., Pugnana, A., & Guidotti, R. (2023). Solving imbalanced learning with outlier detection and features reduction. Machine Learning, Advance online publication. https://doi.org/10.1007/ s10994-023-06448-0

Mihajlović, A., Gajić, J., & Papić, T. (2024). Analysis of the cost-effectiveness of the university Instagram marketing campaign using A/B testing. Sinteza 2024 Conference Proceedings (pp. 266–273). https://doi.org/10.15308/sinteza-2024-266-273 mljourney. (2024, June 15). Importance of feature scaling in machine learning. ML Journey. https://mljourney.com/importance-of-feature-scaling-in-machine-learning/

Northcott, C., Herbert, J., Barrea, A., & Maher, C. (2021). Should Facebook advertisements promoting a physical-activity smartphone app be image- or video-based, and should they promote benefits of being active or the app attributes? Translational Behavioural Medicine, 11(12), 2136–2143. https://doi.org/10.1093/tbm/ibab

Nwaimo, C. S., Adegbola, A. E., & Adegbola, M. D. (2024). Data-driven strategies for enhancing user engagement in digital platforms. International Journal of Management & Entrepreneurship Research, 6(6), 1854–1868. https://doi.org/10.51594/ijmer.v6i6.1170

Osakwe, J., Shilongo, A., & Ziezo, M. M. (2023). Optimising customer segmentation in digital marketing using predictive analytics: A review of literature. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4662191

Patil, S., Raut, K., Palsodkar, P., Singh, T., Dubey, Y., & Umate, R. (2022). Click-prediction learning for effective advertising. 2022 International Conference on Emerging Trends in Engineering and Management Science (ICETEMS 2022). IEEE. https://doi.org/10.1109/ICETEMS56252. 2022.10093291

Prihatiningsih, T., Panudju, R., & Prasetyo, I. J. (2024). Digital advertising trends and effectiveness in the modern era: A systematic literature review. Golden Ratio of Marketing and Applied Psychology of Business, 4(2). https://doi.org/10.52970/grmapb.v4i2.505

Prosekov, A. Y. (2020). Characteristics and key limitations of traditional methods for accounting hunting animals and digital technologies for solving the existing problems (Review). Agricultural Science Euro-North-East, 21(4), 341–354. https://doi.org/10.30766/2072-9081. 2020.21.4.341-354

Raufi, B., Finnegan, C., & Longo, L. (2024). A comparative analysis of SHAP, LIME, ANCHORS, and DICE for interpreting a dense neural network in credit-card fraud detection. In Communications in Computer and Information Science (pp. 365–383). Springer. https://doi.org/10.1007/978-3-031-63803-9_20

Rodríguez-Pérez, R., & Bajorath, J. (2020). Interpretation of machine-learning models using Shapley values: Application to compound potency and multi-target activity predictions. Journal of Computer-Aided Molecular Design, 34(10), 1013–1026. https://doi.org/10.1007/s10822-02000314-0

Santos, C. (2023). Towards an interpretable advertisement click prediction [Unpublished master’s thesis]. Universidade Católica Portuguesa. scikit-learn. (2019). sklearn. pre-processing.MinMaxScaler—scikit-learn 0.22.1 documentation. Scikitlearn. https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMax Scaler.html

Stewart, K., Dalakas, V., & Eells, D. (2023). If a picture is worth 1000 words, how much is a video worth? The influence of ad format on ticket purchasers’ ad clicks. Journal of Promotion

Management, Advance online publication. https://doi.org/10.1080/10496491.2023.2279767

Tavares, D., Remondes, J., & Lima, A. M. (2023). Facebook ads interaction campaigns. In Advances in Marketing, Customer Relationship Management, and E-Services (pp. 23–51). IGI Global. https://doi.org/10.4018/978-1-6684-9146-1.ch002

Xie, W., Dotsch, R., Bos, M., Bart, Y., Han, Z., & Liu, Y. (2024). Congruence affects story-ad engagement on social media. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4790627

Yang et al. (2019). AiAds: Automated and intelligent advertising system for sponsored search. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’19, pp. 1881–1890). ACM. https://doi.org/10.1145/3292500.3330782

Yang, Y., & Zhai, P. (2022). Click-through rate prediction in online advertising: A literature review. Information Processing & Management, 59(2), Article 102853. https://doi.org/10.1016/j.ipm. 2021.102853

Ye, H.-J., Chen, H.-Y., Zhan, D.-C., & Chao, W.-L. (2020). Identifying and compensating for feature deviation in imbalanced deep learning. arXiv. https://doi.org/10.48550/arxiv.2001.01385

Yoon, S. (2022). Introduction to the special issue on the future of advertising. International Journal of Advertising, 41(5), 823–826. https://doi.org/10.1080/02650487.2022.2074557

Zhao, Z., & Harinen, T. (2019). Uplift modeling for multiple treatments with cost optimisation. 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA) (pp. 422431). IEEE. https://doi.org/10.1109/DSAA.2019.00057

Downloads

Published

31-01-2026

Issue

Vol. 25 No. 1 (2026): Journal of Information and Communication Technology (JICT) Vol.25, No.1, January 2026

Section

Articles

License

Copyright (c) 2026 Journal of Information and Communication Technology

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Zahirah, K. F., Poernomo, W., Yusuf, E., & Bernardus, D. (2026). Evaluating the Effectiveness of Digital Product Advertisement Type using Machine Learning and Shapley Additive Explanations Analysis. Journal of Information and Communication Technology, 25(1), 79-106. https://doi.org/10.32890/jict2026.25.1.5

Research impact

Harvested 2026-09-06
1 citations, from Scopus — the highest of the sources checked

Counts differ between services because each indexes a different body of literature. None of them is the whole picture.

Identifiers DOI 10.32890/jict2026.25.1.5 OpenAlex W7128537702 Scopus 105031424728