Cost-Sensitive Structured Perceptron Incorporating Category Hierarchy for Named Entity Recognition

Authors

  • Shohei Higashiyama Graduate School of System Informatics Kobe University, Japan
  • Blondel Mathieu NTT Communication Science Laboratories, Kobe University, Japan
  • Kazuhiro Seki Faculty of Intelligence and Informatics, Konan University, Japan
  • Kuniaki Uehara Faculty of Intelligence and Informatics, Konan University, Japan

DOI:

https://doi.org/10.32890/jict2015.14.1

Keywords:

Named entity recognition, category hierarchy, cost-sensitive learning, biomedical text mining

Abstract

Named Entity Recognition (NER) is a fundamental natural language processing task for the identifi cation and classifi cation of expressions into predefi ned categories, such as person and organization. Existing NER systems usually target about 10 categories and do not incorporate analysis of category relations. However, categories often belong naturally to some predefi ned hierarchy. In such cases, the distance between categories in the hierarchy becomes a rich source of information that can be exploited. This is intuitively useful particularly when the categories are numerous. On that account, this paper proposes an NER approach that can leverage category hierarchy information by introducing, in the structured perceptron framework, a cost function more strongly penalizing category predictions that are more distant from the correct category in the hierarchy. Experimental results on the GENIA biomedical text corpus indicate the effectiveness of the proposed approach as compared with the case where no cost function is utilized. In addition, the proposed approach demonstrates the superior performance over a representative work using multi-class support vector machines on the same corpus. A possible direction to further improve the proposed approach is to investigate more elaborate cost functions than a simple additive cost adopted in this work.

 

References

Chieu, H., & Ng, H. (2003). Named entity recognition with a maximum entropy approach. In Proceedings of the 7th Conference on Natural Language Learning (CoNLL-2003), pages 160–163. Journal of ICT, 14, 2015, pp: 1–

Chieu, H. L., & Ng, H. T. (2002). Named entity recognition: A maximum entropy approach using global information. In Proceedings of the 19th International Conference on Computational Linguistics, pages 1–7.

Collins, M. (2002). Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1-8.

Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., & Singer, Y. (2006). Online passive-aggressive algorithms. The Journal of Machine Learning Research (JMLR), 7, 551-585.

Dekel, O., Keshet, J., & Singer, Y. (2004). Large margin hierarchical classification. In Proceedings of the 21st International Conference on Machine Learning (ICML), 27-42.

Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., & Weischedel, R. (2004). The automatic content extraction (ACE) program–tasks, data, and evaluation. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), 837-840.

Finkel, J., Grenager, T., & Manning, C. (2005). Incorporating non-local information into information extraction systems by gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL), 363–370.

Fleischman, M. (2001). Automated subcategorization of named entities. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (ACL), 25-30.

Fleischman, M., & Hovy, E. (2002). Fine grained classification of named entities. In Proceedings of the 19th International Conference on Computational Linguistics (COLING), 1-7.

Florian, R., Ittycheriah, A., Jing, H., & Zhang, T. (2003). Named entity recognition through classifier combination. In Proceedings of the 7th Conference On Natural Language Learning (CoNLL-2003), 168-171.

Freund, Y., & Schapire, R. (1999). Large margin classification using the perceptron algorithm. Machine Learning, 37(3),277–296.

Grishman, R., & Sundheim, B. (1996). Message understanding conference-6: A brief history. In Proceedings of the 16th International Conference on Computational Linguistics (COLING), 466-471.

Isozaki, H., & Kazawa, H. (2002). Efficient support vector classifiers for named entity recognition. In Proceedings of the 19th International Conference on Computational Linguistics, 1-7.

Johansson, R., & Nugues, P. (2008). Dependency-based semantic role labeling of propbank. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP), 69–78. Journal of ICT, 14, 2015, pp: 1–

Kim, J., Ohta, T., Tateisi, Y., & Tsujii, J. (2003). Genia corpus–a semantically annotated corpus for bio-textmining. Bioinformatics, 19 (Suppl.1),180-182.

Kim, J., Ohta, T., Tsuruoka, Y., Tateisi, Y., & Collier, N. (2004). Introduction to the bio-entity recognition task at JNLPBA. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (JNLPBA), pages 70-75.

Lafferty, J., McCallum, A., and Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (ICML), 282-289.

Lee, K., Hwang, Y., Kim, S., & Rim, H. (2004). Biomedical named entity recognition using two-phase model based on SVMs. Journal of Biomedical Informatics, 37(6),436-447.

McCallum, A. and Li, W. (2003). Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, 188-191.

Ohta, T., Tateisi, Y., & Kim, J. (2002). The genia corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the 2nd International Conference on Human Language Technology Research (HLT 2002), pages 82-86.

Rahman, S. A., & Omar, N. (2013). Transforming noun phrase structure form into rules to detect compound nouns in Malay sentences. Journal of Information and Communication Technology, 12,161-173.

Ritter, A., Clark, Mausam, S., & O, Etzioni. (2011). Named entity recognition in tweets: An experimental study. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 1524-1534.

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6),386–408.

Sekine, S., & Isahara, H. (2000). IREX: IR and IE evaluation project in Japanese. In Proceedings of the 2nd of the Language Resources and Evaluation Conference (LREC), 1475–1480.

Sekine, S. and Nobata, C. (2004). Definition, dictionaries and tagger for extended named entity hierarchy. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), 1977–1980.

Sekine, S., Sudo, K., & Nobata, C. (2002). Extended named entity hierarchy. In Proceedings of the 3rd International Conference on Language

Resources and Evaluation (LREC 2002), 1818–1824. Journal of ICT, 14, 2015, pp: 1–

Shalev-Shwartz, S., Singer, Y., & Srebro, N. (2007). Pegasos: Primal estimated sub-gradient solver for SVM. In Proceedings of the 24th International Conference on Machine Learning, 807–814.

Song, H., Son, J., Noh, T., Park, S., and Lee, S. (2012). A cost sensitive partof-speech tagging: differentiating serious errors from minor errors. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL), 1025–1034.

Tjong Kim Sang, E. F., & De Meulder, F. (2003). Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the 7th Conference on Natural Language Learning (CoNLL-2003), 142–147.

Tsochantaridis, I., Hofmann, T., Joachims, T., & Altun, Y. (2004). Support vector machine learning for interdependent and structured output spaces. In Proceedings of the 21st International Conference on Machine Learning (ICML).

Zamin, N., Oxley, A., & Bakar, Z. A. (2013). Projecting named entity tags from a resource rich language to a resource poor language. Journal of Information and Communication Technology, 12:, 121–146.

Zhou, G., & Su, J. (2002). Named entity recognition using an HMM-based chunk tagger. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, 473–480.

Zhou, G., & Su, J. (2004). Exploring deep knowledge resources in biomedical name recognition. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (JNLPBA), 96–99.

Downloads

Published

28-04-2015

Issue

Vol. 14 (2015): Journal of Information and Communication Technology (JICT) Vol. 14, 2015

Section

Articles

How to Cite

Higashiyama, S., Mathieu, B., Seki, K., & Uehara, K. (2015). Cost-Sensitive Structured Perceptron Incorporating Category Hierarchy for Named Entity Recognition. Journal of Information and Communication Technology, 14, 1-20. https://doi.org/10.32890/jict2015.14.1

Research impact

Harvested 2026-09-06
0 citations recorded so far

Counts differ between services because each indexes a different body of literature. None of them is the whole picture.

Identifiers DOI 10.32890/jict2015.14.1 OpenAlex W4241295301 Scopus 85182138748