Welcome to the InfoSci Platform
Reference Hub3
Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification

Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification

Weihao Huang, Jiaojiao Chen, Qianhua Cai, Xuejie Liu, Yudong Zhang, Xiaohui Hu
Copyright: © 2022 |Volume: 18 |Issue: 1 |Pages: 16
ISSN: 1548-3924|EISSN: 1548-3932|EISBN13: 9781799893684|DOI: 10.4018/IJDWM.303673
Cite Article Cite Article

MLA

Huang, Weihao, et al. "Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification." IJDWM vol.18, no.1 2022: pp.1-16. http://doi.org/10.4018/IJDWM.303673

APA

Huang, W., Chen, J., Cai, Q., Liu, X., Zhang, Y., & Hu, X. (2022). Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification. International Journal of Data Warehousing and Mining (IJDWM), 18(1), 1-16. http://doi.org/10.4018/IJDWM.303673

Chicago

Huang, Weihao, et al. "Hierarchical Hybrid Neural Networks With Multi-Head Attention for Document Classification," International Journal of Data Warehousing and Mining (IJDWM) 18, no.1: 1-16. http://doi.org/10.4018/IJDWM.303673

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Document classification is a research topic aiming to predict the overall text sentiment polarity with the advent of deep neural networks. Various deep learning algorithms have been employed in the current studies to improve classification performance. To this end, this paper proposes a hierarchical hybrid neural network with multi-head attention (HHNN-MHA) model on the task of document classification. The proposed model contains two layers to deal with the word-sentence level and sentence-document level classification respectively. In the first layer, CNN is integrated into Bi-GRU and a multi-head attention mechanism is employed, in order to exploit local and global features. Then, both Bi-GRU and attention mechanism are applied to document processing and classification in the second layer. Experiments on four datasets demonstrate the effectiveness of the proposed method. Compared to the state-of-art methods, our model achieves competitive results in document classification in terms of experimental performance.