Visual and Text Sentiment Analysis through Hierarchical Deep Learning Networks

Book excerpt: This book presents the latest research on hierarchical deep learning for multi-modal sentiment analysis. Further, it analyses sentiments in Twitter blogs from both textual and visual content using hierarchical deep learning networks: hierarchical gated feedback recurrent neural networks (HGFRNNs). Several studies on deep learning have been conducted to date, but most of the current methods focus on either only textual content, or only visual content. In contrast, the proposed sentiment analysis model can be applied to any social blog dataset, making the book highly beneficial for postgraduate students and researchers in deep learning and sentiment analysis. The mathematical abstraction of the sentiment analysis model is presented in a very lucid manner. The complete sentiments are analysed by combining text and visual prediction results. The book’s novelty lies in its development of innovative hierarchical recurrent neural networks for analysing sentiments; stacking of multiple recurrent layers by controlling the signal flow from upper recurrent layers to lower layers through a global gating unit; evaluation of HGFRNNs with different types of recurrent units; and adaptive assignment of HGFRNN layers to different timescales. Considering the need to leverage large-scale social multimedia content for sentiment analysis, both state-of-the-art visual and textual sentiment analysis techniques are used for joint visual-textual sentiment analysis. The proposed method yields promising results from Twitter datasets that include both texts and images, which support the theoretical hypothesis.