Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

Sentiment Analysis could be performed on textual data and
indicates the general ‘tone’ or emotional state of the writing. It is important
in business, for instance in marketing, to determine customer
opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance
has been obtained by first converting the text to a grayscale image and
then using a BLSTM and deep CNN, specifically ResNet, to classify
the data. This paper investigates the addition of more context to the
original text using a pre-trained BERT model to produce contextualised
textual images. This produces a marked improvement over the previous
results. The proposed BERT-BLSTM-ResNet model outperforms the
BERT model on smaller datasets and above a threshold data size, the
BERT performance is comparable.