Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

Harpreet Singh; Na Helian; Roderick Adams; Yi Sun

doi:10.1007/978-3-031-84353-2_20

Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

Harpreet Singh
, Na Helian
, Roderick Adams
, Yi Sun

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

Sentiment Analysis could be performed on textual data and indicates the general ‘tone’ or emotional state of the writing. It is important in business, for instance in marketing, to determine customer opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance has been obtained by first converting the text to a grayscale image and then using a BLSTM and deep CNN, specifically ResNet, to classify the data. This paper investigates the addition of more context to the original text using a pre-trained BERT model to produce contextualised textual images. This produces a marked improvement over the previous results. The proposed BERT-BLSTM-ResNet model outperforms the BERT model on smaller datasets and above a threshold data size, the BERT performance is comparable.

Original language	English
Title of host publication	Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings
Editors	Leszek Rutkowski, Marcin Korytkowski, Rafal Scherer, Ryszard Tadeusiewicz, Witold Pedrycz, Jacek M. Zurada
Publisher	Springer Nature
Pages	234-243
Number of pages	10
ISBN (Print)	9783031843525
DOIs	https://doi.org/10.1007/978-3-031-84353-2_20
Publication status	Published - 17 Feb 2025
Event	The 23rd International Conference on Artificial Intelligence and Soft Computing 2024 - Zakopane, Poland Duration: 16 Jun 2024 → 20 Jun 2024 https://icaisc.eu/

Publication series

Name	Lecture Notes in Computer Science
Volume	15164 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	The 23rd International Conference on Artificial Intelligence and Soft Computing 2024
Abbreviated title	ICAISC 2024
Country/Territory	Poland
City	Zakopane
Period	16/06/24 → 20/06/24
Internet address	https://icaisc.eu/

Keywords

BERT in NLP
Contextualised Textual Images
Deep 2D CNN on Text
sentiment Analysis

Access to Document

10.1007/978-3-031-84353-2_20Licence: Unspecified

Accepted_Manuscript
© 2025 Springer Nature. This is the accepted manuscript version of a conference paper/proceeding which has been published in final form at https://doi.org/10.1007/978-3-031-84353-2_20
Accepted author manuscript, 628 KBLicence: Unspecified

Cite this

Singh, H., Helian, N., Adams, R., & Sun, Y. (2025). Using BERT to Generate Contextualised Textual Images for Sentiment Analysis. In L. Rutkowski, M. Korytkowski, R. Scherer, R. Tadeusiewicz, W. Pedrycz, & J. M. Zurada (Eds.), Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings (pp. 234-243). (Lecture Notes in Computer Science; Vol. 15164 LNAI). Springer Nature . https://doi.org/10.1007/978-3-031-84353-2_20

Singh, Harpreet ; Helian, Na ; Adams, Roderick et al. / Using BERT to Generate Contextualised Textual Images for Sentiment Analysis. Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings. editor / Leszek Rutkowski ; Marcin Korytkowski ; Rafal Scherer ; Ryszard Tadeusiewicz ; Witold Pedrycz ; Jacek M. Zurada. Springer Nature , 2025. pp. 234-243 (Lecture Notes in Computer Science).

@inproceedings{21e3f944b9d84307851a4d0df815eb43,

title = "Using BERT to Generate Contextualised Textual Images for Sentiment Analysis",

abstract = "Sentiment Analysis could be performed on textual data and indicates the general {\textquoteleft}tone{\textquoteright} or emotional state of the writing. It is important in business, for instance in marketing, to determine customer opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance has been obtained by first converting the text to a grayscale image and then using a BLSTM and deep CNN, specifically ResNet, to classify the data. This paper investigates the addition of more context to the original text using a pre-trained BERT model to produce contextualised textual images. This produces a marked improvement over the previous results. The proposed BERT-BLSTM-ResNet model outperforms the BERT model on smaller datasets and above a threshold data size, the BERT performance is comparable.",

keywords = "BERT in NLP, Contextualised Textual Images, Deep 2D CNN on Text, sentiment Analysis",

author = "Harpreet Singh and Na Helian and Roderick Adams and Yi Sun",

note = "{\textcopyright} 2025 Springer Nature. This is the accepted manuscript version of a conference paper/proceeding which has been published in final form at https://doi.org/10.1007/978-3-031-84353-2\_20; The 23rd International Conference on Artificial Intelligence and Soft Computing 2024, ICAISC 2024 ; Conference date: 16-06-2024 Through 20-06-2024",

year = "2025",

month = feb,

day = "17",

doi = "10.1007/978-3-031-84353-2\_20",

language = "English",

isbn = "9783031843525",

series = "Lecture Notes in Computer Science",

publisher = "Springer Nature ",

pages = "234--243",

editor = "Leszek Rutkowski and Marcin Korytkowski and Rafal Scherer and Ryszard Tadeusiewicz and Witold Pedrycz and Zurada, \{Jacek M.\}",

booktitle = "Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings",

address = "Netherlands",

url = "https://icaisc.eu/",

}

Singh, H, Helian, N , Adams, R & Sun, Y 2025, Using BERT to Generate Contextualised Textual Images for Sentiment Analysis. in L Rutkowski, M Korytkowski, R Scherer, R Tadeusiewicz, W Pedrycz & JM Zurada (eds), Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings. Lecture Notes in Computer Science, vol. 15164 LNAI, Springer Nature , pp. 234-243, The 23rd International Conference on Artificial Intelligence and Soft Computing 2024, Zakopane, Poland, 16/06/24. https://doi.org/10.1007/978-3-031-84353-2_20

Using BERT to Generate Contextualised Textual Images for Sentiment Analysis. / Singh, Harpreet; Helian, Na ; Adams, Roderick et al.
Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings. ed. / Leszek Rutkowski; Marcin Korytkowski; Rafal Scherer; Ryszard Tadeusiewicz; Witold Pedrycz; Jacek M. Zurada. Springer Nature , 2025. p. 234-243 (Lecture Notes in Computer Science; Vol. 15164 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

AU - Singh, Harpreet

AU - Helian, Na

AU - Adams, Roderick

AU - Sun, Yi

PY - 2025/2/17

Y1 - 2025/2/17

N2 - Sentiment Analysis could be performed on textual data and indicates the general ‘tone’ or emotional state of the writing. It is important in business, for instance in marketing, to determine customer opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance has been obtained by first converting the text to a grayscale image and then using a BLSTM and deep CNN, specifically ResNet, to classify the data. This paper investigates the addition of more context to the original text using a pre-trained BERT model to produce contextualised textual images. This produces a marked improvement over the previous results. The proposed BERT-BLSTM-ResNet model outperforms the BERT model on smaller datasets and above a threshold data size, the BERT performance is comparable.

AB - Sentiment Analysis could be performed on textual data and indicates the general ‘tone’ or emotional state of the writing. It is important in business, for instance in marketing, to determine customer opinions and trends, and in analysing social media to help weed out inappropriate or discriminatory language. Recently improved performance has been obtained by first converting the text to a grayscale image and then using a BLSTM and deep CNN, specifically ResNet, to classify the data. This paper investigates the addition of more context to the original text using a pre-trained BERT model to produce contextualised textual images. This produces a marked improvement over the previous results. The proposed BERT-BLSTM-ResNet model outperforms the BERT model on smaller datasets and above a threshold data size, the BERT performance is comparable.

KW - BERT in NLP

KW - Contextualised Textual Images

KW - Deep 2D CNN on Text

KW - sentiment Analysis

UR - https://www.scopus.com/pages/publications/105010226008

U2 - 10.1007/978-3-031-84353-2_20

DO - 10.1007/978-3-031-84353-2_20

M3 - Conference contribution

SN - 9783031843525

T3 - Lecture Notes in Computer Science

SP - 234

EP - 243

BT - Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings

A2 - Rutkowski, Leszek

A2 - Korytkowski, Marcin

A2 - Scherer, Rafal

A2 - Tadeusiewicz, Ryszard

A2 - Pedrycz, Witold

A2 - Zurada, Jacek M.

PB - Springer Nature

T2 - The 23rd International Conference on Artificial Intelligence and Soft Computing 2024

Y2 - 16 June 2024 through 20 June 2024

ER -

Singh H, Helian N , Adams R , Sun Y. Using BERT to Generate Contextualised Textual Images for Sentiment Analysis. In Rutkowski L, Korytkowski M, Scherer R, Tadeusiewicz R, Pedrycz W, Zurada JM, editors, Artificial Intelligence and Soft Computing - 23rd International Conference, ICAISC 2024, Proceedings. Springer Nature . 2025. p. 234-243. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-84353-2_20

Using BERT to Generate Contextualised Textual Images for Sentiment Analysis

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this