Comparing BERT Against Traditional Machine Learning Models in Text Classification

Eduardo C. Garrido-Merchan; Roberto Gozalo-Brizuela; Santiago  Gonzalez-Carvajal

doi:10.47852/bonviewJCCE3202838

Authors

Eduardo C. Garrido-Merchan Quantitative Methods Department, Comillas Pontifical University, Spain
Roberto Gozalo-Brizuela Quantitative Methods Department, Comillas Pontifical University, Spain
Santiago Gonzalez-Carvajal Artificial Intelligence Department, Polytechnic University of Madrid, Spain

DOI:

https://doi.org/10.47852/bonviewJCCE3202838

Keywords:

BERT, natural language processing, machine learning, comparison

Abstract

The BERT model has arisen as a popular state-of-the-art model in recent years. It is able to cope with NLP tasks such as supervised text classification without human supervision. Its flexibility to cope with any corpus delivering great results has make this approach very popular in academia and industry. Although, other approaches have been used before successfully. We first present BERT and a review on classical NLP approaches. Then, we empirically test with a suite of different scenarios the behaviour of BERT against traditional TF-IDF vocabulary fed to machine learning models. The purpose of this work is adding empirical evidence to support the use of BERT as a default on NLP tasks. Experiments show the superiority of BERT and its independence of features of the NLP problem such as the language of the text adding empirical evidence to use BERT as a default technique in NLP problems.

Received: 10 March 2023 | Revised: 4 April 2023 | Accepted: 20 April 2023

Conflicts of Interest

The authors declare that they have no conflicts of interest to this work.