Classifying encyclopedia articles : Comparing machine and deep learning methods and exploring their predictions

Brenon, Alice and Moncla, Ludovic and Mcdonough, Katherine (2022) Classifying encyclopedia articles : Comparing machine and deep learning methods and exploring their predictions. Data and Knowledge Engineering, 142: 102098. ISSN 0169-023X

Abstract

This article presents a comparative study of supervised classification approaches applied to the automatic classification of encyclopedia articles written in French. Our dataset is composed of 17 volumes of text from the Encyclopédie by Diderot and d'Alembert (1751-72) including about 70,000 articles. We combine text vectorization (bag-of-words and word embeddings) with machine learning methods, deep learning, and transformer architectures. In addition evaluating these approaches, we review the classification predictions using a variety of quantitative and qualitative methods. The best model obtains 86% as an average f-score for 38 classes. Using network analysis we highlight the difficulty of classifying semantically close classes. We also introduce examples of opportunities for qualitative evaluation of "misclassifications" in order to understand the relationship between content and different ways of ordering knowledge. We openly release all code and results obtained during this research.