VARD2:a tool for dealing with spelling variation in historical corpora

Baron, Alistair and Rayson, Paul (2008) VARD2:a tool for dealing with spelling variation in historical corpora. In: Postgraduate Conference in Corpus Linguistics, 2008-05-22.

[img]
Preview
PDF (BaronRaysonAston2008.pdf)
BaronRaysonAston2008.pdf

Download (807kB)

Abstract

When applying corpus linguistic techniques to historical corpora, the corpus researcher should be cautious about the results obtained. Corpus annotation techniques such as part of speech tagging, trained for modern languages, are particularly vulnerable to inaccuracy due to vocabulary and grammatical shifts in language over time. Basic corpus retrieval techniques such as frequency profiling and concordancing will also be affected, in addition to the more sophisticated techniques such as keywords, n-grams, clusters and lexical bundles which rely on word frequencies for their calculations. In this paper, we highlight these problems with particular focus on Early Modern English corpora. We also present an overview of the VARD tool, our proposed solution to this problem, which facilitates pre-processing of historical corpus data by inserting modern equivalents alongside historical spelling variants. Recent improvements to the VARD tool include the incorporation of techniques used in modern spell checking software.

Item Type: Contribution to Conference (Paper)
Journal or Publication Title: Postgraduate Conference in Corpus Linguistics
Uncontrolled Keywords: /dk/atira/pure/researchoutput/libraryofcongress/qa75
Subjects:
Departments: Faculty of Science and Technology > School of Computing & Communications
ID Code: 41666
Deposited By: ep_importer_comp
Deposited On: 13 Aug 2008 08:43
Refereed?: No
Published?: Published
Last Modified: 20 Sep 2019 23:20
URI: https://eprints.lancs.ac.uk/id/eprint/41666

Actions (login required)

View Item View Item