TAC 2011 MultiLing pilot overview

Giannakopoulos, G. and El-Haj, M. and Favre, B. and Litvak, M. and Steinberger, J. and Varma, V. (2011) TAC 2011 MultiLing pilot overview. In: Text Analysis Conference (TAC) 2011, MultiLing Summarisation Pilot :. TAC, Maryland, USA.

[thumbnail of Summarization2011_MultiLing_overview.proceedings]
Preview
PDF (Summarization2011_MultiLing_overview.proceedings)
Summarization2011_MultiLing_overview.proceedings.pdf - Published Version

Download (359kB)

Abstract

The Text Analysis Conference MultiLing Pilot of 2011 posed a multi-lingual summarization task to the summarization community, aiming to quantify and measure the performance of multi-lingual, multi-document summarization systems. The task was to create a 240–250 word summary from 10 news texts, describing a given topic. The texts of each topic were provided in seven languages (Arabic, Czech, English, French, Greek, Hebrew, Hindi) and each participant generated summaries for at least 2 languages. The evaluation of the summaries was performed using automatic (AutoSummENG, Rouge) and manual processes (Overall Responsiveness score). The participating systems were 8, some of which providing summaries across all languages. This paper provides a brief description for the collection of the data, the evaluation methodology, the problems and challenges faced, and an overview of participation and corresponding results.

Item Type:
Contribution in Book/Report/Proceedings
ID Code:
71274
Deposited By:
Deposited On:
17 Oct 2014 11:25
Refereed?:
Yes
Published?:
Published
Last Modified:
29 Feb 2024 01:36