Automating Video Annotation

Hartley, Ed (2004) Automating Video Annotation. PhD thesis, Lancaster University.

Text (2004HartleyPhD)
2004HartleyPhD.pdf - Published Version
Available under License Creative Commons Attribution-NonCommercial-NoDerivs.
Download (24MB)

Abstract

Video annotation enables novel applications in moving picture production, archiving and other domains. Annotation associates descriptive material with moving picture sequences. Here, a bi-directional approach is adopted that combines bottom-up image feature extraction with a top-down content representation model. This model contains a generic framework for time-based visual content representation, from the narrative and event levels, to visual objects and their attributes. The model is developed from a combination of: interval graphs to represent temporal relationships, the application of scripts and conceptual dependency to represent events, and conceptual graphs to represent object interrelationships. Worked examples show that the model can be refined for specific domain applications. Continuous domain feature visual-object identification and location techniques are described. A transform domain colour and spatial frequency feature extraction and edge detection method is developed. It is shown that modifying a video decoder produces parallel feature and image streams. An end-user tool architecture, its implementation status, a methodology to support annotation through incremental knowledge base instantiation and overt "binding" of features to perceived objects using search and inference is described. Extensions to the semiotic model and their application to the development of the top-down content model are described. The basis of these developments in an analysis of video content representation techniques and their relationship to artificial intelligence, narrative, film theory, semiotics, continuous domain content-based image retrieval and still and moving picture compression technologies is described. An overall qualitative evaluation is performed. An assessment of the method’s psychological plausibility is made. Outlines of quantitative evaluation and further research are given.