1.1 A linguistic gesture representation strategy

The CoGesT 1.0 transcription system has been developed as a systematic, linguistically based and heuristically oriented formalism for transcribing and annotating gestures in video corpora. The context of this development is the goal of providing a theoretical foundation for the design of corpus-based multilingual multimodal lexica.

The development strategy is three-pronged:

1. a multimodal digital video corpus is designed, recorded and processed;
2. a full specification of conditions on lexicon structure is developed;
3. corpus information is mapped from the corpus representation into a specific multilingual, multimodal lexicon by a two-step process of substring extraction and class hierarchy induction.

The first part of the corpus processing phase was the development of a gesture transcription and annotation system, CoGesT 1.0, through several revision cycles, and the annotation of the corpus with this system (on audio speech corpus annotation see Gibbon et al. (2000,1997)). The present report is concerned with the second part of the corpus processing phase and describes the initial stages of a formal reconstruction of the syntax of the CoGesT 1.0 transcription and annotation conventions, with proposals for simplifying the conventions. The result will be known as CoGesT 1.1.