Gesture and speech form an integrated communicative system. This integration has been investigated for the purpose of communication as well as learning. Recent research attempting to explain and model this integration focusses on how gesture influences speech, and how gesture – as a special form of action – forces the speaker to incorporate sensorimotor information into the representation. In these cognitive approaches, the fact that gesture is employed purposefully in various communicative genres, and thus interacts with speech for different discursive purposes, is disregarded.
Yet, types of communicative genres have been shown to bear different cognitive and communicative requirements on the speaker and hearer: For example, a narration requires different structural knowledge and verbal skills from a speaker than an explanation. While in competent speakers, the appearance of gestures in different communicative genres might occur fluent, in children who still learn to cope with the variety of discursive contexts, the role of gesture for thinking and communicating might be better observable: One possibility is that for particular genres, gestures are required resources for depicting strategies that facilitate mutual understanding, and children discover them with increasing experience in interaction, and with growing cognitive and linguistic competencies. Another – not exclusive – possibility is that gesture plays an important role on the way to master increasingly changing communicative
requirements, which becomes visible when children learn to produce different communicative genres. Thus, in addition to sensorimotor information that is recruited for the act of speaking, gesture might be an integral resource of speaking in relation to different communicative purposes. This possibility raises new questions for models of gesture-speech integration: Is gesture use depending on the experience in communicative genre, and if so, how is the pragmatic aspect incorporated into the production of speech?
In our proposed project, we apply qualitative and quantitative analyses as well as computational evaluation methods in investigating longitudinally how the integration of gesture and speech develops in 4 and 5 year-old preschool children and how context (meaning the communicative genre and physical surroundings) relates to the different gestural techniques that children use in interaction. The results of our project will provide a crucial contribution towards a comprehensive developmental theory of gesture and speech integration involving cognitive and pragmatic aspects of speaking.
Funded by the DFG (German Research Foundation)
Prof. Dr. Katharina J. Rohlfing, Paderborn University
Prof. Dr. Friederike Kern, Bielefeld University
Dr.-Ing. Kirsten Bergmann, CITEC, Bielefeld University
Prof. Dr.-Ing. Stefan Kopp, CITEC, Bielefeld University