Pro­gram

Abstract: NEUROGES (Lausberg, 2013; Lausberg, 2018) is an objective and reliable system for the analysis of speech-accompanying hand movements and gestures. A review of 18 empirical studies using NEUROGES in combination with ELAN demonstrates a good reliability of the system (Lausberg and Sloetjes 2015).
Since neuropsychological research provides evidence that some gesture types are generated in the right hemisphere independent from left-hemispheric speech production (Feyereisen, 1987;Kita & Lausberg, 2008; Hogrefe et al., 2010), and the existence of gesture - speech mismatches has been demonstrated (McNeill, 1992; Goldin-Meadow et al., 1993), NEUROGES offers the methodological approach to analyze gestures as a means of expression per se. The ongoing stream of hand movement behavior is segmented and classified according to movement criteria into more and more fine-grained units, resulting in a distinct analysis of gestures based on their visual appearance. The analysis algorithm is shown on http://neuroges.neuroges-bast.info/neuroges-analysis-system and illustrated by video samples.

(max. 12 participants; open for any researcher and student involved in empirical gesture research; 
Participants should bring their own laptop and preferably have ELAN installed).

!! This tutorial session is full!

Abstract: This tutorial is based on a recent paper (Pouw, Trujillo, Dixon, under review) and will address objective and time-effective methods to quantify temporal properties of gesture kinematics and acoustic markers of speech, where we focus on common challenges and possible solutions that come with the complexities of studying multimodal language. In order to provide a comprehensive training experience, our interactive session will include collecting audio-visual recordings and applying machine learning motion tracking analyses, of multimodal (dyadic) interactions, processing and analyzing the data, and creating useful data visualizations. In each step, we will engage participants in discussions on best-practices and encourage active participation. Once collected, data will be made available via file-sharing so participants can actively follow along with all of the data-processing steps. The goal of the tutorial is therefore to provide a basic training in, and stimulate an active discussion about how we as multimodal language researchers can converge on best-practices. We hope to further promote the utilization of new quantitative methods for capturing speech and gesture dynamics.

(max. 15 participants; Participants should bring their own laptop. Software requirements will be send via email in advance.)
 
Reference:
Pouw**, W., Trujillo**, J.P., & Dixon, J.A. (in press). The Quantification of Gesture-speech Synchrony: A Tutorial and Validation of Multi-modal Data Acquisition Using Device-based and Video-based Motion Tracking. Behavior Research Methods. doi: 10.31234/osf.io/jm3hk4

Abstract: There is neurocognitive support for gestures being closely related to spoken language (Willems & Hagoort, 2007) and evidence that gestures support language learning, comprehension and memory (Macedonia & Klimesch, 2014) but how to best use them in diverse classrooms is up for debate. Teachers everywhere are challenged by the need to include children who have very different abilities. Codified classroom gestures to support foreign language word learning may be especially helpful here.
This practical tutorial explores using gestures on the level of morphology in language teaching. It begins by reporting on a theater-based experiment on spatial preposition learning with refugee and grade six pupils in Germany and Poland. Following this input we will use gestures as the primary teaching tool to investigate a traditional story adapted for beginning learners of English.

(max. 20 individuals; open for gesture researchers, linguists as well as teachers and students of education who are interested in multimodal foreign language learning and teaching.)

References:
Macedonia, M., & Klimesch, W. (2014). Long-Term Effects of Gestures on Memory for Foreign   Language Words Trained in the Classroom. Mind, Brain, and Education, 8, 74–88.
Willems, R. M., & Hagoort, P. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain and Language, 101, 278–289. doi.org/10.1016/j.bandl.2007.03.004