Published on August 11, 2007
Semiotic characteristics of speech-accompanying gestures in children and adults’ narrative: Semiotic characteristics of speech-accompanying gestures in children and adults’ narrative Sotaro Kita (Univ. of Birmingham) Catherine Wood (Univ. of Bristol) Speech accompanying gestures: Speech accompanying gestures Speech and gesture are an integral part of our communicative practice. Speech-accompanying gestures seem to be cultural universal. Even congenitally blind individuals sometimes spontaneously produce gesture, when talking to another blind person (Iverson andamp; Goldin-Meadow, 1997). Children at the one-word stage already systematically combine a word and a gesture (e.g., Goldin-Meadow andamp; Butcher, 2003). "Representational gestures": 'Representational gestures' Speech-accompanying gestures come in different varieties (e.g., McNeill, 1992): Emblems the form-meaning relation is determined by convention) Beats the form is more or less constant regardless of the context of use. Representational gestures (iconic and pointing gestures) the form of gesture flexibly changes according to what is being expressed by the gesture. Similarity (iconic gesture) and spatial contiguity (pointing gestures) shape the form of gestures. We focus on 'representational gestures' How to represent things in representational gestures: How to represent things in representational gestures The same idea can be expressed in different ways. E.g. how can one represent the object in an event in which a round object moves across. pointing, fist, C-hand Successive gestures are positioned so that relative location of events and objects are clear. Spatial relations in the 'gesture space' are analogous to the spatial relations among the referents. Goal of the study: Goal of the study This study investigates how the nature of gestural representation produced in narrative differs between children and adults. We focus on two aspects of gestural representation: How characters in a story is represented. How spatial relationships in a story is represented. What kind of differences do we expect? Cognitive development and gestural representation: Cognitive development and gestural representation Symbolic distancing (Werner andamp; Kaplan, 1963) Children start out with representations in which the 'signifier' (depicting element) is similar to the 'signified' (depicted content). The symbolic distance between the signifier and the signified decreases in development. Body movement imitating body movement =andgt; Body movement representing something else than body movement. The body as a representational medium becomes more flexible. Cognitive development and gestural representation: Cognitive development and gestural representation 'De-anchoring' action from the physical world (Piaget, 1951), which makes action to be a more autonomous symbol (Werner andamp; Kaplan, 1963). In children, 'gesture space is actual space' (McNeill, 1992, p.303). Children's gesture space does not merely represent the story space, but it also remains to be the physical space in which children perform action. In adult, gesture space is a representation of the story space. It is a virtual (imaginary) stage on which different entities are located and move around. It is a self-contained stage, independent from actual space surrounding the speaker. Cognitive development and gestural representation: Cognitive development and gestural representation Thus, based on the conventional theories of symbolic development, we expect increasing autonomy of gestural movement (the signifier) from The gestural content (the signified) The physical world Gestural movement becomes less and less like real action in the physical world and becomes representationally more flexible, in the course of development. Embodied cognition and gestural representation: Embodied cognition and gestural representation However, there is also a reason to believe that gestural movement might become more and more like real action in the physical world. Body movements play a role not only in physical manipulation of the environment and communication, but also in thinking (e.g., Alibali, et al. 2000; Kita, 2000; Schwartz andamp; Black, 1999). Embodied cognition and gestural representation: Embodied cognition and gestural representation Development of embodied cognition might lead to gestures that manipulate information flow in a story. Body as a medium of representation =andgt; Body as a manipulator of representation. Methods: Methods 20 English speakers each in the following age groups: 3 year olds (M = 3;8, Range 3;3-4;3) 9 year olds (M = 9;4, Range 9;0-10;0) Adults (18+) They narrated five animated cartoons, which depict movements of two protagonists (Özyurek, Kita, andamp; Allen, 2000). Their spontaneous speech-accompanying gestures were analyzed. Example of the stimuli: Example of the stimuli roll up Gestural representation of the protagonist: Gestural representation of the protagonist How are the protagonists represented? when their motion is depicted when their location is indicated Different possibilities Protagonist+Plane Manipulated Protagonist Hand as Protagonist Body as Protagonist (moving protagonist, only) (Hand) Point at Protagonist Head pointing at Protagonist Locating protagonists: Locating protagonists * * * = Age difference, p andlt; .05 * Protagonists in motion: Protagonists in motion * * * = Age difference, p andlt; .05 * * Representation of protagonists: Representation of protagonists In both types of representation of protagonists, Manipulated Protagonist Gs increased with age Head pointing Gs increased with age For gestures locating protagonists, Pointing at Protagonist decreased with age. For gesture for moving protagonists, Hand as Protagonist Gs decreased with age. Body as Protagonist Gs decreased with age. Manipulated Protagonist Gs increased with age: Manipulated Protagonist Gs increased with age This increase is consistent with the idea that older participants 'construct' narrative in the sense that they manipulate/control the flow of in formation in discourse. Gestural representation becomes increasingly similar to (manipulative) action in the real world. Manipulation of information (rather than physical entity). Development of embodied cognition Head pointing at Protagonist increased with age : Head pointing at Protagonist increased with age This increase is consistent with the idea that the signifier becomes increasingly autonomous with age (Werner andamp; Kaplan, 1963). Different signifiers (hand or head) can serve the same function (indicating a vector). Body as a representational medium becomes increasing flexible. Protagonist as Body Gs decreased with age: Protagonist as Body Gs decreased with age This decrease is consistent with the idea that symbolic distance increases with age. Gestural representation becomes increasingly different from signified action in the real world. Features of gesture space: Features of gesture space Gesture space Size Dependence on the physical environment (Use of physical 'props' in gesture space.) Gesture space size: Representing the top and bottom: Gesture space size: Representing the top and bottom We test whether children construct a gestural 'stage' that is independent from the physical environment. OR they regard the physical environment to be the story world, and immerse themselves into it. Children have 'over-sized' gestures as if they are in the story world (McNeill, 1992, no quantitative data). The 'top of the hill' would be much higher than children’s head, and the 'bottom of the hill' is close to the floor. Gesture height coding: Gesture height coding Where in the gesture space was the top and the bottom of the hill in the stimuli. We coded the height of 'top' and 'bottom' for gestures that depicted the protagonists' motion or indicated the protagonists' location. Representation of "Top": Representation of 'Top' * * * = Age difference, p andlt; .05 * Representation of "Bottom": Representation of 'Bottom' * * = Age difference, p andlt; .05 * Representation of the ground: Representation of the ground We test whether children construct a gestural 'stage' that is independent from the physical environment. How do they represent the ground on which the protagonists moved along? the ground is imaginary OR, a physical prop stands for the ground The prop could be the surface of a table or the body part such as the arm or the palm. Motion gestures in contact with an object or another body part: Motion gestures in contact with an object or another body part = Age difference, p andlt; .05 * * Children's gesture space: Children's gesture space Gesture size Children construct gesture space as if their body is the protagonists' body in the story world. Contact with a prop Children use solid surface in the physical environment to represent the solid surface in the story world. Children's gesture space: Children's gesture space Namely, Children regard the physical environment surrounding them to be the story world, and immerse themselves into it. In contrast, adults construct gesture 'stage' independent from the physical environment, and delimited in size. Summary: In the course of development...: Summary: In the course of development... Gestural movement becomes less and less like real action in the physical world, and becomes representationally more flexible. Hand movement can represent something else than hand movement. The same semiotic function (pointing) can be realized by different body parts (hand vs. head). Summary: In the course of development...: Summary: In the course of development... Gestural movement becomes more and more like real action in the physical world, and manipulate/control what happens in the gestured world. Gestures represent protagonists as if to hold and manipulate them. Summary: In the course of development...: Summary: In the course of development... Gesture space becomes less and less immersive 'virtual reality', and becomes smaller and more clearly differentiated from the actual space that surrounds the speaker. Narrative production becomes less of 're-living' of events the speaker witnessed, but becomes more of controlled construction of a narrated world. Acknowledgment: Acknowledgment This study was supported by National Science Foundation (USA). The data was collected in collaboration with Shanley Allen and Asli Özyürek. End: End Slide34: Piaget, J. (1951). Play, dreams, and imitation in childhood. New York: Norton. Werner, H., andamp; Kaplan, B. (1963). Symbolic formation. An organismic-developmental approach to language and the expression of thought. New York: John Wiley andamp; Sons, Inc.