Ginevra Castellano

Information about Ginevra Castellano

Published on November 23, 2007

Author: aksu

Source: authorstream.com

Content

Slide1:  Expressive gesture in interaction: the role of movement and gesture in emotion Ginevra Castellano, Antonio Camurri, Gualtiero Volpe WP6 HUMAINE Workshop, Paris, March, 10-11, 2005 Infomus Lab http://infomus.dist.unige.it Department of computer science, systems and telematic, University of Genoa What is expressive gesture :  What is expressive gesture Gesture: support to verbal communication, movement of the body containing information Expressive gesture: high-level non-verbal expressive and emotional communication (Camurri et al., 2004) Artistic context: gesture as conveyor of information related to the emotional domain (dance or music performances) Expressive gesture in HCI :  Expressive gesture in HCI Aims:  to communicate emotions to users  to recognize users’ emotional engagement Expressive gesture: movement as  conveyor of emotional information  component of an emotional process Expressive gesture analysis: a layered approach (1) (Camurri et al., 2004, 2005):  Expressive gesture analysis: a layered approach (1) (Camurri et al., 2004, 2005) Modelling techniques: prediction of emotions (e.g., multiple regression, neural networks, decision trees) Techniques for gesture segmentation, representation of gestures as trajectories in virtual, expressive spaces Video and audio processing techniques (computer vision techniques on the incoming images, signal processing on audio signals) Physical signals, video and audio pre-processing techniques (e.g. motion detection, audio filtering, etc.) ↑ ↑ ↑ Expressive gesture analysis: a layered approach (2):  Expressive gesture analysis: a layered approach (2) Not only analysis but also synthesis of expressive gesture Experiments on expressive gesture are carried out with Eyesweb open software platform (Camurri et al., 2000) Perspective: mapping info about users’ behaviour onto real time generation of expressive behaviour of virtual agents such ECAs Our activity in HUMAINE:  Our activity in HUMAINE Expressive gesture as a component of an emotional process Component-process model of emotion provided by Klaus Scherer (GERG) has been investigated (Scherer, 1984, 2000; Scherer and Zentner, 2001) We used motor activation component to evaluate the emotional engagement of users exposed to emotional stimuli Music, emotion and movement:  Music, emotion and movement Research in collaboration with Professor Klaus Scherer’s group (GERG, Geneva Emotion Research Group) Aim: to investigate the relationship between emotions induced by musical stimuli and movement Pilot experiment: are there correlations between the emotional characterizations of music excerpts and human movement ? Continuous measures of emotions:  Continuous measures of emotions Music as induction technique Music and emotion: time-varying relationship Several indicators  Problem: conscious Vs unconscious measurements  Verbal report  Physiological measures  Coding of non-verbal behavior, subject interfaces: from sliders to multimodal interfaces Idea: laser pointer as semi-conscious interface through which movement to communicate an emotional experience generated by music A pilot experiment (1):  A pilot experiment (1) 20 subjects equipped with a laser pointer and asked to move it on a white wall in front of them while listening to music excerpts Stimuli: a set of classical music excerpts provided by GERG  Grouped in four characterizations defined on the basis of valence and energy: slow positive, slow negative, fast positive and fast negative A pilot experiment (2):  A pilot experiment (2) Method  Each subject listened to four music excerpts, one for each emotional characterization  Trajectories performed by the subjects moving the laser pointer on the wall have been recorded  Questionnaire to indicate emotions felt by subjects during listening to music Preliminary analysis:  Preliminary analysis Aim: to look for correlations among features of the trajectories performed by subjects with the laser pointer and emotional characterization of the music excerpt a subject was listening to Global and static analysis: integration of the laser trajectories over time  To obtain for each video file with the movement of the laser a bitmap summarizing the trajectory followed during the whole listening  Each bitmap represents a graphical subject response (GSR) from the listening of a single music excerpt  Is it possible to separate the GSRs in classes and to verify if these classes can be correlated with the characterization of the music excerpts? CLUSTERING ANALYSIS Extraction of global trajectories:  Extraction of global trajectories Patch summarizing the path followed by the laser pointer (Eyesweb platform) ..\Presentazione\Presentazione.eyw Identification and measure of trajectories features:  Identification and measure of trajectories features To identify a collection of descriptors  Related to specific features of the trajectory patterns  Angularity, rarefaction, spatial occupation, vertical symmetry, horizontal symmetry, central symmetry, compactness, lateral location,vertical location, angular tendency, spatial extension Providing measures for relevant trajectory features  Manual annotation  Unambiguous criteria  Patterns are evaluated with a value from 0 to 4 with respect to each specific feature  Five evaluators An example: angularity:  An example: angularity The trajectories drawn by the laser can be smooth (0) or angular (4)     Smooth trajectory: wavy, soft lines  Angular trajectory: direct, sharp, nervous lines Smooth trajectory Angular trajectory An example: rarefaction:  An example: rarefaction The pattern can be thick and intense (0) or rarefied (4)  White pixels / total pixels in the boundary rectangle  Thick trajectory: high degree of filling of the occupied space  Rarefied trajectory: low degree of filling of the occupied space Thick trajectory Rarefied trajectory Statistical analysis: mean of all the ratings of all the features for the four emotions :  Statistical analysis: mean of all the ratings of all the features for the four emotions  Useful for deciding how to realize a clustering analysis Hypotheses to be verified during the clustering analysis:  Hypotheses to be verified during the clustering analysis Angularity, rarefaction and compactness seem to explain the motor activation analyzed with this static and global analysis: critical features Slow patterns: low angularity, high rarefaction, low compactness Fast patterns: high angularity, low rarefaction, high compactness Clustering global trajectories:  Clustering global trajectories Eyesweb patch with a block implementing the K-Means algorithm Aim: to verify if the grouping create clusters that are consistent with the emotional characterizations of the music excerpts used to induce the emotions in the subjects Choice of the best type of clustering: two clusters, three features Two different classifications: fast/slow and positive/negative Results:  Results Fast/slow patterns explained by angularity only Positive and the negative patterns don’t distinguish from each others Subjects, moving the laser pointer, synchronize with the rhythm of the excerpts  If the velocity of the music increases, consequently the velocity of the arm movement increases as well as the direction changes frequency There could be a sort of correlation between characteristics of the music listened to and movement performed Resonance between music and motor activation Future developments :  Future developments Dynamic analysis of laser pointer trajectories: how can be correlated with the musical structure at different time scales Aim: to discover how rules can be established to recognize emotions of users Possible perspective: to contribute to define the role of attention in emotion-oriented systems such ECAs Applications:  Applications Motor rehabilitation Multimedia content analysis through novel affective interfaces (e.g. mobiles, embedded systems, new media) Music industry: music information retrieval from huge databases based on emotional responses Artistic and musical applications Cultural applications, museums, and science centers References:  References Camurri A., Hashimoto, S., Ricchetti, M., Trocca, R., Suzuki, K., and Volpe, G., (2000), “Eyesweb – Toward Gesture and Affect Recognition in Interactive Dance and Music Systems”, Computer Music Journal, 24:1, pp. 57-69, MIT Press, Spring 2000. Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., and Volpe, G., (2004), “Multimodal Analysis of Expressive Gesture in Music and Dance Performances”, in A.Camurri, G. Volpe, (Eds.), “Gesture-based Communication in Human-Computer Interaction”, LNAI 2915, Springer Verlag, 2004. Camurri, A., De Poli, G., Leman, M., and Volpe, G., (2005), “Communicating Expressiveness and Affect in Multimodal Interactive Systems”, IEEE MultiMedia, January-March 2005, pp.43-53. Scherer, K.R., (1984), “On the nature and function of emotion: a component process approach”, in K.R. Scherer & P. Ekman (Eds.), Approaches to emotion (pp.293-317). Hillsdale, NJ: Erlbaum. Scherer, K.R., (2000), “Emotions as episodes of subsystem synchronization driven by nonlinear appraisal processes”, in Lewis, M. & Granic, I. (Eds.) Emotion, Development, and Self-Organization (pp. 70-99). New York/Cambridge: Cambridge University Press. Scherer K.R., Zentner M.R., (2001), “Emotional effects of music: production rules”, In P.N. Juslin & J.A.Sloboda (Eds). Music and emotion: Theory and research (pp. 361-392). Oxford: Oxford University Press.

Related presentations


Other presentations created by aksu

Student Led Conferences
17. 12. 2007
0 views

Student Led Conferences

ch07rev
24. 02. 2008
0 views

ch07rev

Fowad Murtaza
28. 11. 2007
0 views

Fowad Murtaza

ACCA Quality Assurance
06. 11. 2007
0 views

ACCA Quality Assurance

Speech Power GBS
07. 11. 2007
0 views

Speech Power GBS

Verlinden
07. 11. 2007
0 views

Verlinden

Japan and World War II
13. 11. 2007
0 views

Japan and World War II

Southwick History no sound
14. 11. 2007
0 views

Southwick History no sound

450 Ch9
20. 11. 2007
0 views

450 Ch9

palmer
21. 11. 2007
0 views

palmer

elephants
26. 11. 2007
0 views

elephants

Goldfisch New Varieties
14. 12. 2007
0 views

Goldfisch New Varieties

DictatorsPPT
24. 12. 2007
0 views

DictatorsPPT

Tiorco
02. 01. 2008
0 views

Tiorco

HYBRIDIZATION w3
04. 01. 2008
0 views

HYBRIDIZATION w3

E97 Presentation Maintenance
01. 11. 2007
0 views

E97 Presentation Maintenance

Caesar
07. 01. 2008
0 views

Caesar

Heavy Interactions
27. 09. 2007
0 views

Heavy Interactions

NashvilleBid2008
28. 11. 2007
0 views

NashvilleBid2008

lec9
29. 12. 2007
0 views

lec9

ferrari nicola lngs
03. 12. 2007
0 views

ferrari nicola lngs

cartilage students
30. 12. 2007
0 views

cartilage students

infant training
06. 03. 2008
0 views

infant training

Joseph Bremer
14. 03. 2008
0 views

Joseph Bremer

guzelotesi
02. 10. 2007
0 views

guzelotesi

1120122139 11 40 12 00
18. 03. 2008
0 views

1120122139 11 40 12 00

PHS3APAOC
26. 03. 2008
0 views

PHS3APAOC

2006 05 26 1
27. 03. 2008
0 views

2006 05 26 1

InvestInIsrael
07. 04. 2008
0 views

InvestInIsrael

Davis SEWPtalk2005 10 20
30. 03. 2008
0 views

Davis SEWPtalk2005 10 20

lrp final
09. 04. 2008
0 views

lrp final

GEO205 powerpoint 11
10. 04. 2008
0 views

GEO205 powerpoint 11

karnups4 russian crisis 1998
13. 04. 2008
0 views

karnups4 russian crisis 1998

milton
03. 01. 2008
0 views

milton

educause2001
03. 10. 2007
0 views

educause2001

What is Range
31. 12. 2007
0 views

What is Range

DAVOS
04. 01. 2008
0 views

DAVOS

4 1 Fitzwater
27. 09. 2007
0 views

4 1 Fitzwater