rusko

Information about rusko

Published on August 9, 2007

Author: Pumbaa

Source: authorstream.com

Content

SPEECH IS MORE THAN ONLY ITS LINGVISTIC CONTENT:  SPEECH IS MORE THAN ONLY ITS LINGVISTIC CONTENT Institute of Informatics of the Slovak Academy of Sciences Dubravska cesta 9, 847 05 Bratislava, Slovakia [email protected] Rusko Milan Institute of Informatics of the Slovak Academy of Sciences Expressive speech :  Expressive speech 'Expressive speech' designates the whole vocal display of a speaker. It consists: Linguistic information part of information that can be encoded in general written text message Various additional information on the speaker – age, cultural background, education, sex, attempt, relation to the listener, individuality etc. (The expression 'individuality' is used here to denote personality, mood (attitude) and emotions of a speaker.) Expressive speech :  Expressive speech Expresion =andgt; =andgt; Impression - - andgt; SPEECH - - andgt; Personality (and temperament):  Personality (and temperament) Personality is considered to be a set of constant features of an individual. Temperament is that aspect of personality that is genetically based, inborn. . Ancient Greeks – 2 dimensions of temperament =andgt; 4 types of temperament: sanguine type (cheerful and optimistic, pleasant to be with) choleric type (quick, hot temper, often an aggressive nature) phlegmatic type (characterized by slowness, laziness, and dullness) melancholy type (sad, even depressed, pessimistic view of world) Generalized model of personality:  Generalized model of personality personality p have n dimensions, and so it can be represented by a following vector (Egges, A., Kshirsagar, S., Magnenat-Thalmann, N. [2]: . The OCEAN model„The Big Five“ model of personality:  The OCEAN model „The Big Five' model of personality . Traditional psychological classification of personality dimensions Five Factor Model [Digman 1990, Mc.Rae, John 1992]:  Traditional psychological classification of personality dimensions Five Factor Model [Digman 1990, Mc.Rae, John 1992] Mood and Emotion:  Mood and Emotion Mood (attitude) can be defined as a rather static state of being, that is less static than personality and less fluent than emotions. Mood can be defined as one-dimensional (e.g. good or bad mood) or perhaps multi-dimensional (feeling in love, being paranoid etc.) (Ksirsagarandamp;Magnenat-Thalmann[5]) Generalized model of emotion:  Generalized model of emotion An emotional state has a similar structure as personality, but it changes over time. Defined as an m-dimensional vector, where all m emotion intensities are represented by a value in the interval [0,1] . The actual emotional state is dependent on the preliminary evolvement of emotins. A need to model the emotins respecting their previous trends (history). An emotional state history ωt is defined, that contains all emotional states until et, thus : Generalized model of mood:  Generalized model of mood Egges continues with defining the individual ITas a triple (p, mt, et), where mt represents the mood of the individual at a time t. Mood dimension is defined as a value in the interval [-1,1]. k mood dimensions =andgt; the mood can be described as follows: The mood and emotional values are changing in time =andgt; Both have to be updated regularly. Basic emotions:  Basic emotions There are many theories of emotions and many different classifications exist. This table, taken from Ortony, A., Turner, T. J. [6] gives a short overview of basic emotion sets used by different authors. Placement on emotion dimensions:  Placement on emotion dimensions Pleasure Happy andlt;======andgt; Unhappy Pleased andlt;======andgt;Annoyed Satisfied andlt;======andgt;Unsatisfied Contented andlt;======andgt;Melancholic Hopeful andlt;======andgt;Despairing Relaxed andlt;======andgt; Bored Arousal Stimulated andlt;======andgt; Relaxed Excited andlt;======andgt;Calm Frenzied andlt;======andgt; Sluggish Jittery andlt;======andgt; Dull Wide-awake andlt;======andgt;Sleepy Aroused andlt;======andgt;Unaroused Dominance Controlling andlt;======andgt; Controlled Influential andlt;======andgt;Influenced In control andlt;======andgt; Cared-for Important andlt;======andgt; Awed Dominant andlt;======andgt;Submissive Autonomous andlt;======andgt; Guided Semantic differential scales are often used for measuring emotion dimensions. A Set of dimensions as proposed by Mehrabian andamp; Russell (1974, Appendix B, p. 216)[7]. It is evident that the authors have included moods and personality dimensions in this system too. Acoustic correlates of emotions:  Acoustic correlates of emotions Problem: speech parameters involved in expression of personality, moods and emotions are shared for all the components of expressivity. Decoding the expressive speech code is very subjective. Nevertheless, a general set of the speech parameters responsible for the expression of emotion can be constructed. There are three main categories of speech correlates of emotion: • Pitch contour • Timing • Voice quality It is believed that value combinations of these speech parameters are used to express vocal emotion.(Schröder M.[8]) Pitch contour :  Pitch contour Pitch contour is a representation of the intonation of an utterance, which describes the nature of accents and the overall pitch range of the utterance. Pitch is expressed as fundamental frequency (F0). One of the most frequently used methods for F0 measurement is the method using autocorrelation function of the LP residual. Parameters include average pitch, pitch range, contour slope, and final lowering. Intonation contour:  Intonation contour Models of intonation - two main categories: Phonetic Phonological The phonetic models (e.g. Fujisaki model, Tilt model, MOMEL and many others) model the intonation curve. The phonological model (e.g. ToBI) is used to model the speaker's concept of distribution of accents in the intonational phrase. Automatic intonation contour analysis in Fujisaki editor:  Automatic intonation contour analysis in Fujisaki editor Pitch contour analysis in PRAAT with ToBI labels:  Pitch contour analysis in PRAAT with ToBI labels Timing:  Timing Timing Speed that an utterance is spoken Rhythm Duration of emphasized syllables The results of measurement of syllable and phoneme lengths are often given in a form of z-scores (the instantaneous value is normalized be the mean value of the same elements in the whole database. Parameters: speech rate, hesitation pauses, exaggeration... Voice quality :  Voice quality Voice quality denotes the overall ‘character’ of the voice, which includes effects such as whispering, hoarseness, breathiness, and intensity. The voice quality is influenced mainly by: function of glottis function of the vocal tract A detailed classification scheme was published by Laver [9]. Slide20:  Analysis of the glottal function:  Analysis of the glottal function The analysis of the glottal function is generally done using source-filter model of speech production [10]. The glottal function is obtained from the speech signal by inverse filtering. One of the most efficient inverse filtering methods uses Discrete Linear Prediction – DLP (El-Jaroudi A., Makhoul J., [11]) to obtain the inverse filter coefficients and to filter the speech signal. The resultant DLP residual function is considered as a representative of a derivative of glottal volume velocity function. Time and spectral domain characteristics of the glottal function:  Time and spectral domain characteristics of the glottal function Time characteristics OQ, Open Quotient – ratio of the open phase of the glottal waveform to the period of the pulse. OQ predicts the values for the amplitudes of the lower harmonics. (increased value of OQ is correlated with an increase in the amplitude of the lower harmonics in the voice spectrum.) CQ, Closing Quotient – ratio of the closing phase of the glottal pulse to the period of the pulse. These characteristics has been recently often replaced by AQ – Amplitude quotient and NAQ-Normalized amplitude quotient (Alku [12]). EE, Excitation Strength – amplitude of the negative peak, calculated after the positive peak. EE is correlated with the overall intensity of the signal. A decrease in EE is correlated with a breathy voice. RK, Glottal Symmetry/Skew – ratio of the closing phase to the opening phase of the differentiated glottal pulse. RK affects mainly the lower harmonics; the more symmetrical the pulse, the greater their amplitude. Spectral characteristics H1-H2– the amplitude of the first harmonic (H1) compared to the amplitude of the second harmonic (H2). An indicator of the relative length of the opening phase of the glottal pulse (Hanson 1997). H1-A1– the amplitude of the first harmonic (H1) compared to the strongest harmonic in the first formant (A1). Reflects the first formant bandwidth spectral tilt - Expected to be large and positive for breathy voices and small and/or negative for creaky voices H1-A2– the amplitude of the first harmonic (H1) compared to the amplitude of the strongest harmonic in the second formant (A2). An indicator of spectral tilt at the mid formant frequencies. Large and positive for breathy voices and small and/or negative for creaky voices. H1-A3– the amplitude of the first harmonic (H1) compared to the amplitude of the strongest harmonic in the third formant (A3). An indicator of spectral tilt at the higher formant frequencies. Large and positive for breathy voices and small and/or negative for creaky voices. Glottal pulse analysis in APARAT:  Glottal pulse analysis in APARAT Analysis of the vocal tract:  Analysis of the vocal tract Methods of vocal tract shape estimation include x-ray, computer tomography and magnetic resonance methods. stationary sound production only .Cheaper and quicker method – computing of the vocal tract shape from the speech signal complementary to glottal pulse analysis from the speech signal. (e.g. vocal tract shape computation from LPC derived reflection coefficients). - allows for analysis of the dynamic behavior of the articulators. Similar information can be obtained by formant analysis using homomorphic deconvolution (cepstrum) or LPC spectrum analysis. Static analysis by synthesis using articulatory synthesizer :  Static analysis by synthesis using articulatory synthesizer (TRACTSYN) Dynamic analysis by synthesis (articulatory synth. TRACTSYN):  Dynamic analysis by synthesis (articulatory synth. TRACTSYN) Acoustic correlates of emotions applied in speech synthesis:  Acoustic correlates of emotions applied in speech synthesis Vision: Speech Sound Mining:  Vision: Speech Sound Mining Aim: to extract information from supra-segmental and extra-linguistic layers Where to look for information: time domain a) quantity (lengths of segments) b) rhythm frequency domain a) long term characteristics b) short term characteristics model based characteristics a) glottal excitation function b) articulatory model Vision: Speech Sound Mining:  Vision: Speech Sound Mining How to define a set of speech sound objects? Objective methods of analysis (pattern recognition) Subjective methods (impression of the listener) Possible objects: Speech sound event Speech sound act Speech sound gesture Speech sound characteristic Speech sound characteristic change Vision: Speech Sound Mining:  Vision: Speech Sound Mining First steps to be accomplished: Speech corpus building Annotation of SSO Boundary markers Frequencies of occurence of SSO Concordances of SSO Correlation among different sets of objects (pitch SSO, accent SSO, rhythmic SSO, timbre SSO, etc.) Semantic representation of SSO Cross cultural semantic analysis Vision: Speech Sound Mining:  Vision: Speech Sound Mining Traditional methods used in NLP and data mining will be applicable: Bag of words  Bag of SSO WordNet  SSO semantic net e.t.c. Research on the relation between lingvistic and paralingvisticandamp;extralingvistic information. Creation of a complex (holistic) model of the speech signal as an information carrier in communication. Thank you for your attention:  Thank you for your attention Milan Rusko Institute of Informatics Slovak Academy of Sciences [email protected]

Related presentations


Other presentations created by Pumbaa

christmas
16. 08. 2007
0 views

christmas

VERTEBRATES
12. 10. 2007
0 views

VERTEBRATES

Lec1Ch1 2IntroandHardware
15. 10. 2007
0 views

Lec1Ch1 2IntroandHardware

naos harbor project
22. 10. 2007
0 views

naos harbor project

MLA
05. 09. 2007
0 views

MLA

Bronx health disparities
05. 09. 2007
0 views

Bronx health disparities

writewithppt
05. 09. 2007
0 views

writewithppt

Persuading
05. 09. 2007
0 views

Persuading

JF KENNEDY
23. 10. 2007
0 views

JF KENNEDY

M3infectiousdisease sept
23. 10. 2007
0 views

M3infectiousdisease sept

Gerard Fries
24. 10. 2007
0 views

Gerard Fries

hen
04. 10. 2007
0 views

hen

prworkshop 07
02. 11. 2007
0 views

prworkshop 07

reptiles
26. 10. 2007
0 views

reptiles

csf
02. 11. 2007
0 views

csf

Turin OIAs
14. 11. 2007
0 views

Turin OIAs

Chapter15Personality1
17. 11. 2007
0 views

Chapter15Personality1

Passing Off
16. 08. 2007
0 views

Passing Off

Resurrection Slides
16. 08. 2007
0 views

Resurrection Slides

crucifixion
16. 08. 2007
0 views

crucifixion

MMRv2004PPT
28. 12. 2007
0 views

MMRv2004PPT

SlideShow2006 web
03. 01. 2008
0 views

SlideShow2006 web

17 a b
07. 10. 2007
0 views

17 a b

LBA2000
29. 10. 2007
0 views

LBA2000

nchrp w43
05. 01. 2008
0 views

nchrp w43

Personality Disorders
09. 08. 2007
0 views

Personality Disorders

plenary4slides
09. 08. 2007
0 views

plenary4slides

chuang
09. 08. 2007
0 views

chuang

AALAS 02
07. 11. 2007
0 views

AALAS 02

Test score gaps Rev
05. 09. 2007
0 views

Test score gaps Rev

Presentacion PFIF Mar 2005
22. 10. 2007
0 views

Presentacion PFIF Mar 2005

09 user generated content 200407
17. 10. 2007
0 views

09 user generated content 200407

T4 08
22. 10. 2007
0 views

T4 08

Fry1441Lec19
28. 12. 2007
0 views

Fry1441Lec19

ruggiero
05. 09. 2007
0 views

ruggiero

Ferhat Ozcam
26. 11. 2007
0 views

Ferhat Ozcam

OpeningRestrInNYC
05. 09. 2007
0 views

OpeningRestrInNYC

etacharacteristicspp
02. 10. 2007
0 views

etacharacteristicspp

TEVTA
14. 02. 2008
0 views

TEVTA

2 Unit 1 lifestyle
20. 02. 2008
0 views

2 Unit 1 lifestyle

gc
03. 01. 2008
0 views

gc

INDIA06 11 CHengevoss Military
04. 03. 2008
0 views

INDIA06 11 CHengevoss Military

oil moc nyc072407
05. 09. 2007
0 views

oil moc nyc072407

travel
10. 03. 2008
0 views

travel

loh intranets portals
11. 03. 2008
0 views

loh intranets portals

Parasitology
09. 08. 2007
0 views

Parasitology

APrisonEpistles
25. 03. 2008
0 views

APrisonEpistles

2006080704
26. 03. 2008
0 views

2006080704

THE FUTURE OF AVIATION
26. 03. 2008
0 views

THE FUTURE OF AVIATION

GrayHarborWarm
07. 04. 2008
0 views

GrayHarborWarm

econ 3171 ppt slides ch 17
09. 04. 2008
0 views

econ 3171 ppt slides ch 17

PandemicPreparedness compress
10. 04. 2008
0 views

PandemicPreparedness compress

FMLAmediaframing
13. 04. 2008
0 views

FMLAmediaframing

P1D IntroReview 121407
16. 04. 2008
0 views

P1D IntroReview 121407

Gold Mine Pesentation
17. 04. 2008
0 views

Gold Mine Pesentation

IFM Barnhill 06 12 2001
22. 04. 2008
0 views

IFM Barnhill 06 12 2001

ce presentation
29. 02. 2008
0 views

ce presentation

lastdays welsh
16. 08. 2007
0 views

lastdays welsh

coop ppp
23. 11. 2007
0 views

coop ppp

ELL NM
09. 08. 2007
0 views

ELL NM

cbsss3
15. 10. 2007
0 views

cbsss3

Passion2
09. 08. 2007
0 views

Passion2

Personality Disorders Handout
09. 08. 2007
0 views

Personality Disorders Handout

Gaukinlec Marxism
14. 12. 2007
0 views

Gaukinlec Marxism

SÃtningsled
26. 11. 2007
0 views

SÃtningsled

buspres usA
29. 12. 2007
0 views

buspres usA

a thai way
16. 06. 2007
0 views

a thai way

MSG350 laahs SW2 show notes
09. 08. 2007
0 views

MSG350 laahs SW2 show notes

howe project
13. 03. 2008
0 views

howe project

PrayersandWritingsSt Edmund
09. 08. 2007
0 views

PrayersandWritingsSt Edmund

joshi revised
03. 10. 2007
0 views

joshi revised

harkins ref data
05. 09. 2007
0 views

harkins ref data

3474
02. 01. 2008
0 views

3474

bruschi
08. 10. 2007
0 views

bruschi

115 tunnelling 2006
15. 11. 2007
0 views

115 tunnelling 2006

SectorMeeting PrivateSchools
05. 09. 2007
0 views

SectorMeeting PrivateSchools

070227 JointBudgetHearingFi nal
05. 09. 2007
0 views

070227 JointBudgetHearingFi nal

DerryHymanRoughDraft
07. 12. 2007
0 views

DerryHymanRoughDraft

2Hochstein Trends
21. 11. 2007
0 views

2Hochstein Trends

HJepardy
16. 08. 2007
0 views

HJepardy

dcc life cycle
09. 08. 2007
0 views

dcc life cycle

Presentation Liu
16. 10. 2007
0 views

Presentation Liu

sr AkzoNobel4
12. 10. 2007
0 views

sr AkzoNobel4

s409 guha
03. 01. 2008
0 views

s409 guha