swimming style classification from color video seq

Information about swimming style classification from color video seq

Published on April 17, 2008

Author: Tirone

Source: authorstream.com

Content

Swimming Style Classification from Color Video Sequences:  Swimming Style Classification from Color Video Sequences Speaker:Ming-Je Liu [email protected] Visual Information Processing Laboratory, Department of CS, National Cheng-Chi University Outline:  Outline Introduction System Framework Color-based Segmentation Connected Component Detection Regression Analysis Motion Classification Demo and Experimental Results Conclusions Objective:  Objective To classify swimming motion into four styles, namely, Back-stroke (仰式) Breast-stroke (蛙式) Butterfly (蝶式) Free style (自由式) using video taken with an above-water camera looking down as the swimmer is approaching. Motivation:  Motivation Two distinct purposes for analyzing sports video: To organize and segment the raw footage into semantic units for easy indexing and retrieval. To perform qualitative and quantitative analysis of human motion to identify weak points and offer suggestions for improvement. Key Assumptions:  Key Assumptions Constraints for source video clips: The swimmer is sited in a FINA Olympic standard pool with distinct lane ropes. The video is taken with an above-water camera looking down as the swimmer is approaching. There is only one swimmer within the camera’s field of view. System Framework:  System Framework Color-based Segmentation:  Color-based Segmentation Original color image The hue component RGB  HSV conversion. Label the pixels with a hue value between 0.3 and 1.5 0 / 2π 0.3 1.5 Segmented regions Connected Component Detection:  Connected Component Detection Locate connected components. Morphological filtering to remove scattered blobs. Exclude components that: Have centroids close to the boundaries Have size smaller than 500 pixels Only the two largest blobs are retained. Regression Analysis:  Regression Analysis To estimate the relative position of the limbs efficiently. Other possible approaches for carrying out the calculation: Thinning algorithms to reduce the blob to a single line and approximate its slope Hough transform Regression Analysis (Cont.):  Regression Analysis (Cont.) Divide X-axis and Y-axis of the points of a blob into two array: X = [ p1, p2,…, pN ], Y = [ q1, q2,…, qN ] Treating X as independent variable and Y as dependent variable, we can fit these points with a line , where Regression Analysis (Cont.):  Regression Analysis (Cont.) The correlation coefficient: where R can be used to remove lane rope blob. Regression Analysis (Cont.):  Regression Analysis (Cont.) Detected regions and corresponding regression lines: Motion Classification:  Motion Classification Decision tree: the rules are mainly derived from the characteristics of the upper body movements. Example: in certain subsequence of the backstroke style, the arms will be held in the vertical direction. Slope a will be large and aspect ratio (AR) will be small. A score of 10 is assigned to the frame’s backstroke score if |a|>3 , AR <0.4. Scoring system:  Scoring system Score assignment is based on the measure of proximity to the prominent features of a specific style. Four types of backstroke: Ambiguities:  Ambiguities Scoring tree:  Scoring tree # of regions a1 ∙a2 a, AR 2 1 Breaststroke(3) Breaststroke(1) Butterfly(5) Butterfly(8) Free style(5) Free style(6) Free style(8) Backstroke(5) Backstroke(6) Backstroke(8) Backstroke(10) |a|<0.3, AR<1 |a|<0.3, AR<2 |a|<0.3, AR<3.5 |a|<0.3, AR<4 0.3<|a|<1.3, 1.2<AR<2.2 0.5<|a|<1.1, 1.4<AR<2.0 0.7<|a|<0.9, 1.6<AR<1.8 |a|>0.9, AR<0.7 |a|>1.5, AR<0.6 |a|>2, AR<0.5 |a|>3, AR<0.4 … Scoring tree (Cont.):  Scoring tree (Cont.) # of regions a1 ∙a2 |a1+a2| Butterfly (8) Negative slope found on Butterfly (10) Butterfly (5) Breaststroke(5) a, AR … … … 2 1 >0 <0 <0.2 right left |a1|<0.5 and |a2|<0.5 Linear scoring system :  Linear scoring system A continuous scoring system is adopted to simplify the tree structure as well as increase the precision of the recognition results. A weighting factor is added to take into account the influence from the previous frames. Linear scoring system (Cont.):  Linear scoring system (Cont.) Weighting factor:  Weighting factor Complete cycle length:(NTSC Standard, 29.97frames/sec) Butterfly:28 frames Back-stroke:25 frames Breast-stroke:30 frames Freestyle:20 frames Since it only makes sense to relate the motion patterns within one cycle, k should be less than 20. In practice, k is usually set between 5 and 10 to account for dropped frames. Weighting factor (cont.):  Weighting factor (cont.) if (count[fly] > count [breast]) weight[breast] *= (1-weight[fly]); else if(count [back] > count [breast]) weight[breast] *= (1-weight[back]); else if(count [free] > count [breast]) weight[breast] *= (1-weight[free]); else if(count [breast] > count [fly]) weight[fly] *= (1-weight[breast]); else if(count [breast] > count [back]) weight[back] *= (1-weight[breast]); else if(count [breast] > count [free]) weight[free] *= (1-weight[breast]); Fine-tuning the weight: Implementation:  Implementation Borland® C++ builder 6.0 Analyze existing video clips or live DV input through IEEE 1394. Processing the input data at a rate over 20 frames per second using a computer with a Pentium 4® 1.7 GHz CPU. Software available for download at: http://vision.cs.nccu.edu.tw/SSC Demo:  Demo Experimental Results:  Experimental Results We have tested our system with a collection of about 50 video clips obtained from http://swim.ee We obtain robust classification results for all the video clips that satisfy the constraints discussed previously. Experimental Results (Cont.):  Experimental Results (Cont.) Experimental Results (Cont.):  Experimental Results (Cont.) Experimental Results (Cont.):  Experimental Results (Cont.) Experimental Results (Cont.):  Experimental Results (Cont.) Experimental Results (Cont.):  Experimental Results (Cont.) Conclusions:  Conclusions The features extracted from the upper body parts are sufficient for classifying swimming style. The proposed decision tree and scoring system prove to work well for video clips satisfying proper constraints. Future Work:  Future Work Detection of the turn movement to enable analysis of IM (individual medley). The ability to identify the beginning and end of a motion cycle. Combine above-water and under-water video to analyze whole body movements. Backward problem:  Backward problem The scoring system is no longer valid due to the design of swimming suit the limbs are sheltered from water splash Possible solution: find Textured Motion of water splash use water splash as another feature to build a new scoring system Q & A:  Q & A

Related presentations


Other presentations created by Tirone

Orthographic Projection
09. 01. 2008
0 views

Orthographic Projection

Strategic Stdzn seminar OAS
08. 05. 2008
0 views

Strategic Stdzn seminar OAS

EGDI presentation
07. 05. 2008
0 views

EGDI presentation

Item 7 Appendix EESCP
02. 05. 2008
0 views

Item 7 Appendix EESCP

0316CIPFA olympics
02. 05. 2008
0 views

0316CIPFA olympics

tom shepard
30. 04. 2008
0 views

tom shepard

worldhunger
24. 04. 2008
0 views

worldhunger

Richard Dunn
22. 04. 2008
0 views

Richard Dunn

MILE 0409
21. 04. 2008
0 views

MILE 0409

koreanhistory
18. 04. 2008
0 views

koreanhistory

Unit V Civil War
14. 01. 2008
0 views

Unit V Civil War

nutrition
07. 02. 2008
0 views

nutrition

Ancient Greek Drama
15. 02. 2008
0 views

Ancient Greek Drama

roofframing
11. 01. 2008
0 views

roofframing

chemicalhazards
14. 01. 2008
0 views

chemicalhazards

anesthesie 2006 cornet
15. 01. 2008
0 views

anesthesie 2006 cornet

prepandehhazards
18. 01. 2008
0 views

prepandehhazards

200105Craft
19. 01. 2008
0 views

200105Craft

rcra4
20. 01. 2008
0 views

rcra4

presentation tatt
17. 01. 2008
0 views

presentation tatt

Galvatech 04 Presentation
22. 01. 2008
0 views

Galvatech 04 Presentation

turner
23. 01. 2008
0 views

turner

ASC Raws0606 final 041906
23. 01. 2008
0 views

ASC Raws0606 final 041906

Day2 091002
24. 01. 2008
0 views

Day2 091002

Jubilee Presentation Bob Smith
04. 02. 2008
0 views

Jubilee Presentation Bob Smith

April 6 2005
04. 02. 2008
0 views

April 6 2005

Mercedes
05. 02. 2008
0 views

Mercedes

060423am
25. 01. 2008
0 views

060423am

McGrath Toronto2
29. 01. 2008
0 views

McGrath Toronto2

English project presentation
14. 02. 2008
0 views

English project presentation

pps 329
14. 02. 2008
0 views

pps 329

2002 MNM Fatalilties
22. 02. 2008
0 views

2002 MNM Fatalilties

jfm DAY2 WEB
10. 01. 2008
0 views

jfm DAY2 WEB

PORTRAITS Photographers
25. 02. 2008
0 views

PORTRAITS Photographers

DiabetesandPregnancy 000
28. 02. 2008
0 views

DiabetesandPregnancy 000

A Ryan
03. 03. 2008
0 views

A Ryan

Roman Art
03. 03. 2008
0 views

Roman Art

Pro2
17. 01. 2008
0 views

Pro2

Course chp sv
22. 01. 2008
0 views

Course chp sv

Hantavirus
25. 03. 2008
0 views

Hantavirus

IoVolcanism
21. 01. 2008
0 views

IoVolcanism

Steinmetz Presentation
11. 02. 2008
0 views

Steinmetz Presentation

articles 71120 durrani
07. 04. 2008
0 views

articles 71120 durrani

Hugosson
30. 01. 2008
0 views

Hugosson

CATBREEDS
13. 01. 2008
0 views

CATBREEDS

07 UseCases
11. 01. 2008
0 views

07 UseCases

l murdock
24. 01. 2008
0 views

l murdock

art study guide
25. 02. 2008
0 views

art study guide

empoverview
15. 01. 2008
0 views

empoverview

Stockholm Carnegie
12. 02. 2008
0 views

Stockholm Carnegie

39689158
09. 01. 2008
0 views

39689158

Kimmy Auto Dog Wash
10. 01. 2008
0 views

Kimmy Auto Dog Wash

file 33502505
28. 03. 2008
0 views

file 33502505

77449 55
05. 03. 2008
0 views

77449 55

saltlakecity05
13. 02. 2008
0 views

saltlakecity05

IED Lec 1 2007
06. 02. 2008
0 views

IED Lec 1 2007

proficiency
14. 01. 2008
0 views

proficiency

religiousemblems07
07. 02. 2008
0 views

religiousemblems07

CTW Presentation for Website
25. 01. 2008
0 views

CTW Presentation for Website