WETICE2004 Albrechtsen

Information about WETICE2004 Albrechtsen

Published on March 3, 2008

Author: Marco1

Source: authorstream.com

Content

Slide1:  Work Centered Evaluation of Collaborative Systems – the Collate Experience Hanne Albrechtsen Cognitive Systems Engineering Centre Risoe National Laboratory Roskilde Denmark Outline :  Outline The Collate Collaboratory Evaluation Main results The Collate Collaboratory :  The Collate Collaboratory The problem of cross-cultural film archive research: Valuable historic document collections exist but scattered in national archives Sources mostly not available online Lack of access to content Valuable expert domain knowledge exists but mostly inaccessible to externals Tacit knowledge, insufficiently documented Lack of technology support for collaborative research The Collate Collaboratory :  The Collate Collaboratory Goals: To construct a “Collaboratory in use” Preserve historic documents in a distributed repository Content: European film documentation (20ies and 30ies) Metadata (cataloguing, indexing and annotation) Content providers / pilot users: Deutsches Filminstitut – DIF, Frankfurt, Germany Filmarchiv Austria, Vienna, Austria Národní Filmový Archiv, Prague, Czechia Technology developers: Fraunhofer IPSI, Darmstadt, Germany University of Bari, LACAM Lab, Bari, Italy Sword ICT S.r.l., Bari, Italy Evaluation partner: Risø National Laboratory, Systems Analysis Dept, Denmark Digital Repository:  Digital Repository Data Historic film censorship documents (legal docs, application & decision forms, correspondence, etc.) Press material Photos (stills, portraits) Film posters Digital video fragments Metadata – value-added information by multi-national film experts  Indexing & annotation Slide7:  Censorship documents Cataloguing and Keyword Indexing:  Cataloguing and Keyword Indexing Slide9:  Evaluation of web-based collaboratory Evaluation framework:  Evaluation framework 1. Conceptual framework and method: Cognitive work analysis Frameworks for work analysis and prototype evaluation Work centered approach a. Analytical evaluation: Is the design right? Input: evaluators examine interface Laboratory experiments - ‘think aloud’, eye-tracking b. Empirical evaluation: Is it the right design? Input: Empirical work studies and experience from field experiments Cognitive work analysis and empirical analysis of archival work:  Cognitive work analysis and empirical analysis of archival work 1. Goals: To evaluate how the Collate prototypes support film archive research (work content), collaboration and knowledge sharing; acceptability 2. Inputs: Phase 1: Field studies of three film archives involved in Collate Input for empirical evaluation: identification of recurrent course of events in collaboratory film research (construction and mediation of knowledge) Phase 2: Field experiments on the course of events in collaborative research task scenario by use of Collate prototypes (at evaluation workshop) Cognitive work analysis: Levels of work analysis :  Cognitive work analysis: Levels of work analysis Cognitive work analysis: Prototype evaluation:  Cognitive work analysis: Prototype evaluation The means ends abstraction hierarchy :  The means ends abstraction hierarchy Slide15:  Empirical evaluation: Evaluation workshop Focus on the archivists’ (collaborative) work with the censorship material and the prototype’s support of: Common strategies Collaborative decisions Means and ends of common workspace Slide16:  Design of evaluation workshop Participants: Two expert representatives from each of three Collate archives Evaluation team from Risø (four evaluators) Study method: Participant observation of collaborative research task scenario: To produce a censorship history of two films, using the Collate prototype (Film 1: Potemkin; film 2: Die drei von der Tankstelle) Settings and events: Introductory session for all Two laboratory sessions, each for one film Two groups of archive representatives and evaluators Concluding focus interview, including debriefing of group work Ethics: Aware participants (purpose of experiment known to all) Slide17:  Design of collaborative task scenario for evaluation workshop Collaborative task: To produce a censorship history of two films Panzerkreuzer Potemkin, version 1930 Die Drei von der Tankstelle Suggested activities: Find censorship documents and materials (search) Coordinate the work Link documents and materials (cataloguing and indexing) Write your comments (annotation) Find keywords for comments Categorise comments Enter meta-information: comments on comments Slide18:  Session at the evaluation workshop Empirical analysis:  Empirical analysis Dimensions of the framework used for empirical analysis of collaborative knowledge integration:  Dimensions of the framework used for empirical analysis of collaborative knowledge integration What are the archivists’ common strategies for problem solving and how does the prototype support them? What are the archivists’ collaborative decisions and how does the prototype support them? What are the means and ends of the common workspace for knowledge integration and how does the prototype support them? Slide21:  Common Strategies: Choose a core document Slide22:  Collaborative Decisions: Interpretation Collaborative decisions: 2. Assign keywords:  Collaborative decisions: 2. Assign keywords Common workspace:  Common workspace Slide25:  Main results from evaluation workshop User experiences Slide26:  Main results from empirical evaluation - reflections Annotation module: Main advantages: Creativity and innovation of film archival research; new professional networking; innovations in research concepts and communication Main challenges: 1. Diverse conceptions of the value of making previously’ invisible’ work public; role allocations; ‘write comments-or-perish’; 2. The risk of implicit formalisation of informal work activities and professional networking Indexing module: Main advantages: Standardisation of categories and terminology for subject access to digital collection Main challenges: 1. Diverse understandings of concepts and terminology amongst archive staff; diverse professional and personal approaches to indexing work 2. (How) will end-users understand singular representations of collaboratory semantics? Slide27:  Main results from empirical evaluation of knowledge integration by use of the Collate prototype Indexing feature in the collaboratory: Main advantages: Standardisation of categories and terminology for subject access to digital collection Main challenges: 1. The archivists have developed new strategies: from focus on singular documents to censorship history and case -> choose a core document 2. The archivists need to establish additional frames of reference for interpretation of single documents (cultural, historical, research perspectives) -> deeper semantics of work 3. The new deep semantics of knowledge integration goes across the divide between an indexing module on the one hand and an annotation module on the other hand Slide28:  Future research avenues 1. Categorical complexity in computer-mediated collaborative research: Collaborating actors’ intersecting and diverse conceptions of the semantics of work (examples: annotation and indexing in Collate collaboratory) Aim: Models for work-based design of tools for knowledge exchange and organisation in (collaborative) research 2. Collaborative task situations as a main unit of analysis for design of research collaboratories: Prototypical task situations and the users’ need for computer-mediated support Aim: Construction of common work spaces for collaborative integration of knowledge and as test beds for technology development Censorship Keyword Map:  Censorship Keyword Map COLLATE System Layers:  COLLATE System Layers text docs photos annotations videos <XML metadata> <XML metadata> <XML metadata> <XML metadata> Metadata management cataloguing annotation interlinking indexing terminology development Structural manipulations internal external external Scientific discourse Interaction Collaboration Layer Task Layer Domain Metadata Layer Operational Layer Interface Layer

Related presentations


Other presentations created by Marco1

Introduction to Graph Rewriting
10. 01. 2008
0 views

Introduction to Graph Rewriting

shoretel6 1
08. 01. 2008
0 views

shoretel6 1

0907
09. 01. 2008
0 views

0907

indi food
10. 01. 2008
0 views

indi food

barclays
11. 01. 2008
0 views

barclays

EVERYONES BUSINESS
12. 01. 2008
0 views

EVERYONES BUSINESS

newbrnresus
15. 01. 2008
0 views

newbrnresus

Information Campaign Salvi
15. 01. 2008
0 views

Information Campaign Salvi

24346
15. 01. 2008
0 views

24346

Wildlife  And  Conservation
15. 01. 2008
0 views

Wildlife And Conservation

Baxter
16. 01. 2008
0 views

Baxter

tyner purdue
23. 01. 2008
0 views

tyner purdue

italian renaissance
05. 02. 2008
0 views

italian renaissance

The Tudors
11. 02. 2008
0 views

The Tudors

Attitudes 2
17. 01. 2008
0 views

Attitudes 2

Textiles
25. 01. 2008
0 views

Textiles

Bristol MurderXNtMvFSZ
28. 01. 2008
0 views

Bristol MurderXNtMvFSZ

McCarthy
30. 01. 2008
0 views

McCarthy

2 Saving Money Time HEEP Area 1
07. 02. 2008
0 views

2 Saving Money Time HEEP Area 1

5WR N Africa SW Asia
13. 02. 2008
0 views

5WR N Africa SW Asia

Fundraising
15. 02. 2008
0 views

Fundraising

gabidimeworkshop
22. 01. 2008
0 views

gabidimeworkshop

Hmong American Elders
29. 01. 2008
0 views

Hmong American Elders

R2 Solutions
20. 02. 2008
0 views

R2 Solutions

Festivals around the world
22. 02. 2008
0 views

Festivals around the world

Mercury Mars Venus
24. 01. 2008
0 views

Mercury Mars Venus

BookofMormonChapter7 8and9
12. 02. 2008
0 views

BookofMormonChapter7 8and9

CNE354HellGreece4
25. 02. 2008
0 views

CNE354HellGreece4

Lessons from Bees
25. 02. 2008
0 views

Lessons from Bees

PS 280 13 mood disorders
27. 02. 2008
0 views

PS 280 13 mood disorders

ASEE Annual Conf 2005 OFI
14. 01. 2008
0 views

ASEE Annual Conf 2005 OFI

fitness
29. 02. 2008
0 views

fitness

PrimalSeasCvt
11. 02. 2008
0 views

PrimalSeasCvt

Gregory Thomas PPT
14. 03. 2008
0 views

Gregory Thomas PPT

Poster Template FOR VOLUNTEERS
27. 03. 2008
0 views

Poster Template FOR VOLUNTEERS

Respirators
22. 01. 2008
0 views

Respirators

All about me
14. 04. 2008
0 views

All about me

The Popcorn Process
07. 02. 2008
0 views

The Popcorn Process

79989
18. 04. 2008
0 views

79989

ASFP
21. 04. 2008
0 views

ASFP

Forest3 302
22. 04. 2008
0 views

Forest3 302

auxFundamentals
24. 04. 2008
0 views

auxFundamentals

ShowalterPPT
11. 01. 2008
0 views

ShowalterPPT

umist
04. 02. 2008
0 views

umist

L7 global
08. 05. 2008
0 views

L7 global

unit 1 2
19. 03. 2008
0 views

unit 1 2

2005121133941157
30. 04. 2008
0 views

2005121133941157

sykianaki1 10d04
02. 05. 2008
0 views

sykianaki1 10d04

fraseri
02. 05. 2008
0 views

fraseri

ORNISBGCore2007
05. 02. 2008
0 views

ORNISBGCore2007

agriscience basic ag awarness
24. 01. 2008
0 views

agriscience basic ag awarness

Community Greening Training
10. 03. 2008
0 views

Community Greening Training

introToIR
17. 01. 2008
0 views

introToIR

ingels 160305
24. 01. 2008
0 views

ingels 160305

Delhot
10. 01. 2008
0 views

Delhot

olimpic games
14. 04. 2008
0 views

olimpic games

mmo 32
05. 03. 2008
0 views

mmo 32

11 04
04. 02. 2008
0 views

11 04

cccarlos
28. 02. 2008
0 views

cccarlos

funding Opps and guidance
21. 01. 2008
0 views

funding Opps and guidance

session 29 a Karen Donnet Jones
17. 01. 2008
0 views

session 29 a Karen Donnet Jones

XueJishan Boulder 2006 11
16. 04. 2008
0 views

XueJishan Boulder 2006 11

Covenant Counseling Presentation
28. 01. 2008
0 views

Covenant Counseling Presentation

LV PERF
20. 02. 2008
0 views

LV PERF

1866 07 E Dalen Geog
17. 01. 2008
0 views

1866 07 E Dalen Geog