HALT conference 3 8 08 Kondo Brown

Information about HALT conference 3 8 08 Kondo Brown

Published on March 30, 2008

Author: Simo

Source: authorstream.com

Content

Program evaluation activities in the undergraduate Japanese program at the University of Hawaii at Manoa:  Program evaluation activities in the undergraduate Japanese program at the University of Hawaii at Manoa This four-year (2006-2010) project is funded by the National Resource Center East Asia (NRCEA) Kimi Kondo-Brown E-mail: [email protected] Menu for today’s presentation:  Menu for today’s presentation 1. Define “program evaluation” 2. Discuss the goals of the project 3. Give an overview (what has been done and what’s ahead?) 4. Provide information about the data collection instruments 5. Present data analyses conducted to date 6. Discuss the future uses of the evaluation data obtained from this project What is “program evaluation”?:  What is “program evaluation”? “The systematic collection and analysis of all relevant information necessary to promote the improvement of a curriculum and assess its effectiveness within the context of the particular institutions involved.” - JD Brown (1989, p. 223) - Goal 1. Evaluate whether or not Japanese language students at UHM demonstrate the expected proficiency goal after completing the 2-year foreign language requirement. :  Goal 1. Evaluate whether or not Japanese language students at UHM demonstrate the expected proficiency goal after completing the 2-year foreign language requirement. Regular track on spoken and written Japanese (15-16 credit hrs) JPN101 (100)  102  201  202 Oral communication track (12 credit hrs) JPN111  112  211  212 JPN 101 (4 credits, meets 4 times a week, for “true beginners” ) JPN 100 (3 credits, meets 3 times a week, for students with some language background, covers the same material as 101, covers more quickly) According the current JPN 202/212 course syllabi, the expected proficiency goals for students completing either track appear to be at the Intermediate-level on the ACTFL guidelines. Incoming students Goal 2: Identify concrete and adequate proficiency goals for undergraduate Japanese majors upon graduation:  Goal 2: Identify concrete and adequate proficiency goals for undergraduate Japanese majors upon graduation Graduating Japanese majors are required to complete 36 credit hrs: JPN 350 [Introduction to Japanese linguistics], 370 [Language in Japanese Society] JPN 401, 402, 407E, and one of 407B, 407C, 407D [advanced-level reading courses] EALL 271 and 272 [Japanese literature in translation] 12 credit hours in approved elective courses The proficiency goal for graduating majors in East Asian languages is very general : “At the undergraduate level, language skill courses are aimed at developing a high level of proficiency in both the spoken and written aspects of the languages. …” (2006-07 UHM catalogue) We need to identify concrete or measurable proficiency objectives. More than 30 elective courses to choose from. For example: 420 Fourth-Level Spoken Japanese (3) 421 Japanese Composition (3) 423 Advanced Listening and Speaking (3) 471 OkinawanLanguage and Culture I 495 Internship Program: Travel Industry Project overview: YEAR 1 (FALL 2006-SPRING 2007) :  Project overview: YEAR 1 (FALL 2006-SPRING 2007) the listening section of the Japanese placement test Project overview: YEAR 2 (FALL 2007-SPRING 2008):  Project overview: YEAR 2 (FALL 2007-SPRING 2008) Project overview: YEAR 3 (FALL 2008-SPRING 2009):  Project overview: YEAR 3 (FALL 2008-SPRING 2009) Project overview: YEAR 4 (FALL 2009-SPRING 2010):  Project overview: YEAR 4 (FALL 2009-SPRING 2010) The data collection instruments #1: The ACTFL proficiency guidelines (www.actfl.org/ ):  The data collection instruments #1: The ACTFL proficiency guidelines (www.actfl.org/ ) Language teachers, textbook authors, and test developers have increasingly adopted the ACTFL proficiency guidelines to establish curricular standards and graduation benchmarks. May not be generated from a particular theoretical construct or framework. As far as the ACTFL OPI is concerned, its validity, reliability, and usefulness for assessment purposes have been widely reported in recent literature (including Japanese). One of our plans is to have in-house certified ACTFL OPI testers and create a system where students' oral proficiency levels can be regularly assessed. Currently, in Hawaii, no certified ACTFL OPI testers in Japanese. From a research perspective, the data may give us valuable information about its usefulness as a testing device. The data collection instruments #2: The listening placement subtest :  The data collection instruments #2: The listening placement subtest The listening test has 38 multiple-choice items. Most items are at the Intermediate-level on the ACTFL guidelines. For each item, the students listen to a recorded conversation in Japanese. Each dialogue is read once and at a natural rate of speech, and is followed by one or two questions. The students answer the questions on the scan sheet provided. My study (Kondo-Brown, 2006) shows that the reliability estimate for the listening test was excellent (.93), and it was highly inter-correlated with two other placement subtests (.745 and .886). The placement data of incoming students who took the placement test in 2003 (N=136) :  The placement data of incoming students who took the placement test in 2003 (N=136) The data collection instruments #3: The STAMP reading test :  The data collection instruments #3: The STAMP reading test The STAMP is a computer-adaptive on-line test developed by the Center for Applied Second Language Studies. URL of the STAMP: http://casls.uoregon.edu/stamp2.php The current version is designed to discriminate those students who are within Novice-Low to Intermediate-High levels on the ACTFL guidelines. The results of the pilot STAMP reading test administered at UHM ( beginning of Spring 2005) :  The results of the pilot STAMP reading test administered at UHM ( beginning of Spring 2005) The pilot STAMP reading test was administered to 201, 202, 301 students (N=234). Note: Unfortunately, the 2005 version was not yet designed to discriminate those students who may be at the Intermediate-High or above levels. Some students who were judged to be I-M might have received higher scores if the instrument had been designed for higher proficiency levels. The data collection instruments #4: Language background and learning surveys:  The data collection instruments #4: Language background and learning surveys The survey data are collected to adequately interpret the test results. The background questionnaire askes a variety of questions such as family language background, previous Japanese language studies, visits to Japan, etc. The language learning survey was created based on the motivation and pedagogical preference subscales used in Schmidt and Watanabe’s (2001) motivation study. Data analysis #1: Selected background information who participated in the listening and reading pretests.:  Data analysis #1: Selected background information who participated in the listening and reading pretests. Data analysis #2: Listening pretest score distributions (N=150) :  Data analysis #2: Listening pretest score distributions (N=150) Data analysis #3: Reading pretest score distributions (N=170) :  Data analysis #3: Reading pretest score distributions (N=170) Data analysis #4 : Language Learning Survey: Motivational orientations and pedagogical preferences (N=270) :  Data analysis #4 : Language Learning Survey: Motivational orientations and pedagogical preferences (N=270) 5=Strongly agree 1=Strongly Disagree How can we use the evaluation data obtained from this project? :  How can we use the evaluation data obtained from this project? 1. Teaching: Time to change? Share the information with teachers within the program so that they will be able to decide if we are satisfied with the overall performance of our students, and if not, identify what specific curriculum changes should be made. 2. Research: Individual differences -Investigate the expected individual differences in the students’ proficiency outcomes at the end of the 2-year and 4-year programs. What factors are contributing to the differences? -Some students have a head start in listening and/or reading ability over their classmates. What is the impact of this situation on learning and teaching? 3. Community: Needs for articulation -Better respond to assessment and curriculum development needs in Japanese at the local, national, and international communities -Articulation between Japanese programs on different UH campuses, -Teacher training for graduate students in our and other language-related programs

Related presentations


Other presentations created by Simo

MCC Presentation Kick Off v3
28. 09. 2007
0 views

MCC Presentation Kick Off v3

Ch 19
01. 10. 2007
0 views

Ch 19

eric tr03
05. 12. 2007
0 views

eric tr03

managing pain
12. 12. 2007
0 views

managing pain

LHC ATLAS Canada McPherson
27. 09. 2007
0 views

LHC ATLAS Canada McPherson

Cosplay
05. 11. 2007
0 views

Cosplay

Gerhard Haag TPA Pres
16. 11. 2007
0 views

Gerhard Haag TPA Pres

finalpresentation
16. 11. 2007
0 views

finalpresentation

D1 10 Rad Olszewski
19. 11. 2007
0 views

D1 10 Rad Olszewski

Fire Safety Version 2
17. 12. 2007
0 views

Fire Safety Version 2

Stress Management Presentation
18. 12. 2007
0 views

Stress Management Presentation

virtualzoo 000
03. 01. 2008
0 views

virtualzoo 000

11 Grasses
04. 10. 2007
0 views

11 Grasses

Paquette MERLOT MAKES HISTORY
28. 12. 2007
0 views

Paquette MERLOT MAKES HISTORY

A THEORY OF HUMAN NEED lecture
07. 01. 2008
0 views

A THEORY OF HUMAN NEED lecture

NEOPOLIOS 2 final
15. 11. 2007
0 views

NEOPOLIOS 2 final

IROS2003
04. 01. 2008
0 views

IROS2003

copyright07
26. 02. 2008
0 views

copyright07

07 Paul Mullen
28. 02. 2008
0 views

07 Paul Mullen

vandergriffs future army
04. 03. 2008
0 views

vandergriffs future army

simile 3
11. 03. 2008
0 views

simile 3

munk economydemographyoil
18. 12. 2007
0 views

munk economydemographyoil

IS1
12. 03. 2008
0 views

IS1

globalspaces
14. 03. 2008
0 views

globalspaces

forum IRTG
18. 03. 2008
0 views

forum IRTG

D1L1
27. 03. 2008
0 views

D1L1

week 05
13. 04. 2008
0 views

week 05

haetc05
13. 12. 2007
0 views

haetc05

Affiliates
29. 10. 2007
0 views

Affiliates

psmp3 anna smrz
21. 11. 2007
0 views

psmp3 anna smrz

New Utility Marking Rule
04. 12. 2007
0 views

New Utility Marking Rule

Create a teaching webpage
05. 11. 2007
0 views

Create a teaching webpage

nomepoesia
21. 11. 2007
0 views

nomepoesia

Hedjazi presentation
19. 12. 2007
0 views

Hedjazi presentation

cre pp rcga rbca seminar
08. 11. 2007
0 views

cre pp rcga rbca seminar

erincook
03. 12. 2007
0 views

erincook

Lecture 110201
07. 11. 2007
0 views

Lecture 110201

ELI slideshow
23. 12. 2007
0 views

ELI slideshow

los meses
05. 11. 2007
0 views

los meses

2007 Laptop Handbook Slides
28. 11. 2007
0 views

2007 Laptop Handbook Slides