NicosiaRaymondPawson

Information about NicosiaRaymondPawson

Published on August 26, 2007

Author: Wanderer

Source: authorstream.com

Content

Realist Evaluation &Realist Synthesis:  Realist Evaluation andamp; Realist Synthesis Ray Pawson QMSS Seminar Nicosia Sept 2006 Presentation I:  Presentation I Basic Ideas of Realist Evaluation Key principles of realist evaluation :  Key principles of realist evaluation The nature of programmes: Programmes are ‘embedded’ Programmes are ‘active’. Programmes are ‘theories’ Principal research tasks: Hypothesise the key mechanisms (M) Hypothesise the key contexts (C) Explain the outcome pattern (O) Programmes are ‘embedded’.They are always inserted into pre-existing social situations:  Programmes are ‘embedded’. They are always inserted into pre-existing social situations Educational initiatives operate at different levels: Ideas Individuals Institution Infrastructure Programmes are theories:  Programmes are theories A ‘role model’ programme theory also known as ‘Dishy-David-Beckham-theory’ ‘Interviewer: But do you think the fact that these good-looking blokes are footballers has any effect on girls' attitude to playing football? Girl: No, I think it has more effect on them watching football, well not the football - the guys (general laughter and agreement)’. Programmes are ‘active’They are ‘active’ in the sense that their intended effects work through the reasoning and volition of their subjects.:  Programmes are ‘active’ They are ‘active’ in the sense that their intended effects work through the reasoning and volition of their subjects. Even ‘mechanical’ interventions like the free distribution of smoke alarms depend on the subject’s ideas. Alarms sometimes work but also end up: not installed (can’t be bothered) removed (always going off) battery flat (too fiddly to change) battery ‘walks’ (to Walkman or TV remote) Realist Evaluation: Mechanisms, Contexts and Outcomes:  Realist Evaluation: Mechanisms, Contexts and Outcomes The same programme mechanism will have different outcomes in different contexts Don’t ask ‘what works?’ Rather, investigate: ‘what works for whom in what circumstances?’ Example: Prisoner Education:  Example: Prisoner Education A Practitioner Theory ' The men who are more likely to be changed are best described as ‘mediocre’. You shouldn’t look for high-flyers. They’re likely to come from a deprived background with a poor and maybe non-existent school record. They will be mediocre criminals too. They’ll have gone on from petty crime, street crime to drugs or armed robbery or something. Then when they come onto the program, they’re mediocre or worse. They just survive the first semester but gradually they build up getting C’s and B’s. So by the end, they’ve actually come a long, long way. And that’s what changes ‘em. It’s not so much a case of ‘rehabilitation’ as ‘habilitation’.' Hypothesise key generative mechanismsThe process of how subjects interpret and act upon the intervention stratagem is known as the programme ‘mechanism’ :  Hypothesise key generative mechanisms The process of how subjects interpret and act upon the intervention stratagem is known as the programme ‘mechanism’ Programme - prisoner education (PE) (M1) PE might provide qualifications to allow ex-inmates to compete for jobs (M2) PE might boost confidence and provide social skills to reduce aggressive outbursts in ex-cons (M3) PE might increase cognitive skills and allow ex-prisoners to reason through their difficulties (M4) PE might increase presentational and reasoning skills enabling them to become clever criminals (M5) PE might provide shelter from violent prison culture for more vulnerable prisoners M6, M7, M8, etc. etc. Hypothesise the key contexts‘Contexts’ are the set of surrounding conditions that favour or hinder the programme mechanisms :  Hypothesise the key contexts ‘Contexts’ are the set of surrounding conditions that favour or hinder the programme mechanisms Prisoner Education - for ‘whom’ and in ‘what circumstances’ (C1) Prior experience - poverty, drugs, violence etc. (C2) Prior education (C3) Prior criminal activities (C4) Prison culture (C5) Neighbourhood and locality culture (C6) Economic structure C7, C8, C9 etc. etc. Explain the outcome pattern Programme outcomes are always complex. M & C theories are tested by examining outcomes for different sub-groups of subjects:  Explain the outcome pattern Programme outcomes are always complex. M andamp; C theories are tested by examining outcomes for different sub-groups of subjects Who benefited most? Improved rehabilitation by age 17-21 22-25 26-30 31-35 36over 1st 4th 5th 2nd 3rd Improved rehabilitation by prior education Grade 10 Grade11 Grade 12 Post-Secondary 2nd 1st 3rd 4th Presentation 2 Basic Ideas of Realist Synthesis :  Presentation 2 Basic Ideas of Realist Synthesis Contrasting views of ‘systematic reviews’ :  Contrasting views of ‘systematic reviews’ Meta-analysis perspective Programmes have effects Evaluation measures effect sizes Systematic review seeks mean effect The realist understanding Programmes are theories Evaluation is theory-testing Systematic review is theory-synthesis Slide14:  Example I: Naming andamp; Shaming Examples of Public Disclosure Initiatives Megan’s Law andamp; sex offender registration Outing prostitute’s johns School league tables Inspection and special measures for ‘failing schools’ Hospital and surgeon report cards Pub-watch bans and exclusions Local press adverts for poll tax non-payment and council rent arrears Beach cleanliness standards and kite marking Slide15:  Realist Synthesis: Naming andamp; Shaming Examples of Public Disclosure Initiatives Megan’s Law andamp; sex offender registration Outing prostitute’s johns School league tables Inspection and special measures for ‘failing schools’ Hospital and surgeon report cards Pub-watch bans and exclusions Local press adverts for poll tax non-payment and council rent arrears Beach cleanliness standards and kite marking Public notification – basic theory:  Public notification – basic theory i) Identification: in which the performance or behaviour in question is observed and then classified, measured, rated, ranked, verified, etc. ii) Naming: in which information on, and the identity of, the failing or deviant party is disclosed, publicized, disseminated, notified, published, broadcast, registered etc. iii) Public sanction: in which the broader community acts on the disclosure in order to shame, reprimand, reproach, censure, control, influence, supervise the named party. iv) Recipient response: in which behavioural change follows the sanction, with the subjects being shamed, regretful, penitent, contrite, restrained, re-integrated etc. Slide17:  NAMED BUT WHO IS SHAMED? Poll-tax protesters named in the local newspapers Sex offenders under community notification Under-performing schools identified in league tables Motor manufacturers named in the Car Theft Index Poll-tax protesters named in the local newspapers:  Poll-tax protesters named in the local newspapers Poll tax non-payment – there were a great many sanctions imposed: fines, wage arrest, court appearances etc. Protesters thus ignored or even celebrated disclosure of their names in the local press. Policy abandoned. Sex offenders under community notification:  Sex offenders under community notification Notified Sex Offenders are resentful or scared by public attention. One key result is non-compliance with registration and displacement to other localities. Re-offence rates remain static. Under-performing schools identified in league tables:  Under-performing schools identified in league tables Schools respond to league table positions tactically - increasing resources to marginal candidates (middle grades) and excluding/ not entering hopeless ones (lowest grades). Grades improve. Motor manufacturers named in the Car Theft Index:  Motor manufacturers named in the Car Theft Index Car manufacturers – are embarrassed by adverse publicity and loss of reputation (and downturn in sales) - respond with genuine improvements in vehicle security. Car crime goes down. Is there a theory to accommodate these differences in outcome?:  Is there a theory to accommodate these differences in outcome? Merton’s Typology of aspirations to group membership of eligibles and non-eligibles The theory under further refinement:  The theory under further refinement For Nandamp;S to ‘work’ the following configuration should be in place: the named party should be an ‘aspirational insider’ the shaming mechanism should be dovetailed with other mechanism (market sanctions) the disclosure should carry intense (but controllable) media interest the disclosed data should unambiguous both in allocating blame and in suggested remedial action the disclosing authority should have had exemplary watchdog credentials, which are operated benignly Example II. Megan’s Law - Programme Theory:  Example II. Megan’s Law - Programme Theory STEP ONE Problem Identification Identify high-risk released sex offenders and create valid and reliable registers STEP TWO Public disclosure Issue bulletins, press releases, call meeting to identify released offenders to their community STEP THREE Sanction Instigation Community joins with police and probation to increase surveillance of suspicious behaviour STEP FOUR Offender response Community actions shame offenders and decrease opportunity of further offence Evidence fragment one: could the law have made a difference? - a ‘retrospective simulation’:  Evidence fragment one: could the law have made a difference? - a ‘retrospective simulation’ Petrosino andamp; Petrosino 136 serious sex offences 36 previous offence 12 stranger predatory offences 6 could potentially respond to community notification 100 no previous offence 24 known to victim 6 offender from out of state Evidence fragment two: did the law effect recidivism? A matched trial:  Evidence fragment two: did the law effect recidivism? A matched trial Pre-intervention sample sex recidivism 22% Post-intervention sample sex recidivism 19% Pre-intervention sample arrest slow Post- intervention sample arrest significantly quicker Schram andamp; Milloy Evidence fragment three: how did practitioners respond? Office talk:  Evidence fragment three: how did practitioners respond? Office talk 'The Law is an unfunded mandate' 'Special Bulletin Notification added more work to already over-worked agents' 'There is more pressure to baby sit with SBN cases simply because they are SBN cases' Zevitz and Farkas Key findings synthesised:  Key findings synthesised Study 1: Following The introduction of the law, detection increase more sharply than deterrence. Study 2: The chances of community surveillance of stranger predatory offences remain low and offender may lie low Study 3: practitioner attention becomes increasingly focused on SBN cases because of community harassment And Finally:  And Finally What it feels like to do realist synthesis … THANK YOU FOR YOUR ATTENTION

Related presentations


Other presentations created by Wanderer

RCM2 Ganesan
17. 08. 2007
0 views

RCM2 Ganesan

System Architect
21. 09. 2007
0 views

System Architect

Customer Retention
28. 09. 2007
0 views

Customer Retention

intrusion detection monitoring
07. 10. 2007
0 views

intrusion detection monitoring

baptista
10. 10. 2007
0 views

baptista

YoungEntrepreneurs China
11. 10. 2007
0 views

YoungEntrepreneurs China

presentation proposed programme
12. 10. 2007
0 views

presentation proposed programme

ch19 lecture
12. 10. 2007
0 views

ch19 lecture

balla reinhart F100 1 pres
18. 10. 2007
0 views

balla reinhart F100 1 pres

Erasmus Charte Universitarie
23. 10. 2007
0 views

Erasmus Charte Universitarie

essayformatTHESIS
26. 08. 2007
0 views

essayformatTHESIS

Sharon
26. 08. 2007
0 views

Sharon

7 Panama Esp
22. 10. 2007
0 views

7 Panama Esp

preference
07. 11. 2007
0 views

preference

Go Forth
17. 08. 2007
0 views

Go Forth

sf wireless
29. 10. 2007
0 views

sf wireless

Convulsoes Neonatais e Epilepsia
28. 12. 2007
0 views

Convulsoes Neonatais e Epilepsia

ch7F07govt2302
31. 12. 2007
0 views

ch7F07govt2302

Presidential character
03. 01. 2008
0 views

Presidential character

potma
03. 01. 2008
0 views

potma

Maitland
09. 10. 2007
0 views

Maitland

overweight obesity
08. 08. 2007
0 views

overweight obesity

Minority Stress Gray APA2006
08. 08. 2007
0 views

Minority Stress Gray APA2006

MoAc0304
08. 08. 2007
0 views

MoAc0304

japanese02s gyro
26. 08. 2007
0 views

japanese02s gyro

Bjorn AFCEATTN CDRNeurath
19. 11. 2007
0 views

Bjorn AFCEATTN CDRNeurath

Rosenzweig Presentation
29. 12. 2007
0 views

Rosenzweig Presentation

martin weller lams
20. 07. 2007
0 views

martin weller lams

26221
26. 08. 2007
0 views

26221

NAATPN Presentation
11. 12. 2007
0 views

NAATPN Presentation

D Levy Transp
21. 09. 2007
0 views

D Levy Transp

646family
24. 02. 2008
0 views

646family

AfricanSlaveTrades
26. 02. 2008
0 views

AfricanSlaveTrades

mms 04 13 elearning
27. 06. 2007
0 views

mms 04 13 elearning

Mathematical Moodle final
27. 06. 2007
0 views

Mathematical Moodle final

inbrieffeb07
28. 02. 2008
0 views

inbrieffeb07

LaWeyl
08. 08. 2007
0 views

LaWeyl

hurricane katrina
13. 03. 2008
0 views

hurricane katrina

meaning
27. 11. 2007
0 views

meaning

transparents Berleur
18. 03. 2008
0 views

transparents Berleur

ChinaandJapanPt2
25. 03. 2008
0 views

ChinaandJapanPt2

BroadbandServies
26. 03. 2008
0 views

BroadbandServies

sinclair prc precip
03. 10. 2007
0 views

sinclair prc precip

8 Mru Patel
07. 04. 2008
0 views

8 Mru Patel

COE 9 Jan 06
28. 03. 2008
0 views

COE 9 Jan 06

060928 Energy Challenges Thun
30. 03. 2008
0 views

060928 Energy Challenges Thun

program
27. 11. 2007
0 views

program

adam smith
09. 04. 2008
0 views

adam smith

crcagu03
10. 04. 2008
0 views

crcagu03

Susan Wachter
13. 04. 2008
0 views

Susan Wachter

MickLilley MacquarieBank
14. 04. 2008
0 views

MickLilley MacquarieBank

experiencia
19. 06. 2007
0 views

experiencia

Europa desde el Cielo 2126
19. 06. 2007
0 views

Europa desde el Cielo 2126

Estrategia empresarial
19. 06. 2007
0 views

Estrategia empresarial

Estadio Allianz Arena 2125
19. 06. 2007
0 views

Estadio Allianz Arena 2125

Esculturas hechas con Arena 2124
19. 06. 2007
0 views

Esculturas hechas con Arena 2124

sesame
26. 11. 2007
0 views

sesame

pierre danon
22. 04. 2008
0 views

pierre danon

Globos Aerostaticos 2133
19. 06. 2007
0 views

Globos Aerostaticos 2133

Gaturro oficina
19. 06. 2007
0 views

Gaturro oficina

Frases para reflexionar 2131
19. 06. 2007
0 views

Frases para reflexionar 2131

Football
19. 06. 2007
0 views

Football

twilight
26. 08. 2007
0 views

twilight

WW
04. 01. 2008
0 views

WW

EvidenceMatters
19. 06. 2007
0 views

EvidenceMatters

p6 alina
26. 08. 2007
0 views

p6 alina

Fantasia 2042
19. 06. 2007
0 views

Fantasia 2042

older adults nut
08. 08. 2007
0 views

older adults nut

mellange presentation en
27. 06. 2007
0 views

mellange presentation en

SH Presentation Sunny Hills
26. 08. 2007
0 views

SH Presentation Sunny Hills

lecture23
08. 08. 2007
0 views

lecture23

IAFC IndiaPresentation
17. 08. 2007
0 views

IAFC IndiaPresentation

00017079
26. 08. 2007
0 views

00017079

04RandomVariables
07. 12. 2007
0 views

04RandomVariables

Lecture 24 Muhammed and Islam
17. 08. 2007
0 views

Lecture 24 Muhammed and Islam

00 norby
26. 08. 2007
0 views

00 norby

CN7 Learning2
14. 12. 2007
0 views

CN7 Learning2

Fumar Mata
19. 06. 2007
0 views

Fumar Mata

do dont show
16. 06. 2007
0 views

do dont show

Direc TV
16. 06. 2007
0 views

Direc TV

BAM CIDOC 2006 folien
16. 06. 2007
0 views

BAM CIDOC 2006 folien

Dr Anwar1
16. 06. 2007
0 views

Dr Anwar1

biouml gcb 2003
16. 11. 2007
0 views

biouml gcb 2003

PPConference 28 02 07 E Marcova
14. 03. 2008
0 views

PPConference 28 02 07 E Marcova

dominguezhills
26. 08. 2007
0 views

dominguezhills

Festival de hielo Harbin
19. 06. 2007
0 views

Festival de hielo Harbin

ePHocus update Rebecca Hills
26. 08. 2007
0 views

ePHocus update Rebecca Hills

Gerenciamiento 2132
19. 06. 2007
0 views

Gerenciamiento 2132

30 mw a si machine
26. 08. 2007
0 views

30 mw a si machine

excursions
19. 06. 2007
0 views

excursions

tl outcomes
12. 10. 2007
0 views

tl outcomes

MIC 03 Dodge WebQuest
27. 06. 2007
0 views

MIC 03 Dodge WebQuest

mms 04 16 dim
27. 06. 2007
0 views

mms 04 16 dim

PRESENTACION PROGRESO
22. 10. 2007
0 views

PRESENTACION PROGRESO

Errores irreparables 1988
19. 06. 2007
0 views

Errores irreparables 1988

EDLafcoPresent092805 chew
26. 08. 2007
0 views

EDLafcoPresent092805 chew

AH summer 07
26. 08. 2007
0 views

AH summer 07

1 Intro class1
26. 08. 2007
0 views

1 Intro class1

Murrieta PKS
08. 08. 2007
0 views

Murrieta PKS