Axelrad

Information about Axelrad

Published on January 4, 2008

Author: Denise

Source: authorstream.com

Content

The Cycling of a Decision Threshold: A System Dynamics Model of the Taylor Russell Diagram:  The Cycling of a Decision Threshold: A System Dynamics Model of the Taylor Russell Diagram Elise Axelrad Weaver, Ph.D. and George Richardson, Ph.D. Center for Policy Research April 20, 2001 The Cycling of Decision Thresholds:  The Cycling of Decision Thresholds In his book, Hammond (1996) presents the following ideas: Any decision threshold based on a statistically uncertain measure will inevitably yield some error and injustice in policy outcomes (a duality of error: false positives and false negatives) Oscillations in public and professional attitudes (with implicit policy thresholds) exist: Schlesinger’s (1986) proposal of “regular oscillations” in the dominance of political parties Oscillations between cautious conservatism and risky innovation in bridge design, as “the accumulation of successful experience” makes designers bold until a “collosal failure” takes everyone by surprise Cycles tend to be around 30 years long, across decision domains Hammond suggests that oscillations may be a result of the duality of error Hammond’s exhortation::  Hammond’s exhortation: “For if such oscillations can be shown to exist, and if they can be shown to have a definite period...then we have at hand not only a means for predicting our future political climate far in advance, but an important phenomenon that strongly invites, indeed, demands, analysis and interpretation.” We propose a system dynamics model to represent and explore Hammond’s idea in a rigorous way Another Example of a Reversal in Policy Formation:  Another Example of a Reversal in Policy Formation Use of SAT testing in admissions 1967 University of California began using SAT scores in admissions decisions 1999 University of California faculty voted their preference to exclude SAT scores from admissions decisions 2001 Richard Atkinson, University of California President, publicly advocates removing SATs from admissions. Example of a Decision Threshold for Policy Formation :  Example of a Decision Threshold for Policy Formation Illustration: Students are admitted to an academic program partially according to SAT score set at a given threshold The SAT score is used to predict academic success The GPA at graduation is the measure of true academic success Taylor Russell Diagram r = .5, cutoff = 50:  Taylor Russell Diagram r = .5, cutoff = 50 Positive on test Negative on test Graduates Non-Graduates Decision Threshold False + False - Taylor Russell Diagram r = .5, cutoff = 80:  Taylor Russell Diagram r = .5, cutoff = 80 Positive on test Negative on test Decision Threshold False + False - Graduates Non-Graduates Taylor Russell Diagram r = .5, cutoff = 20:  Taylor Russell Diagram r = .5, cutoff = 20 Positive on test Negative on test Decision Threshold False + False - Graduates Non-Graduates Taylor Russell Diagram Highly Certain Test (r = .99):  Non-Graduates Taylor Russell Diagram Highly Certain Test (r = .99) Positive on test Negative on test Decision Threshold False + False - Graduates Slide10:  They want to reduce the number of individuals for whom the decision to accept was falsely negative: high potential for success but unacceptable SAT scores Stakeholders in the Duality of Error Constituency Concerned with Unfair Disadvantage: Constituency Concerned with Maintaining Standards: They want to reduce the number of individuals for whom the decision to accept was falsely positive: low potential for success but acceptable SAT scores Slide11:  Duality of Error Due to the Decision Threshold Decision Threshold False + False - Slide12:  Decision Threshold as a Stock (accumulating increases and decreases) Slide13:  Decision Threshold Responding to Stakeholder Pressure Disadvantaged Constituency (DC) High Standards Constituency (HSC) Slide14:  Cycling of Policy Threshold: Historic Discontent Disadvantaged Constituency (DC) High Standards Constituency (HSC) Slide15:  Cycling of Policy Threshold: Key Parameters For both constituencies: Tolerated Number of False Cases Relative Weight on History Time to Respond to Pressure Threshold Change per Unit Pressure Disadvantaged Constituency (DC) High Standards Constituency (HSC) Slide16:  Cycling of Policy Threshold: More Key Parameters Time Constants (Forgetting, Retention) Initial value of Threshold Lookup functions (correlation) Dissatisfaction per Error Unexplored Parameters:  Unexplored Parameters Lookup Function translation of threshold choice to number of errors involves correlation, thresholds for judgment and success currently set at r = 0.7, variable threshold for judgment, and threshold for success fixed. Pace of Error Generation and Error Detection Baseline model assumes that in every month, there is an assessment of number of errors due to threshold choice Threshold Associated with “Success” Baseline model assumes a fixed threshold of “success” Slide18:  Graph of Cycling of Policy Threshold SAT Score Baseline Case Slide19:  Threshold Change per Unit Pressure Time to Respond to Pressure - represents policy makers’ responsiveness to constituent pressure: more responsiveness means wider amplitude and very slightly lower frequency Slide20:  Initial Threshold Value - represents policy makers’ initial setting for decision threshold: no long term effect Slide21:  Tolerated Number of False Cases - represents constituents’ sensitivity to errors not a key difference in the long run for either frequency or amplitude Slide22:  Time Constant for Forgetting - represents time it takes for an error to dissipate from constituent memory more time to forget lowers frequency and raises amplitude Slide23:  Relative Weight of History - represents constituents’ memory or weighting for accumulated past errors relative to current error more weight on history means lower frequency and higher amplitude of cycling; no weight on history means no cycling at all. Slide24:  Different Weight on History for Each Constituency - represents differential weight on history by constituents - if one group puts more weight on history, it doesn’t change the center of the cycling, but it does reduce the amplitude. Key Parameter Summary:  Key Parameter Summary Parameters not affecting frequency: initial decision threshold policy maker sensitivity: increases amplitude,but doesn’t affect frequency much in the long run tolerated number of errors by constituents: does not have much effect on frequency or amplitude Parameters affecting frequency relative weight on history & time to forget: longer memories lower frequency and raise amplitude. Next Steps:  Next Steps Ground the conceptual model in data from one or more institutions (especially look for cases where false negatives could be detected) Incorporate the effects of correlations between the judgment and success measures, time to generate and detect errors, and the threshold for success Explore and operationalize the conversion variables (from errors to stakeholder pressure to changes in threshold) Consider whether there is any impact of true negatives and true positives on cycling Explore limit cycle and open loop structure of model Summary:  Summary A systems dynamics model is under development. According to Hammond (1996), any uncertain test where a threshold is used as a policy decision tool leads to unavoidable injustice to some constituency. The pressure on the decision threshold from stakeholders representing the false positives and the false negatives will oppose. These opposing pressures will cause a cycling of the decision threshold over time. A key parameter affecting frequency and amplitude of cycling is constituent weighting of past errors.

Related presentations


Other presentations created by Denise

Salt
17. 12. 2007
0 views

Salt

ayurveda ppt
10. 12. 2007
0 views

ayurveda ppt

the ETD Practitioner2
02. 10. 2007
0 views

the ETD Practitioner2

07 01 trng Day 1 2
03. 10. 2007
0 views

07 01 trng Day 1 2

Bordeaux blurb
01. 10. 2007
0 views

Bordeaux blurb

PPP Proposal B Sp 06
03. 12. 2007
0 views

PPP Proposal B Sp 06

slides nov18 2005
01. 11. 2007
0 views

slides nov18 2005

d 040413 08548
02. 11. 2007
0 views

d 040413 08548

Sonja Bedford
05. 11. 2007
0 views

Sonja Bedford

aggression talk
05. 11. 2007
0 views

aggression talk

Dog Guides Presentation
16. 11. 2007
0 views

Dog Guides Presentation

BT prep for CT pcp
22. 11. 2007
0 views

BT prep for CT pcp

030702 arb wrkshop slides pds r1
08. 11. 2007
0 views

030702 arb wrkshop slides pds r1

BRICS ESOMAR charts FINAL
23. 11. 2007
0 views

BRICS ESOMAR charts FINAL

NS2 2 Feb 06
02. 11. 2007
0 views

NS2 2 Feb 06

Web Semantique2 2007
20. 11. 2007
0 views

Web Semantique2 2007

boykin
21. 12. 2007
0 views

boykin

Building Relationships
24. 12. 2007
0 views

Building Relationships

Engineer 2020 MDN South Dakota
28. 12. 2007
0 views

Engineer 2020 MDN South Dakota

APOLLO TRANSFER 3
31. 12. 2007
0 views

APOLLO TRANSFER 3

Wawrzynek HERC BEE2v1
03. 01. 2008
0 views

Wawrzynek HERC BEE2v1

presentation wales
06. 11. 2007
0 views

presentation wales

class02a
13. 11. 2007
0 views

class02a

2006
05. 11. 2007
0 views

2006

The FRIENDS Program
30. 12. 2007
0 views

The FRIENDS Program

camano p1
18. 12. 2007
0 views

camano p1

cannes2002 v2
19. 02. 2008
0 views

cannes2002 v2

sample Toothpaste assignment
24. 02. 2008
0 views

sample Toothpaste assignment

Structure from motion
27. 02. 2008
0 views

Structure from motion

AFD 070828 004
29. 02. 2008
0 views

AFD 070828 004

DeasonPaul presentation2002
05. 03. 2008
0 views

DeasonPaul presentation2002

1DMotion
11. 03. 2008
0 views

1DMotion

Bird Flu Slides English
30. 03. 2008
0 views

Bird Flu Slides English

gw cpuc 03
01. 01. 2008
0 views

gw cpuc 03

invocation
29. 12. 2007
0 views

invocation

phpulsenov
05. 11. 2007
0 views

phpulsenov

41 cshema presentation 07
07. 01. 2008
0 views

41 cshema presentation 07

PS142 lecture6
23. 12. 2007
0 views

PS142 lecture6

bioch27 2007
09. 10. 2007
0 views

bioch27 2007

kulichkov
27. 09. 2007
0 views

kulichkov

Tempo Para Os Filhos –
29. 05. 2007
0 views

Tempo Para Os Filhos –

pcreap
28. 12. 2007
0 views

pcreap

pregold presentation
26. 11. 2007
0 views

pregold presentation

trickortreat
05. 11. 2007
0 views

trickortreat

xplymr
15. 11. 2007
0 views

xplymr

weinernato4
07. 11. 2007
0 views

weinernato4