OWASP AppSec2006 Seattle Security Metrics

Information about OWASP AppSec2006 Seattle Security Metrics

Published on August 30, 2007

Author: Barbara

Source: authorstream.com

Content

The OWASP Application Security Metrics Project:  The OWASP Application Security Metrics Project Bob Austin Application Security Metrics Project Lead KoreLogic, Inc. [email protected] 804.379.4656 Presentation Objectives:  Presentation Objectives Drivers for Security Metrics Review the Project Plan. Work Accomplished To Date, Next Steps Provide Application Security Metrics Resources Solicit Feedback and Participation Slide3:  The Best Metrics….Can Answer Hard Questions How secure am I? Am I better than this time last year? Am I spending the right amount of money? How do I compare to my industry peers (senior management’s favorite question)? Source: Dr. Dan Geer Slide4:  Forrester Survey: 'What are your top three drivers for measuring information security?' Source: 'Measuring Information Security Through Metrics And Reporting', Forrester Research, Inc., May 2006' Report progress to business Better stewardship Base: 40 CISOs and senior security managers Slide5:  Forrester Survey: What do CISOs want to measure? 'As a CISO, if you have a choice of measuring and monitoring up to five areas in security, which ones would you measure?' Base: 34 CISOs and senior security managers. Source: 'Measuring Information Security Through Metrics And Reporting', Forrester Research, Inc., May 2006' Slide6:  Project Goal and Roadmap     Phase One Phase Two Project Goal: Address the current lack of effective application security metrics by identifying, sharing and evolving useful metrics and metric processes to benefit the OWASP community. Current Project Contributors: Jeff Williams (Aspect Security), Cliff Barlow (KoreLogic), Matt Burton (Mitre) http://www.owasp.org/index.php/Category:OWASP_Metrics_Project  Current Project Status  Slide7:  Phase One – Application Security Metrics Baseline Survey Plan Information Capture Analysis Survey Results    http://www.owasp.org/index.php/Metrics_Survey_Form Slide8:  Useful Resources from Research OWASP CLASP Project – 'Monitor Security Metrics' Dr. Dan Geer’s 'Measuring Security' Tutorial Other Initiatives: Securitymetrics.org, Metricon 1.0 Secure Software Development Life Cycle: 'The Security Development Lifecycle', Howard and Lipner, 'Security in the Software Lifecycle', DHS, Cybersecurity Div. Information Security Metrics Standard - ISO 27004 Dr. Larry Gordon, Cybersecurity Economics Research Projects Resources from NIST: Security Metrics Guide for Information Technology Systems, Guide for Developing Performance Metrics for Information Security NIST Software Assurance Metrics and Tool Evaluation (SAMATE) Organizing Metric Types:  Organizing Metric Types Process Metrics Information about the processes themselves. Evidence of maturity. Vulnerability Metrics Metrics about application vulnerabilities themselves Management Metrics specifically designed for senior management Examples Secure coding standards in use Avg. time to correct critical vulnerabilities Examples By vulnerability type By occurrence within a software development life cycle phase Examples % of applications that are currently security 'certified' and accepted by business partners Trending: critical unresolved, accepted risks Slide10:  Opportunities for Metrics - Secure Development Life Cycle (SDL) Secure questions during interviews Concept Designs Complete Test plans Complete Code Complete Deploy Post Deployment Threat analysis Security Review Team member training Data mutation andamp; Least Priv Tests Review old defects Check-ins checked Secure coding guidelines Use tools Learn andamp; Refine External review Source: Microsoft Were software assurance activities conducted at each lifecycle phase? Slide11:  Examples of Application Security Metrics Process Metrics Is a SDL Process used? Are security gates enforced? Secure application development standards and testing criteria? Security status of a new application at delivery (e.g., % compliance with organizational security standards and application system requirements). Existence of developer support website (FAQ's, Code Fixes, lessons learned, etc.)? % of developers trained, using organizational security best practice technology, architecture and processes Management Metrics % of applications rated 'business-critical' that have been tested. % of applications which business partners, clients, regulators require be 'certified'. Average time to correct vulnerabilities (trending). % of flaws by lifecycle phase. % of applications using centralized security services. Business impact of critical security incidents. Examples of Application Security Metrics:  Examples of Application Security Metrics Vulnerability Metrics Number and criticality of vulnerabilities found. Most commonly found vulnerabilities. Reported defect rates based on security testing (per developer/team, per application) Root cause of 'Vulnerability Recidivism'. % of code that is re-used from other products/projects* % of code that is third party (e.g., libraries)* Results of source code analysis**: Vulnerability severity by project, by organization Vulnerabilities by category by project, by organization Vulnerability +/- over time by project % of flaws by lifecycle phase (based on when testing occurs) Source: * WebMethods, ** Fortify Software The Path Forward:  The Path Forward Complete KoreLogic-sponsored surveys Encourage others to complete survey forms Create metrics taxonomy. Test drive it. Collaborate/share with other metrics initiatives 'Will Work for Metrics'. Volunteers needed! Solicit survey participants. Collect survey data. Help analyze survey data Donate useful application security metrics Help plan Phase Two Slide14:  Our Security Metrics Challenge 'A major difference between a 'well developed' science such as physics and some of the less 'well-developed' sciences such as psychology or sociology is the degree to which things are measured.' Source: Fred S. Roberts, ROBE79 'Give information risk management the quantitative rigor of financial information management.' Source: CRA/NSF, 10 Year Agenda for Information Security Research, cited by Dr. Dan Geer Supplemental Slides and Metrics Resources:  Supplemental Slides and Metrics Resources Resources – Security Metrics:  Resources – Security Metrics Security Metrics Standards: ISO 27004 - a new ISO standard on Information Security Management Measurements. Other metrics initiatives - Securitymetrics.org Metricon 1.0 presentations, http://www.securitymetrics.org/content/Wiki.jsp?page=Metricon1.0 Dan Geer’s measuringsecurity tutorial. Pdf, http://geer.tinho.net/usenix Developing metrics programs: Security Metrics Guide for Information Technology Systems, http://csrc.nist.gov/publications/nistpubs/800-55/sp800-55.pdf Guide for Developing Performance Metrics for Information Security, http://csrc.nist.gov/publications/drafts/draft-sp800-80-ipd.pdf Establishing an Enterprise Application Security Program, Tony Canike, OWASP 2005 Metrics-related Tools: NIST Software Assurance Metrics and Tool Evaluation (SAMATE), http://samate.nist.gov/index.php/Main_Page Metrics-related Models, Frameworks: http://www.sse-cmm.org/model/model.asp Current Articles on Metrics www.csoonline.com/metrics/index.htm Metric-related Financial and Econometric Resources: Economics and Security Resource Page, Ross Anderson), http://www.cl.cam.ac.uk/~rja14/econsec.html Dr. Larry Gordon, University of Maryland, Cybersecurity Economics Research Projects, http://www.rhsmith.umd.edu/faculty/lgordon/Cybersecurity%20Economics%20Research%20Projects.html Resources – Software Assurance:  Resources – Software Assurance 'A Clinic to Teach Good Programming Practices', Matt Bishop, http://nob.cs.ucdavis.edu/bishop/talks/2006-cisse-2/clinic.html Team Software Process for Secure Systems (TSP-Secure), http://www.sei.cmu.edu/tsp/tsp-security.html OMG’s Software Assurance Workshop 2007, http://www.omg.org/news/meetings/SWA2007/call.htm DHS Cybersecurity Division Software Assurance Initiatives: Software Assurance Measurement Workshop, Oct, 2006 Software Assurance Program, http://www.psmsc.com/UG2006/Presentations/11_DHS_SwA_Overview_for_PSM.pdf Software Assurance Forum, https://buildsecurityin.us-cert.gov/daisy/bsi/events/521.html CERT Secure Coding Standards, https://www.securecoding.cert.org/confluence/display/seccode/CERT+Secure+Coding+Standards CRA Conference on 'Grand Research Challenges in Information Security andamp; Assurance', http://www.cra.org/reports/trustworthy.computing.pdf Resources – General Software Measures & Metrics:  Resources – General Software Measures andamp; Metrics Measures and Metrics Web Sites, http://www.stsc.hill.af.mil/crosstalk/1999/06/measuresites.asp Software Process Metrics Organizations: http://www.totalmetrics.com/cms/servlet/main2?Subject=Listandamp;ID=3 http://www.swmetrics.org/ Software Metrics Symposium Capability Maturity Model Integration (CMMI) Tenth ANNUAL PSM USERS' GROUP CONFERENCE Performance and Decision Analysis, http://www.psmsc.com/UsersGroup2006.asp History of Software Measurement (Horst Zuse), http://irb.cs.tu-berlin.de/~zuse/metrics/History_00.html NASA Software Engineering Laboratory, Experience Factory: http://sel.gsfc.nasa.gov/website/exp-factory.htm ISO/IEC 15939, Software Engineering - Software Measurement Process Software Metrics Glossary, http://www.totalmetrics.com/cms/servlet/main2?Subject=Listandamp;ID=12 2006 State of Software Measurement Practice Survey, http://www.sei.cmu.edu/sema/presentations/stateof-survey.pdf Slide19:  Really Bad Metrics Advice According to my data, roughly 122.45 percent of this journal's 347,583,712 readers need some sharpening up on how to effectively collect and use metrics. There is less than a 0.0345 percent chance that this column will help. Q: I'm a manager who believes in keeping metrics simple, which is why I've limited the number we collect to 62. But I also want to simplify their collection—do you know where I can find timecard readers designed for bathroom stalls? A: Try voice print-activated stalls with timed door locks. But first, are you really trying to collect 62 metrics? 62? [snicker snort chortle] You're obviously clueless about the 'KISS' principle: Keep It Stupefyingly Strenuous. You can collect a lot more than 62 different metrics. The accepted rule of thumb for the number of metrics you can reasonably work with is this: 'Seven, plus or minus the square of the number of door knobs in your home.' Remember, if something can be measured, it must be measured, and all metrics are equally critical. Q: I feel vindicated. Now I can introduce additional metrics for every obscure area of our process improvement model. Naturally, I plan to drop the whole wad as an enforced edict and then make myself unavailable for a few weeks. A: Bravo! But be sure you don't overcomplicate things by defining every minute detail, such as data integrity standards or what you plan to do with the data. People learn nothing from constant handholding. Your job is to sit back and wait for those reliable numbers to start pouring in. Q: Great! What do you suggest I do with all that data? A: What should you do with the data? Do? That question implies that metrics are a means to some end. Don't waste resources—time spent analyzing metrics is time that could have been spent collecting even more metrics. Q: My boss keeps asking for data on stuff I don't think can be quantified—and it's often common sense stuff he could just ask us! Aren't metrics just a big sham? A: Shhh! You're right, metrics are actually an extensive conspiracy—but an extremely helpful one. When people want to make decisions based on 'facts' rather than 'opinions,' you need metrics to push your personal agenda under the guise of unassailable objectivity. Perception is everything: Politicized emotional drivel: 'Let's try my approach. Her plan isn't working.' Objective insight: 'A consumptive analysis of my plan projects a 84.67 percent increased density of pro-active rationals within six months. However, her key preambulatory vindicators are creating a 24.38 percent downward sloping polymorphic trend. Plus, she wears really cheesy business suits.' Source: http://www.stsc.hill.af.mil/crosstalk/1998/08/backtalk.asp

Related presentations


Other presentations created by Barbara

Solar System
17. 06. 2007
0 views

Solar System

Advanced SQL Injection
30. 08. 2007
0 views

Advanced SQL Injection

PrivateExchange
22. 04. 2008
0 views

PrivateExchange

07 fordjob1
17. 04. 2008
0 views

07 fordjob1

20061011114434853
13. 04. 2008
0 views

20061011114434853

Bruce Lambert Army Corps
10. 04. 2008
0 views

Bruce Lambert Army Corps

SPAC2007 Juan Rodriguez
09. 04. 2008
0 views

SPAC2007 Juan Rodriguez

Chapter7
07. 04. 2008
0 views

Chapter7

tourism chapter 04
30. 03. 2008
0 views

tourism chapter 04

LAC International Trade
28. 03. 2008
0 views

LAC International Trade

feb2006final
27. 03. 2008
0 views

feb2006final

virtualcommunities
26. 03. 2008
0 views

virtualcommunities

Mickey Mouse
26. 06. 2007
0 views

Mickey Mouse

1who gets tb in nyc
27. 09. 2007
0 views

1who gets tb in nyc

lijian
12. 10. 2007
0 views

lijian

O2 Diesel
08. 11. 2007
0 views

O2 Diesel

American Romanticism
30. 08. 2007
0 views

American Romanticism

233nm60
30. 08. 2007
0 views

233nm60

MBA Lecture Series v2
30. 08. 2007
0 views

MBA Lecture Series v2

hep2005 talk MarkVagins
09. 10. 2007
0 views

hep2005 talk MarkVagins

Control Tech
05. 12. 2007
0 views

Control Tech

DasuCMSTriggerUCSD
07. 10. 2007
0 views

DasuCMSTriggerUCSD

ams ppt
30. 08. 2007
0 views

ams ppt

Question Answering
16. 11. 2007
0 views

Question Answering

Facts x about Finland
22. 11. 2007
0 views

Facts x about Finland

OWAS PAppSecEU2006 CLASP Project
30. 08. 2007
0 views

OWAS PAppSecEU2006 CLASP Project

OWASP Flyer Sep06
30. 08. 2007
0 views

OWASP Flyer Sep06

fun with hyperplanes 2007
28. 12. 2007
0 views

fun with hyperplanes 2007

american history
28. 12. 2007
0 views

american history

Frank Garber Presentation
02. 01. 2008
0 views

Frank Garber Presentation

DPS07 65 01 Fritzius
03. 01. 2008
0 views

DPS07 65 01 Fritzius

Teaching Political Sociology
04. 01. 2008
0 views

Teaching Political Sociology

Gaming in Education
07. 01. 2008
0 views

Gaming in Education

Plume tracking hardware
07. 01. 2008
0 views

Plume tracking hardware

Altera
28. 11. 2007
0 views

Altera

dead reckon cdr
07. 01. 2008
0 views

dead reckon cdr

Infections 3
04. 12. 2007
0 views

Infections 3

CMC IR1001
27. 09. 2007
0 views

CMC IR1001

class2 3
16. 11. 2007
0 views

class2 3

mixload
06. 11. 2007
0 views

mixload

web query 0609
07. 11. 2007
0 views

web query 0609

FSA
27. 12. 2007
0 views

FSA

CompanyDossier
29. 09. 2007
0 views

CompanyDossier

Hunting For Black Holes
28. 11. 2007
0 views

Hunting For Black Holes

DAR
20. 02. 2008
0 views

DAR

8 Soci 1015 Chapter7 Family
24. 02. 2008
0 views

8 Soci 1015 Chapter7 Family

ABSSEI Oswald
29. 02. 2008
0 views

ABSSEI Oswald

NeMO Curr Part3 v2
26. 06. 2007
0 views

NeMO Curr Part3 v2

nelson sheinberg Presentation
26. 06. 2007
0 views

nelson sheinberg Presentation

n0002 SPIE1
26. 06. 2007
0 views

n0002 SPIE1

Metric System 1
26. 06. 2007
0 views

Metric System 1

media kit
26. 06. 2007
0 views

media kit

March 14 PMI Presentation
26. 06. 2007
0 views

March 14 PMI Presentation

fountain of age
26. 06. 2007
0 views

fountain of age

Lifting Equation
13. 12. 2007
0 views

Lifting Equation

Dietary Guidelines
04. 03. 2008
0 views

Dietary Guidelines

upshur pc1
10. 03. 2008
0 views

upshur pc1

crossref
30. 08. 2007
0 views

crossref

ddbppt
20. 11. 2007
0 views

ddbppt

DEPBasicsCourse
30. 12. 2007
0 views

DEPBasicsCourse

guerra
12. 11. 2007
0 views

guerra

James F Cooper
30. 08. 2007
0 views

James F Cooper

lubin talk
03. 01. 2008
0 views

lubin talk

NDD presentation compressed
30. 08. 2007
0 views

NDD presentation compressed

madcooper
07. 12. 2007
0 views

madcooper

graduacion1
01. 01. 2008
0 views

graduacion1

GBIF demo Japan081003
27. 11. 2007
0 views

GBIF demo Japan081003

20061019 1732 oberauer hql06
15. 11. 2007
0 views

20061019 1732 oberauer hql06

phpulse oct
05. 01. 2008
0 views

phpulse oct

media searching
26. 06. 2007
0 views

media searching

Smith Core values
17. 06. 2007
0 views

Smith Core values

Smith1
17. 06. 2007
0 views

Smith1

Significance of the Cross
17. 06. 2007
0 views

Significance of the Cross

Sharp
17. 06. 2007
0 views

Sharp

section 2 attitude to food
17. 06. 2007
0 views

section 2 attitude to food

Section4 5
17. 06. 2007
0 views

Section4 5

Spirituality
17. 06. 2007
0 views

Spirituality

sonnet presentation
17. 06. 2007
0 views

sonnet presentation

Star addition tutorial
17. 06. 2007
0 views

Star addition tutorial

stand up comedy
17. 06. 2007
0 views

stand up comedy

SS 1SBrown
17. 06. 2007
0 views

SS 1SBrown

Emerson Transcendentalism
30. 08. 2007
0 views

Emerson Transcendentalism

ABinEurope
23. 11. 2007
0 views

ABinEurope

TextMining 06
03. 10. 2007
0 views

TextMining 06

oct04ach
05. 11. 2007
0 views

oct04ach

SCP2
17. 06. 2007
0 views

SCP2

transcendentalism
30. 08. 2007
0 views

transcendentalism

micro ch03 presentation
04. 10. 2007
0 views

micro ch03 presentation

SC morning
17. 06. 2007
0 views

SC morning

ISIC cobrandNEUenglish
18. 03. 2008
0 views

ISIC cobrandNEUenglish

02b LisbonWeb
30. 12. 2007
0 views

02b LisbonWeb

ProvenceArchitecture
05. 11. 2007
0 views

ProvenceArchitecture

san diego 04
01. 11. 2007
0 views

san diego 04

noemie 2
26. 06. 2007
0 views

noemie 2

Community Service PP 06 FOR WEB
05. 11. 2007
0 views

Community Service PP 06 FOR WEB

Sections3 7
17. 06. 2007
0 views

Sections3 7

ECE TRANS WP29 GRSP 41 inf09e
26. 11. 2007
0 views

ECE TRANS WP29 GRSP 41 inf09e

srwg graz
26. 11. 2007
0 views

srwg graz

Meydan
23. 11. 2007
0 views

Meydan

LWS05
02. 11. 2007
0 views

LWS05

mal 2005 bra
30. 08. 2007
0 views

mal 2005 bra

Standards Aligned Classroom
17. 06. 2007
0 views

Standards Aligned Classroom

steenkampNVDRS
06. 03. 2008
0 views

steenkampNVDRS