PDSI

Information about PDSI

Published on November 21, 2007

Author: Dixon

Source: authorstream.com

Content

PETASCALE DATA STORAGE INSTITUTE:  PETASCALE DATA STORAGE INSTITUTE The Drive to Petascale Computing Faster computers need more data, faster. -- 2001: 10 TF -- 2005: 100 TF -- 2008: 1 PF -- 2011: 10 PF 2015: 100 PF -- PDSI Thrusts: Data Capture Education & Dissemination Innovation Everything Must Scale with Compute Checkpoint at Terabytes/sec Petabyte files Billions of files Revisit programming for Input/Output Data center automation Acceleration for search Computing Speed Parallel I/O Network Speed Memory Archival Storage TFLOP/s GigaBytes/sec Gigabits/sec 1 2.5 25 250 TeraBytes 2,500 .5 5 50 5 50 500 5,000 5 500 50 .5 ‘00 ‘04 ‘08 2012 Year Disk PetaBytes .05 .5 5 50 Metadata Inserts/sec 200 200 20,000 2,000 500 GigaBytes/sec Application Performance Steeped in Terascale Experience:  Steeped in Terascale Experience Seaborg & GPFS PETASCALE DATA STORAGE INSTITUTE PETASCALE DATA STORAGE INSTITUTE:  PETASCALE DATA STORAGE INSTITUTE Peta-Bytes Tera-B/sec Giga-files Mega-CPUs Tera-Bytes Giga-B/sec Mega-files Kilo-CPUs Education & Dissemination Innovation Data Capture Education Workshops Tutorials Course materials Outreach Storage-research-list Collaboration w/ other Scidacs IT Automation Instrumentation Visualization Machine Learning Diagnosis Adaptation App Workloads INCITE resources Trace & replay tools (e.g. BLAST, CCSM, Calore, EVH1, MCNP, GYRO, Sierra, QCD and other Scidacs) API Standards POSIX API Rich metadata Compute-in-disk Archive API Quality of Storage Scaling Further Global/WAN access Federated security Metadata at scale Para-virtualization NFSv4 extended w/ layouts HPC NFS Parallel NFS Secure NFS IETF Standard Strategic Plan Failure Data Capture & publish Computer Failure Data Repository (e.g. LANL’s outages by root cause) Participating Organizations:  Carnegie Mellon University Garth Gibson (PI) University of California, Santa Cruz Darrell Long (co-PI) University of Michigan, Ann Arbor Peter Honeyman (co-PI) Los Alamos National Laboratory Gary Grider (co-PI) Lawrence Berkeley National Laboratory Bill Kramer (co-PI) Oak Ridge National Laboratory Philip Roth (co-PI) Pacific Northwest National Laboratory Evan Felix (co-PI) Sandia National Laboratory Lee Ward (co-PI) PETASCALE DATA STORAGE INSTITUTE Participating Organizations Programming for Storage:  Programming for Storage The Need for Training Programmers for Storage HPC IT managers work for users who program apps Often performance of apps/workflows dependent on storage Many times best solutions would be to change the program Reality is app specialists intolerant of requests to reprogram for better storage performance That is, reprogramming for storage performance often doesn’t get done Approach: Create tools, training to help a priori Give programmers libraries, performance debugging tools that avoid or detect poor storage patterns Give tutorials, case studies, help pages showing weak programming approaches and how to improve them PETASCALE DATA STORAGE INSTITUTE Example from BioInformatics:  Example from BioInformatics Pseudo code example from IT manager -- single thread for( I=0, I<1000, I++){ for( J=0, J<1000, J++){ buf = compute (I,J); f = open( “file_foo”); lseek(f, offset(I,J)); write(f, buf, lengthof(buff)) close(f); }} Buf turns out to be small, unaligned, fixed length Obvious fixes: Open/close outside both loops Malloc sizeof 1000000*lengthof(buff), copy into it in memory, one write at end PETASCALE DATA STORAGE INSTITUTE

Related presentations


Other presentations created by Dixon

Types of Flower Shop
06. 11. 2007
0 views

Types of Flower Shop

ALCATELe salud
30. 11. 2007
0 views

ALCATELe salud

Upanishads
06. 12. 2007
0 views

Upanishads

Teaching World History
25. 10. 2007
0 views

Teaching World History

400 Silent Years
30. 10. 2007
0 views

400 Silent Years

invasion2
31. 10. 2007
0 views

invasion2

2004 06 09 clavell constipation
31. 10. 2007
0 views

2004 06 09 clavell constipation

PresentazioneSofia20 05
01. 11. 2007
0 views

PresentazioneSofia20 05

Ch09
02. 11. 2007
0 views

Ch09

EEA Workshop Buhaug IMO index
06. 11. 2007
0 views

EEA Workshop Buhaug IMO index

reynolds
07. 11. 2007
0 views

reynolds

Week5
15. 11. 2007
0 views

Week5

The best of two worlds
16. 11. 2007
0 views

The best of two worlds

iso e
23. 11. 2007
0 views

iso e

pollination
17. 12. 2007
0 views

pollination

savannas
26. 11. 2007
0 views

savannas

discourse
12. 12. 2007
0 views

discourse

S4 03Dwaine Clarke
25. 12. 2007
0 views

S4 03Dwaine Clarke

Field Forage
28. 12. 2007
0 views

Field Forage

Ethics Principles May 2003 1
29. 12. 2007
0 views

Ethics Principles May 2003 1

Alan Turing is Da Bombe
02. 01. 2008
0 views

Alan Turing is Da Bombe

Chalut1
03. 01. 2008
0 views

Chalut1

Search and Rescue
03. 01. 2008
0 views

Search and Rescue

StigmaLeipzigAtt
04. 01. 2008
0 views

StigmaLeipzigAtt

saworkshop pp addressing uebel
07. 01. 2008
0 views

saworkshop pp addressing uebel

file 10684
07. 01. 2008
0 views

file 10684

Laborin Mario
15. 11. 2007
0 views

Laborin Mario

una madre unica 21186
01. 10. 2007
0 views

una madre unica 21186

BerwickPPT1sp04
10. 12. 2007
0 views

BerwickPPT1sp04

FDIprezentace 2
14. 11. 2007
0 views

FDIprezentace 2

bisc Progress Review 17 june
03. 12. 2007
0 views

bisc Progress Review 17 june

Lecture12Handout
30. 12. 2007
0 views

Lecture12Handout

Beauty05 biglietti
30. 10. 2007
0 views

Beauty05 biglietti

ch14
20. 02. 2008
0 views

ch14

A4081
24. 02. 2008
0 views

A4081

ELECTRONICversion
27. 02. 2008
0 views

ELECTRONICversion

italie powerpoint 04 05
31. 10. 2007
0 views

italie powerpoint 04 05

lecture 11 travel writing
27. 03. 2008
0 views

lecture 11 travel writing

BP ICIW07
31. 10. 2007
0 views

BP ICIW07

GOLINI
29. 10. 2007
0 views

GOLINI

WAYS OF DIVIDING THE WORLD
24. 12. 2007
0 views

WAYS OF DIVIDING THE WORLD

twp
23. 12. 2007
0 views

twp

barrett
02. 01. 2008
0 views

barrett

SLAC 02022005 AMvdB
05. 12. 2007
0 views

SLAC 02022005 AMvdB

Navas 30
23. 11. 2007
0 views

Navas 30

InSeT
16. 11. 2007
0 views

InSeT

Intermediate Microsoft Word
12. 03. 2008
0 views

Intermediate Microsoft Word

shin
11. 12. 2007
0 views

shin

SESAMI Menichelli
29. 10. 2007
0 views

SESAMI Menichelli

Wireless Workshop Tyndall
28. 11. 2007
0 views

Wireless Workshop Tyndall