VALS Keahey

Information about VALS Keahey

Published on August 13, 2007

Author: Malbern

Source: authorstream.com

Content

Virtual Appliances for Scientific Applications:  Virtual Appliances for Scientific Applications Kate Keahey [email protected] Argonne National Laboratory University of Chicago The Grid Metaphor:  The Grid Metaphor How do we store energy? How do we charge for energy? How do we reliably deliver energy? What happens if a power station fails? How do we ensure quality of service? What elements make for a safe and efficient power Grid? How do we make sure that supply meets demand? Computational Grids:  Computational Grids What is the 'unit' of resource usage? How can we manage different computing environments? How can we ensure that disk, CPUs, network are all available? How can we negotiate for computation? How can we use Grid resources as easily and intuitively as we use electrical power today? Provisioning Critical Resources:  Provisioning Critical Resources Quality of Service:  Quality of Service Issues of control Trust management Dynamic relationships Protocols to negotiate SLA-based relationships Enforcement tools What worked Coarse-grained sharing for relatively tight-knit communities with strong incentives to collaborate Non-critical needs Informal relationships What proved difficult: Formal sharing for loosely knit communities Quality of Life:  Quality of Life Lots of heterogeneous resources, none of them good for my application Consistent environment Short-term leasing Changing configuration quickly, quick turnaround Some examples: Support for legacy physics applications Unusual platforms needed by ornitologists Climate scientists need very consistent configurations What worked Access to resources with standardized configuration Tightly-knit communities Everything else proved difficult Workspaces for Grid Computing:  Workspaces for Grid Computing Virtual Workspace Environment definition Resource allocation The GT4 Virtual Workspace Service (VWS) allows an authorized client to deploy and manage workspaces on-demand. GT4 WSRF-based protocol set, leverages multiple GT services Multiple back-ends possible, currently using Xen http://workspace.globus.org Paper: Virtual Workspaces: Achieving Quality of Service and Quality of Life in the Grid, Scientific Programming Journal Workspace Service:  Workspace Service Pool node Trusted Computing Base (TCB) Image Nodeand#x3; Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node The workspace service has a WSRF frontend that allows users to deploy and manage virtual workspaces The VWS manages a set of nodes inside the TCB (typically a cluster). This is called the node pool. Each node must have a VMM (Xen) installed, along with the workspace backend (software that manages individual nodes) VM images are staged to a designated image node inside the TCB VWS Node VWS Service Deploying Workspaces:  Image Nodeand#x3; Deploying Workspaces Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Workspace Deployment Request Workspace metadata Describes the workspace Contextualization information (IP, security,partitions,etc.) Resource Allocation Specifies availability, CPU%, disk, memory, nodes, etc. VWS Service Interacting with Workspaces:  Image Nodeand#x3; Interacting with Workspaces Pool node Trusted Computing Base (TCB) Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node The workspace service publishes information on each workspace as standard WSRF Resource Properties. Users can query those properties to find out information about their workspace (e.g. what IP the workspace was bound to) as well as manage the resources a workspace was assigned Users can interact directly with their workspaces the same way the would with a physical machine. VWS Service The Case of OSG Edge Services:  The Case of OSG Edge Services OSG Edge Services:  OSG Edge Services Requirements: Edge Services are VO-specific Resource usage negotiation and enforcement Features: IP addresses Management Host certificates for Edge Services, naming issues Resource allocation (re)negotiation Integration into the local infrastructure Challenges: Image configuration and maintenance Fine-grain resource usage enforcement Running out of public IPs… Paper: Division of Labor: Tools for Growth and Scalability of Grids, ICSOC 2006 The Case of the OSG Virtual Cluster:  The Case of the OSG Virtual Cluster Image Nodeand#x3; Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node Pool node VWS Service OSG Virtual Cluster:  OSG Virtual Cluster Requirements: Leasing/Glide-ins: resource allocation for VO-specific computation Short execution time, workflows Scientific gateways Features: Describing and managing aggregate workspaces Application-specific configuration on the fly Challenges: Integration with local scheduling infrastructure Paper: Virtual Clusters for Grid Communities, CCGrid 2006 (TR2005) The Case of the STAR Application:  no STAR The Case of the STAR Application no STAR no STAR STAR STAR STAR VWS GRAM STAR GRAM STAR Application:  STAR Application Requirements: Hard-to-install legacy applications Consistent environment requirements Features: Image size (6-10 GB), 8 min deployment time Image Caching Challenges: Integration with local scheduling infrastructure Presentation: Virtual Workspace Appliances, SC06 The Case of the Alice Application:  The Case of the Alice Application Requirements: Pull-based computing model Features: Partition management Blank partitions Partition sharing between workspaces Capability maching Workspace descriptions Factory pre-reqisites Ongoing effort Moving Forward:  Moving Forward Deployment: a chicken and egg problem The Chicken: overcoming Xenophobia Hypervisor installations are invasive Security: the cure or the disease? Infrastructure: scheduling, etc. Incentives The Egg: users Where do I get an image from? VO administrators How do we describe, identify, query for images? Integrated vision of knitting multiple resources together Overall Approach:  Overall Approach Appliance Producer Appliance Deployment Appliance Management Deployment (1):  Deployment (1) Matching Appliances to Resources Appliance meta-data VM image? What VMM, architecture, etc. Resource characteristics What kind of appliances am I willing to deploy? Workspace Service Workspace meta-data VWS Factory pre-conditions Deployment (2):  Deployment (2) Establishing trust in an appliance Assert appliance properties, sign them to the image Direct or indirect assertion Trust the process, not just the person Probe appliances Presentation: Making your workspace secure: establishing trust with VMs in the Grid, SC05 Deployment (3):  Deployment (3) Adapting appliances for deployment IP address delivery Generating certificates Making an appliance work within a specific deployment framework (contextualization) Virtual clusters Application-level configuration Producing Appliances:  Producing Appliances Configuration for the masses The profile of an appliance configurer has changed Building appliances incrementally Appliance attestation Functionality testing Trust the process, not just the person Managing Appliances:  Managing Appliances Security updates Security RSS Feed Bugtraq, US-CERT Security Advisories Will the system still work? Functionality testing Component dependencies Appliance Layers:  Appliance Layers Layered Appliance A set of interdependent layers Appliance layers Less data needs to travel More flexible Faster deployment Trust management Collaborative aspects of configuration System Layer Customization Layer Application Layer VO Layer Virtual Organizations:  Virtual Organizations myVO.org grid-proxy-init Sharing resources: images, hardware, networks, storage facilities, security context Conclusions:  Conclusions We need languages and protocols to describe, discover and name appliances Growing role of a VO Configuration management Virtual networks and namespaces Beyond a security context Sustainable deployment model How does producing, deploying and managing appliances work together? Credits:  Credits Workspace team Tim Freeman, Borja Sotomayor Guest appearances Rick Bradshaw, Predrag Buncic, Narayan Desai, Abhishek Rana, Frank Siebenlist, Doug Olson, Frank Wuerthwein and others.

Related presentations


Other presentations created by Malbern

Ford
04. 08. 2007
0 views

Ford

aganek ams03
18. 09. 2007
0 views

aganek ams03

carl2004
18. 09. 2007
0 views

carl2004

Innovation CEO
18. 09. 2007
0 views

Innovation CEO

Websight TAV WEB July 2004
18. 09. 2007
0 views

Websight TAV WEB July 2004

BlueRidge
18. 09. 2007
0 views

BlueRidge

sis statewide training
18. 09. 2007
0 views

sis statewide training

060307DFID
18. 09. 2007
0 views

060307DFID

bluegene01
18. 09. 2007
0 views

bluegene01

Blue Jean expt 2005
18. 09. 2007
0 views

Blue Jean expt 2005

APA 2005 Strategies for Genetics
18. 09. 2007
0 views

APA 2005 Strategies for Genetics

ross pearce
18. 09. 2007
0 views

ross pearce

present 05
18. 09. 2007
0 views

present 05

mcgee
18. 09. 2007
0 views

mcgee

SIS Asmnt Score Pres 20070522
18. 09. 2007
0 views

SIS Asmnt Score Pres 20070522

mergers
26. 08. 2007
0 views

mergers

WalkTwoMoons
26. 08. 2007
0 views

WalkTwoMoons

a2 pale view narrative technique
26. 08. 2007
0 views

a2 pale view narrative technique

IDAR26
26. 08. 2007
0 views

IDAR26

SundaySept30
26. 08. 2007
0 views

SundaySept30

would you recognize
13. 08. 2007
0 views

would you recognize

Youth
13. 08. 2007
0 views

Youth

WEAD0201
13. 08. 2007
0 views

WEAD0201

wp 10 e
13. 08. 2007
0 views

wp 10 e

Workshop Presentation june 2002
13. 08. 2007
0 views

Workshop Presentation june 2002

DisplacedEllerman
04. 08. 2007
0 views

DisplacedEllerman

Framing Effect and Age
04. 08. 2007
0 views

Framing Effect and Age

Freud2004
04. 08. 2007
0 views

Freud2004

feist ch01Intro
04. 08. 2007
0 views

feist ch01Intro

empowering
04. 08. 2007
0 views

empowering

weston
13. 08. 2007
0 views

weston

Coyote Hills Historic B 000
26. 08. 2007
0 views

Coyote Hills Historic B 000

Olshan cegawrkshp 2
18. 09. 2007
0 views

Olshan cegawrkshp 2

EmbryologyLect5
04. 08. 2007
0 views

EmbryologyLect5

FBrancaLuxembourg3mar
04. 08. 2007
0 views

FBrancaLuxembourg3mar

E Newsletter June2006
18. 06. 2007
0 views

E Newsletter June2006

Urdy
18. 06. 2007
0 views

Urdy

tim feeney
18. 06. 2007
0 views

tim feeney

Sports vision 2004
18. 06. 2007
0 views

Sports vision 2004

PAD Cost Effectiveness
18. 06. 2007
0 views

PAD Cost Effectiveness

Geriatric Exercise Handout
18. 06. 2007
0 views

Geriatric Exercise Handout

Expositions Solidaires
18. 06. 2007
0 views

Expositions Solidaires

erasmus aout05en
18. 06. 2007
0 views

erasmus aout05en

ENGLISH IS FUN
18. 06. 2007
0 views

ENGLISH IS FUN

English Culture Sports2 ready
18. 06. 2007
0 views

English Culture Sports2 ready

6746
18. 06. 2007
0 views

6746

20060410 uncch sils
18. 06. 2007
0 views

20060410 uncch sils

20060408 seaall
18. 06. 2007
0 views

20060408 seaall

20050823 Nagoya
18. 06. 2007
0 views

20050823 Nagoya

060217 CLEMENT B talk
18. 06. 2007
0 views

060217 CLEMENT B talk

DREAM ME
18. 06. 2007
0 views

DREAM ME

evaporaion
15. 06. 2007
0 views

evaporaion

Division with 0
15. 06. 2007
0 views

Division with 0

Division Trouble
15. 06. 2007
0 views

Division Trouble

decimals add subtract
15. 06. 2007
0 views

decimals add subtract

coniferousrees
15. 06. 2007
0 views

coniferousrees

conducing a successful online
15. 06. 2007
0 views

conducing a successful online

Caribou Hills Fire pt1
26. 08. 2007
0 views

Caribou Hills Fire pt1

Caribou Hills Fire pt2
26. 08. 2007
0 views

Caribou Hills Fire pt2

7560
18. 06. 2007
0 views

7560

EOCT GOLF REVIEW ALG
15. 06. 2007
0 views

EOCT GOLF REVIEW ALG

StanescuAndreas
18. 09. 2007
0 views

StanescuAndreas

07Sp Retention
26. 08. 2007
0 views

07Sp Retention

civ3 tech tree rev100
26. 08. 2007
0 views

civ3 tech tree rev100

PPoPP2006
18. 09. 2007
0 views

PPoPP2006

Digitalt innhold
04. 08. 2007
0 views

Digitalt innhold

bateman powerpoint figures
26. 08. 2007
0 views

bateman powerpoint figures

LBC BlueObserving
18. 09. 2007
0 views

LBC BlueObserving

conrad brittannia
26. 08. 2007
0 views

conrad brittannia

PMR bullets2
18. 06. 2007
0 views

PMR bullets2

Dr Levitons Slides
04. 08. 2007
0 views

Dr Levitons Slides