Gold Accounting Manager

Information about Gold Accounting Manager

Published on April 17, 2008

Author: Gourmet

Source: authorstream.com

Content

Gold Accounting Manager:  Gold Accounting Manager Scott Jackson Scalable Systems Software Center SC2002 20 NOV 2002 Introduction:  Introduction In a nutshell, Gold is: A resource bank (allocation management system) tracks and manages resource usage. Much like a bank, it associates a cost to computing resources and allows resource credits to be allocated to users and projects. As jobs complete or as resources are utilized, projects are dynamically charged and resource usage recorded. An accounting system Can be dynamically customized to record any type of accounting data – pacct, sar, node availability, etc. An information service Also functions as a powerful generalized information service useful in a variety of means, such as providing mappings for meta-scheduling mappings of machines to resources, applications, accounts, users, etc. But First, A Little Background…:  But First, A Little Background… Scalable Systems Software Center Research, develop and support an integrated suite of systems software and tools for the effective management and utilization of the highest scale computational resources. SciDAC Scientific Discovery through Advanced Computing – A DOE initiative to improve the impact of scientific computing QBank A dynamic allocation management system developed at PNNL and in use at about a dozen sites. Motivation:  Motivation Show return on investment Funding sources have invested heavily in a supercomputer and require a means to show that it is being utilized efficiently. Fairness Management needs a means to fairly distribute the underlying computing resources (processors, memory, disk) to the various users and projects. Capacity Planning Accurate allocation and usage information is needed to effectively make decisions on resource commitments and new procurements. Centralized Access Control Many organizations want centralized control over which users and projects have access to what machines and for how long. Motivation (from a meta-scheduling slant):  Motivation (from a meta-scheduling slant) Meta-computing Trust issues Guarantees Distributed Accounting Local Control Equitable trade agreement Security Different userids, accounts, execution environments on different resources Nonfunctional Requirements:  Nonfunctional Requirements Scalable Targeting systems with tens of thousands of processors and thousands of simultaneous jobs. Secure Will use strong authentication (no clear-text passwords) to prevent unauthorized access and data encryption to prevent sensitive information from being intercepted (XML-DSIG and XML-ENC). Fault Tolerant Database performs automatic rollbacks on failed transactions. A distributed design that includes data replication will be researched. Nonfunctional Requirements (cont.):  Nonfunctional Requirements (cont.) Open Source Allows free distribution, allows sites to make local modifications, and derived works, and promotes sharing of patches, ports and enhancements from user community. Portable Written in Java – initially tested to a reference Linux platform and expanded to include architectures used at the largest DOE computing facilities. Easy to Use Web accessible GUI (based on PHP and Javascript) will help managers, users and admins gain the access they need from their own PC’s. Operational Characteristics:  Operational Characteristics Supports familiar bank operations Deposits, withdrawals, transfers, refunds, balance checks and bank statements Reservations Before a job runs, a reservation (or hold) is placed on the account based on the wallclock limit. This prevents overdrafts. Quotations In a meta-scheduling environment it is useful to know how much a job is going to “cost” so that you can make a decision on the best place to run your job. Hierarchical Accounts Projects can be nested (trickle down deposits, trickle up withdrawals) Dynamic Resource Management Interaction:  Dynamic Resource Management Interaction Make Deposits, etc. Submit Job Balance Check Make Reservation Start Job Job Completes Remove Reservation & Make Withdrawal Dynamic Resource Management Interaction (with meta-scheduling):  Dynamic Resource Management Interaction (with meta-scheduling) Resource Manager (PBS, LL) Allocation Manager (Gold) 0 2 1 5 3 3 7 Make Deposits, etc. Submit Job Locate Feasible Systems & Obtain Quote Stage Job Balance Check Make Reservation Start Job Job Completes Remove Reservation & Make Withdrawal Meta-Scheduler (Silver) Scheduler (Maui) 6 8 4 Allocations:  Allocations An allocation is a collection of resource credits valid toward an arbitrary group of users, machines and projects and a timeframe for expenditure. Commonly associated with a single project (account), and a set of users and machines. Fine-grained control of who can use how much within a project can be achieved by multiple allocations in the same project. The dimensions of Allocation Management:  The dimensions of Allocation Management Projects Grand Challenge Development Weather Modeling Chem101 Navy SETI Viz … Users Tom Sheri Scientist Developer Admin Workshop Manager … Resources Time DOE PNNL LLNL SDSC ANL MPP1 Colony Jupiter Allocation-User Distribution Possibilities:  Allocation-User Distribution Possibilities Allocation-Machine Distribution Possibilities:  Allocation-Machine Distribution Possibilities Allocation Timeframes:  Allocation Timeframes Allocation Timeframes:  Allocation Timeframes Allocation Timeframes:  Allocation Timeframes Allocation Timeframes:  Allocation Timeframes Allocation Timeframes:  Allocation Timeframes Journaling:  Journaling State Preservation Preserves indefinite historical state of all objects and records Bank Statements Journaling allows bank statements to show balances for any arbitrary time in the past Undo/Redo With a powerful querying/updating comes the potential for rampant administrative mistakes Time Travel You can run any command as if it were an arbitrary date in the past Flexible Charging Mechanism:  Flexible Charging Mechanism Besides CPU, a resource supplier can charge based on the amount of memory, disk, or any other consumable resource as well as quality of service, primetime, nodetype, class, etc. An external pricing engine interface will allow any sort of charging algorithm to be used such as dynamic price adjustment according to load or queue backlog, a query to an external information service or a cached second-price auction result. Traceback Mechanism:  Traceback Mechanism Allows all parties of a transaction (resource requestor and provider) to have a record of the resource utilization and to have a say as to whether or not the job should be permitted to run, based on their independent policies and priorities. A job will only run if all parties are agreeable to the idea that the target resources can be used in the manner and amount requested. MetaLBNL runPNNL Meta Account PNNL LBNL Traceback debit Flexible and Extensible:  Flexible and Extensible Powerful Querying/Updating Capabilities Create, query, modify, delete, undelete Support for operators (equals, less than, not equal, matching, etc.) Conjunctive expression combinations (and, or) Object joined queries Dynamically Extensible New object/record types and their fields can be dynamically created/modified through the regular query language (command line or GUI). This capability turns this system into a generalized information service. This capability is extremely powerful and can be used for meta-scheduling resource-mapping, an interface for persistence for other components, and all varieties of accounting possibilities! Schedule:  Schedule 4Q02 Requirements gathering completed and release initial Resource Management Interface Specs 2Q03 QBank bundled with SSS initial release (possible alpha-testing on Gold) 2Q04 Beta release of Gold 4Q05 Production release of Gold (includes support) Contact Information:  Contact Information Scott Jackson Pacific Northwest National Laboratory [email protected] (509) 376-2205 Scalable Systems Software Center http://www.scidac.org/ScalableSystems QBank documentation and download http://www.emsl.pnl.gov:2080/docs/mscf/qbank

Related presentations


Other presentations created by Gourmet

Christmas Customs
09. 07. 2007
0 views

Christmas Customs

Okan Bayrak
22. 04. 2008
0 views

Okan Bayrak

listenc20
14. 04. 2008
0 views

listenc20

CS IDB
09. 04. 2008
0 views

CS IDB

p8whyusesolar
07. 04. 2008
0 views

p8whyusesolar

Continents Thank You Minnie
30. 03. 2008
0 views

Continents Thank You Minnie

Jairam G LNG
28. 03. 2008
0 views

Jairam G LNG

Wang02
27. 03. 2008
0 views

Wang02

Christmas
09. 07. 2007
0 views

Christmas

2 Enigma
01. 01. 2008
0 views

2 Enigma

Earth Shakes Rattles and Rolls
02. 10. 2007
0 views

Earth Shakes Rattles and Rolls

dino show
12. 10. 2007
0 views

dino show

DocNewsNo1605Documen tNo967
22. 10. 2007
0 views

DocNewsNo1605Documen tNo967

Everest
03. 09. 2007
0 views

Everest

Geography Jeopardy
03. 09. 2007
0 views

Geography Jeopardy

Internet in Nepal
03. 09. 2007
0 views

Internet in Nepal

world wonders
03. 09. 2007
0 views

world wonders

IATA IOSA
03. 09. 2007
0 views

IATA IOSA

RCE Kesennuma Omose E
02. 11. 2007
0 views

RCE Kesennuma Omose E

Harabagiu2005
20. 11. 2007
0 views

Harabagiu2005

Navigating UF March2003
03. 10. 2007
0 views

Navigating UF March2003

TWAR05006 WinHEC05
03. 01. 2008
0 views

TWAR05006 WinHEC05

Rapport Italie Farrace EN
01. 01. 2008
0 views

Rapport Italie Farrace EN

TCE 2004 Farm Safety Report
29. 12. 2007
0 views

TCE 2004 Farm Safety Report

Presentation GEF
23. 11. 2007
0 views

Presentation GEF

Diwali Oct26 06
09. 07. 2007
0 views

Diwali Oct26 06

Deepavali presentation
09. 07. 2007
0 views

Deepavali presentation

Celebrations
09. 07. 2007
0 views

Celebrations

celebrate nashville presentation
09. 07. 2007
0 views

celebrate nashville presentation

Realismo
01. 10. 2007
0 views

Realismo

C Ainsley
15. 10. 2007
0 views

C Ainsley

How has family life changed
24. 02. 2008
0 views

How has family life changed

usability
03. 09. 2007
0 views

usability

orosz 2005
12. 10. 2007
0 views

orosz 2005

EC106FA98CH52
14. 12. 2007
0 views

EC106FA98CH52

bic 1 intro
12. 03. 2008
0 views

bic 1 intro

the peninsula war
10. 10. 2007
0 views

the peninsula war

centennial
09. 07. 2007
0 views

centennial

2003 bell 08 mehallis
19. 06. 2007
0 views

2003 bell 08 mehallis

2003 bell 08 garlander
19. 06. 2007
0 views

2003 bell 08 garlander

2003 bell 07 ericmartin
19. 06. 2007
0 views

2003 bell 07 ericmartin

2003 bell 15 page
19. 06. 2007
0 views

2003 bell 15 page

2003 bell 18 bulkley
19. 06. 2007
0 views

2003 bell 18 bulkley

Hermes
29. 09. 2007
0 views

Hermes

arpresentation2 htm
09. 07. 2007
0 views

arpresentation2 htm

lecture29
05. 01. 2008
0 views

lecture29

2003 bell 07 chrislotspeich
19. 06. 2007
0 views

2003 bell 07 chrislotspeich

Witt Herb Resistance
03. 09. 2007
0 views

Witt Herb Resistance

Satire Power Point
20. 02. 2008
0 views

Satire Power Point

Gpo AH Acuaticos Panama 02 2006
23. 10. 2007
0 views

Gpo AH Acuaticos Panama 02 2006

charla cartagena
22. 10. 2007
0 views

charla cartagena

GCB in Space 4
27. 09. 2007
0 views

GCB in Space 4

2003 bell 08 posey
19. 06. 2007
0 views

2003 bell 08 posey

2003 bell 08 birky
19. 06. 2007
0 views

2003 bell 08 birky

changes to specialty training
03. 01. 2008
0 views

changes to specialty training

2003 bell 12 kristhomke
19. 06. 2007
0 views

2003 bell 12 kristhomke

A Snapshot of Salems Streets
30. 12. 2007
0 views

A Snapshot of Salems Streets

2007910164952
03. 01. 2008
0 views

2007910164952

East Asia and the World Willard
28. 02. 2008
0 views

East Asia and the World Willard

2003 bell 08 bakker
19. 06. 2007
0 views

2003 bell 08 bakker

AAC JET 2004
23. 10. 2007
0 views

AAC JET 2004

Mars1
03. 09. 2007
0 views

Mars1

LeeEddington
01. 11. 2007
0 views

LeeEddington