GBB Victoria

Information about GBB Victoria

Published on October 29, 2007

Author: Yuan

Source: authorstream.com

Content

Slide1:  Montage, Pegasus and ROME G. B. Berriman, J.C. Good, M. Kong, A. Laity IPAC/Caltech J. Jacob, A. Bergou, D. S. Katz JPL R. Williams CACR E. Deelman, G. Singh, M.-H. Su, C. Kesselman ISI THE US NATIONAL VIRTUAL OBSERVATORY Montage:  Montage Version 1.7 approved for public release Download page will be at montage.ipac.caltech.edu Complete users guide including caveats Tested and validated on 2MASS 2IDR images on single processor Linux platforms Tested on 10 WCS projections with mosaics smaller than 2 x 2 degrees and coordinate transformations Equ J2000 to Galactic and Ecliptic First release emphasizes accuracy in photometry and astrometry 20 modules; 7560 Lines of code; 2595 test cases executed 119 defects reported and 116 corrected Montage Test Results Summary:  Montage Test Results Summary Montage: The Grid Years:  Montage: The Grid Years Re-projection is slow (2 min for a2 MASS image, single processor 1.4 GHz Linux box)  parallel processing Grid is an abstraction - array of processors, grid of clusters, … Montage has loosely coupled code - run on any environment Prototype version of a methodology for running on any “grid environment” Many parts of the process can be parallelized Build a Directed Acyclic Graph (DAG) DAG is a script that enables parallelization Describes what is to be run and when, so flow of processing is specified DAG is submitted to standard tools for execution War and Peace Nebula:  War and Peace Nebula Montage and Pegasus:  Montage and Pegasus Pegasus takes the abstract workflow description, locates the compute resources and data and produces a concrete DAG which can be run on the Grid Why ROME and Why Not Apache?:  Why ROME and Why Not Apache? Apache accepts http requests over a TCP/IP network and returns html documents Accepts requests anonymously, parses requests , checks if executable is in path, runs it Works very well when response is fast BUT it has no memory of the request and so cannot manage information and respond to messages Apache’s limitations are exposed when data processing or requests take an indeterminate time (hours, days, even weeks): complex database queries, large-scale image processing or large scale statistical analysis  A simple, portable request management environment which can work in conjunction with existing browsers, HTTP services and custom client environments to provide reliable execution of long-lived jobs and can communicate status information in more detailed ways to clients. ROME Demonstration- Registration:  ROME Demonstration- Registration User preferences:  User preferences ROME Demonstration - Job Submission:  ROME Demonstration - Job Submission Custom order for mosaics of ISSA images submitted to a Linux processor - Job Information Filters:  Job Information Filters ROME Interactive Request Monitor :  ROME Interactive Request Monitor Rho Oph & Orion:  Rho Oph & Orion Slide14:  ROME Architectural Diagram Clients include Browsers, NVO portals, and user-built custom code The heart of ROME is an EJB container tightly coupled with a DBMS Container where special hooks exist to simplify synchronization of user and service interaction Container and DBMS immaterial - during initial development used WebLogic and Informix ROME does not start processing- special “processor” does this Contact ROME (via Servlet URL) to get job parameter Starts CGI program for user Process messages from the CGI program through stdout Process kill or abort requests “Processor” is currently a very simple JAVA VM Can be run anywhere on the net. Can in principle be implemented in other languages. Applications can be as simple as reusing existing CGI programs, but should support more complex processing. ROME Processing Scenario:  ROME Processing Scenario User registers with ROME. This is necessary for messaging (including completion notification). The user “identity” is simply their email address. User submits job to ROME through the User Registration servlet. The user is added to the DBMS. A processor (there can be many) asks ROME for a job to process (through the Get Next Request servlet). The processor starts (or potentially continues talking to) an application (e.g. a CGI program) which does the real work. The application at a minimum emits messages (text printed to stdout) when job started and at the completion. In addition it can optionally emit progress report messages at any time. On completion, all data products of the application will have been saved to a temporary “workspace” in the application file system. This workspace is HTTP accessible and the completion message from the application contains a pointer to this data. All messages are forwarded from the processor to the ROME core where they are stored in the DBMS and forwarded to the user either directly (if they are using a client which can register a message socket with ROME), later (if the user reconnects with such a client), or eventually by email (email is usually only for completion status messages). The user (manually through a browser or with degrees of automation through custom GUI clients) retrieves the data. Whither Next?:  Whither Next? Submit requests through ROME for processing jobs running on grids Montage, others, . . . Support “executives” requesting complex jobs and pipelines Support registries Pass security certificates Open Source EJB server

Related presentations


Other presentations created by Yuan

ISPS Code Final
05. 11. 2007
0 views

ISPS Code Final

6 Flexural Component Design
04. 01. 2008
0 views

6 Flexural Component Design

ABAPresentation
13. 04. 2008
0 views

ABAPresentation

eBDSinGMS
27. 03. 2008
0 views

eBDSinGMS

NM
14. 03. 2008
0 views

NM

market forces
24. 02. 2008
0 views

market forces

VesnaDamnjanovic
20. 02. 2008
0 views

VesnaDamnjanovic

Meeting1
07. 01. 2008
0 views

Meeting1

Age of Limits
07. 01. 2008
0 views

Age of Limits

Deviance
04. 01. 2008
0 views

Deviance

ijcai distinguished 01
27. 09. 2007
0 views

ijcai distinguished 01

sunum VS2008
28. 11. 2007
0 views

sunum VS2008

Stamatakis18 3 04
29. 11. 2007
0 views

Stamatakis18 3 04

cam
04. 12. 2007
0 views

cam

KfW short
24. 10. 2007
0 views

KfW short

Hist12 f Mussolini
31. 10. 2007
0 views

Hist12 f Mussolini

kanwar presentation
02. 11. 2007
0 views

kanwar presentation

IT SAPSD Performance
06. 11. 2007
0 views

IT SAPSD Performance

SpadaPlanck07 ThePhysicsAMS 02
29. 10. 2007
0 views

SpadaPlanck07 ThePhysicsAMS 02

shiphandling
07. 11. 2007
0 views

shiphandling

newslink volkswagen case study
16. 11. 2007
0 views

newslink volkswagen case study

LBL SC Seminar
20. 11. 2007
0 views

LBL SC Seminar

The Divine Flute Vamshinew
22. 11. 2007
0 views

The Divine Flute Vamshinew

MACEDONIA Brussels June 2007
23. 11. 2007
0 views

MACEDONIA Brussels June 2007

5 MotionDance1
23. 11. 2007
0 views

5 MotionDance1

pneumonia
28. 12. 2007
0 views

pneumonia

Adjective Clauses
29. 12. 2007
0 views

Adjective Clauses

2002aahe sotlbridges
01. 01. 2008
0 views

2002aahe sotlbridges

PresentationKaycaCot ton
15. 11. 2007
0 views

PresentationKaycaCot ton

IRSIS
07. 11. 2007
0 views

IRSIS

Dempsey Diane
06. 11. 2007
0 views

Dempsey Diane

Tana Jan07
31. 12. 2007
0 views

Tana Jan07

PPT 29
04. 10. 2007
0 views

PPT 29

umcineurasia
26. 10. 2007
0 views

umcineurasia

Jatoi
03. 01. 2008
0 views

Jatoi

B01HumEco2004
24. 10. 2007
0 views

B01HumEco2004

Predator prey interactions
30. 12. 2007
0 views

Predator prey interactions

Ullman Impact
28. 11. 2007
0 views

Ullman Impact

slides 1 28 05
28. 12. 2007
0 views

slides 1 28 05

RockMinerals 2008SHOW
06. 12. 2007
0 views

RockMinerals 2008SHOW

Floriculture and Floral Design
11. 12. 2007
0 views

Floriculture and Floral Design

atv driptorch
12. 11. 2007
0 views

atv driptorch

TUNISIA
31. 10. 2007
0 views

TUNISIA

DePaoloERSP40406
08. 11. 2007
0 views

DePaoloERSP40406

DixonMonBatchProcess Vents
06. 11. 2007
0 views

DixonMonBatchProcess Vents

NintendoForwardChain ing
24. 12. 2007
0 views

NintendoForwardChain ing

Ozturk tsaps Oct6
01. 11. 2007
0 views

Ozturk tsaps Oct6

NZ Beech Forest
02. 01. 2008
0 views

NZ Beech Forest

Portugal project Tyler
26. 11. 2007
0 views

Portugal project Tyler

DO Lect6
19. 11. 2007
0 views

DO Lect6

Ch 20 powerpoint
03. 10. 2007
0 views

Ch 20 powerpoint

session23 lowenthal
30. 11. 2007
0 views

session23 lowenthal

the flowers of bermuda
14. 12. 2007
0 views

the flowers of bermuda

sw week april06
29. 10. 2007
0 views

sw week april06