lecture26 fall 2003

Information about lecture26 fall 2003

Published on November 19, 2007

Author: Abbott

Source: authorstream.com

Content

Lecture 26 Asynchrony, Shared Objects and Concurrent Programming:  Lecture 26 Asynchrony, Shared Objects and Concurrent Programming This material is not available in the textbook. The notes in the online powerpoint presentation contains much of the explanations given in class. Uniprocessor vs. Multiprocessor Machines:  Uniprocessor vs. Multiprocessor Machines P1 P2 P10 P1 memory Shared memory Uniprocessor Multiprocessor Concurrent Programming:  Concurrent Programming Challenge: Coordinating access to items in shared memory. Persistent vs. Transient Communication:  Persistent vs. Transient Communication Persistent Communication medium: the sending of information changes the state of the medium forever. Example: Blackboard. Transient communication medium: the change of state is only for some limited time period. Example: Talking. Testing Many Primes Concurrently:  Testing Many Primes Concurrently Task: Print all primes from 1 to 1010 in some order Available: A machine with 10 processors Goal: Achieve speed up factor of 10, that is, new time to print all primes should be one tenth of time for single processor. We will partition the inputs among the 10 processors. But what is the best way to do this? Testing Many Primes Concurrently:  Testing Many Primes Concurrently P1 P2 P10 1 109 2x109 1010 Simplest way to split the inputs among processors: Each processor Pi gets 109 consecutive numbers to test. … … Testing Many Primes Concurrently: Code for Simplest Partition:  Testing Many Primes Concurrently: Code for Simplest Partition (define (P i) (let ((counter (+ 1 (* (- i 1) (power 10 9)))) (upto (* i (power 10 9)))) (define (iter) (if (< counter upto) (begin (if (prime? counter) (display counter) #f) (increment-counter) (iter)) 'done)) (iter))) (parallel-execute (P 1) (P 2) ... (P 10)) Load (Un)balancing: work is split unevenly:  Load (Un)balancing: work is split unevenly Some processors have less primes to test… Some composite numbers are easier to test… P1 P2 P10 1 109 2x109 1010 Even this rather simple problem is hard to treat analytically. To achieve load balancing, we need to split the work range dynamically! A Shared Counter Object:  A Shared Counter Object (define (make-shared-counter value) (define (fetch) value) (define (increment) (set! value (+ 1 value)) (define (dispatch m) (cond (((eq? m 'fetch) (fetch)) (eq? m 'increment) (increment)) (else (error “unknown request”)))) dispatch) (define shared-counter (make-shared-counter 1)) This shared counter object can be accessed by two operations: “fetch” (read) and “increment” (restricted form of write). Using the Shared Counter:  Using the Shared Counter (define (P i) (define (iter) (let ((index (shared-counter 'fetch))) (if (< index (power 10 10)) (begin (if (prime? index) (display index) #f) (shared-counter 'increment) (iter)) 'done)) (iter))) (parallel-execute (P 1) (P 2) ... (P 10)) This Solution Doesn’t Work:  This Solution Doesn’t Work It Will Never Work!? :  It Will Never Work!? Real World Example: Walking in the Street Look / Move Fetch / Increment Computers: Accessing Memory Fischer Lynch & Paterson: Impossible to solve!!! Need to “glue” the Fetch / Increment pair into one indivisible operation: Fetch-and-Increment The Fetch-and-Increment Operation:  The Fetch-and-Increment Operation (define (make-shared-counter value) (define (fetch-and-increment) (let ((old value)) (set! value (+ old 1)) old)) (define (dispatch m) (cond (((eq? m 'fetch-and-increment) (fetch-and-increment)) (else (error ``unknown request -- counter'' m)))) dispatch)   Instantaneous Shared Counter Fetch-and-inc A Correct Shared Counter:  A Correct Shared Counter (define shared-counter (make-shared-counter 1)) (define (P i) (define (iter) (let ((index (shared-counter 'fetch-and-increment))) (if (< index (power 10 10)) (begin (if (prime? index) (display index) #f) (iter)) 'done)) (iter)))   (parallel-execute (P 1) (P 2) ... (P 10)) Implementing Fetch-and-Inc:  Implementing Fetch-and-Inc To make the program work we need an “instantaneous” implementation of fetch-and-increment. How can we do this: Special Hardware. Built-in synchronization instructions. Special Software. Use regular instructions -- the solution will involve waiting. Software: Mutual Exclusion Mutual Exclusion:  Mutual Exclusion (mutex 'start) (let ((old value)) (set! value (+ old 1)) old) (mutex 'end)) Only one process at a time can execute these instructions P1 P2 P10 1 1 P2 returns 1 Mutex count The Story of Alice and Bob (these Alice and Bob are unrelated to Rivest, Shamir and Adelman, the new recipients of the Turning award) :  The Story of Alice and Bob (these Alice and Bob are unrelated to Rivest, Shamir and Adelman, the new recipients of the Turning award) * As told by Leslie Lamport The Mutual Exclusion Problem:  The Mutual Exclusion Problem Requirements: Mutual Exclusion: there will never be two dogs simultaneously in the yard. No Deadlock: if only one dog wants to be in the yard it will succeed, and if both dogs want to go out, at least one of them will succeed. Cell Phone Solution:  Cell Phone Solution Coke Can Solution:  Coke Can Solution Flag Solution -- Alice:  Flag Solution -- Alice Flag Solution -- Bob:  Flag Solution -- Bob Flag Solution -- Both:  Flag Solution -- Both Intuition: Why Mutual Exclusion is Preserved:  Intuition: Why Mutual Exclusion is Preserved Each perform: First raise the flag, to signal interest. Then look to see if the other one has raised the flag. One can claim that the following flag principle holds: since Alice and Bob each raise their own flag and then look at the others flag, the last one to start looking must notice that both flags are up. Why is there no Deadlock?:  Why is there no Deadlock? Since Alice has priority over Bob…if neither is entering the critical section, both are repeatedly trying, and Bob will give Alice priority. Unfortunately, the algorithm is not a fair one, and Bob's dogs might eventually grow very anxious :-) The Morals of our Story:  The Morals of our Story The Mutual Exclusion problem cannot be solved using transient communication. (I.e. Cell-phones.) The Mutual Exclusion problem cannot be solved using interrupts or interrupt bits (I.e. Cans) The Mutual Exclusion problem can be solved with one bit registers (I.e. Flags), memory locations that can be read and written (set!-ed). We cheated a little: the arbiter problem… While moving, is the flag really up or down? The Solution and Conclusion:  The Solution and Conclusion (define (Alice) (loop (mutex 'begin) (Alice-dog-in-yard) ;; critical section (mutex 'end) )) Question: then why not execute all the code of the parallel prime-printing algorithm in a critical section? Answer: Amdahl’s Law:  Answer: Amdahl’s Law Speedup Overall = Execution_time_Old / Execution_time_New = 1 Fraction_Enhanced Speedup_Enhanced 1- Fraction_Enhanced + If 40% of the execution time can be sped-up by a factor of 10 using 10 processors. Then Overall Speedup = 1/(0.6 + (0.4/10)) = 1/0.64 = 1.56 Length of Critical Sections :  Length of Critical Sections We want Linear Speedup: 100 processors = 100 times faster   By Amdahl's law that means that Fraction_Enhanced = 1  No Sequential Parts!!!!! For a speedup of only 99 times: 1-Fraction_Enhanced= 0.0001 In other words, we must try and make the parts that are sequential (the critical sections) as short as possible!!!!! Summarizing it all:  Summarizing it all To summarize it all, our world is asynchronous, and yet with a bit of luck we have ways of overcoming this asynchrony to create powerful concurrent algorithms. “Life is the synchronicity of chance”

Related presentations


Other presentations created by Abbott

Fantastic Pictures
31. 07. 2007
0 views

Fantastic Pictures

jovanovi
02. 05. 2008
0 views

jovanovi

ours2
30. 04. 2008
0 views

ours2

golding
28. 04. 2008
0 views

golding

SandersD
22. 04. 2008
0 views

SandersD

MasterclassBasicsOfH edging
18. 04. 2008
0 views

MasterclassBasicsOfH edging

Energy Management Basics
17. 04. 2008
0 views

Energy Management Basics

jpn ma
13. 04. 2008
0 views

jpn ma

0802 Roderick Deane
10. 04. 2008
0 views

0802 Roderick Deane

Athirapally Vazhachaal Falls
26. 09. 2007
0 views

Athirapally Vazhachaal Falls

Lecture7
10. 10. 2007
0 views

Lecture7

qa overview cmu
16. 10. 2007
0 views

qa overview cmu

11 GUIDE presentation Porvoo 10
19. 10. 2007
0 views

11 GUIDE presentation Porvoo 10

Egyptian Quiz
21. 10. 2007
0 views

Egyptian Quiz

AGE200524122005
24. 10. 2007
0 views

AGE200524122005

Henry VIII wwt bam
21. 08. 2007
0 views

Henry VIII wwt bam

DOE LNG FORUM 14
07. 11. 2007
0 views

DOE LNG FORUM 14

matzgdh6
22. 10. 2007
0 views

matzgdh6

BdReviewFINAL2006 lq
16. 11. 2007
0 views

BdReviewFINAL2006 lq

session2
20. 11. 2007
0 views

session2

WWF
26. 11. 2007
0 views

WWF

Chocolate 01
03. 10. 2007
0 views

Chocolate 01

MARE 494 Lecture 12
12. 10. 2007
0 views

MARE 494 Lecture 12

ErnestoRivera
22. 10. 2007
0 views

ErnestoRivera

InherentSafety Westray
10. 12. 2007
0 views

InherentSafety Westray

tema1
24. 10. 2007
0 views

tema1

Libecap powerpoint
04. 01. 2008
0 views

Libecap powerpoint

Assam
05. 01. 2008
0 views

Assam

Samuels Mol Pharm 11 03
24. 02. 2008
0 views

Samuels Mol Pharm 11 03

Stock Market Game
24. 02. 2008
0 views

Stock Market Game

nv specialitypp
29. 10. 2007
0 views

nv specialitypp

104 04MagnConst
13. 11. 2007
0 views

104 04MagnConst

clingan
04. 03. 2008
0 views

clingan

EARNIN x007E 1 2
23. 10. 2007
0 views

EARNIN x007E 1 2

aes
13. 03. 2008
0 views

aes

Semprini
16. 03. 2008
0 views

Semprini

KPMG PresentationSMEfinal
26. 03. 2008
0 views

KPMG PresentationSMEfinal

temple of Zeus
21. 08. 2007
0 views

temple of Zeus

Australia 2 March 20051
28. 09. 2007
0 views

Australia 2 March 20051

maillistnews
05. 10. 2007
0 views

maillistnews

feb 5
27. 09. 2007
0 views

feb 5

ykis06 K F Liu
15. 11. 2007
0 views

ykis06 K F Liu

schreiber
12. 10. 2007
0 views

schreiber

Pisani
30. 10. 2007
0 views

Pisani

200752414331362
11. 10. 2007
0 views

200752414331362

decisiontheoryreview
04. 01. 2008
0 views

decisiontheoryreview

a Macroecologia Alimentar 2
22. 10. 2007
0 views

a Macroecologia Alimentar 2

praesentationschule
02. 11. 2007
0 views

praesentationschule

Wireless Data Business
18. 06. 2007
0 views

Wireless Data Business

WHA userguide Customer2006
18. 06. 2007
0 views

WHA userguide Customer2006

wales
18. 06. 2007
0 views

wales

VON04s I2
18. 06. 2007
0 views

VON04s I2

UTeV Rick Field 12 14 06
18. 06. 2007
0 views

UTeV Rick Field 12 14 06

UDDI Overview Presentation
18. 06. 2007
0 views

UDDI Overview Presentation

2006 02 28T124652
25. 03. 2008
0 views

2006 02 28T124652

SoprolecheAgo 2005
23. 10. 2007
0 views

SoprolecheAgo 2005

ZadehTalk
18. 10. 2007
0 views

ZadehTalk

ihepccc hepix
24. 10. 2007
0 views

ihepccc hepix

jedwards
02. 10. 2007
0 views

jedwards

Wireless Mobile Platform OAEP
15. 06. 2007
0 views

Wireless Mobile Platform OAEP

Taverna Workbench
15. 06. 2007
0 views

Taverna Workbench

Job Services with Genius Portal
15. 06. 2007
0 views

Job Services with Genius Portal

MSN screenshots
15. 06. 2007
0 views

MSN screenshots

Caltech, Peck Fest Jan 05
15. 06. 2007
0 views

Caltech, Peck Fest Jan 05

rslt3218
17. 10. 2007
0 views

rslt3218

6 26
17. 10. 2007
0 views

6 26

vp sigmetrics 04 adversaries
18. 06. 2007
0 views

vp sigmetrics 04 adversaries

calcagno
22. 10. 2007
0 views

calcagno

LegoDesign
31. 12. 2007
0 views

LegoDesign

Ted Hanss Internet2
15. 06. 2007
0 views

Ted Hanss Internet2

Mary and Darnley
21. 08. 2007
0 views

Mary and Darnley

1 partie
17. 10. 2007
0 views

1 partie

Techmission Safe Families
15. 06. 2007
0 views

Techmission Safe Families

Camille Page CrocodileGuy MK8
11. 10. 2007
0 views

Camille Page CrocodileGuy MK8

Salandayia
04. 10. 2007
0 views

Salandayia

B4 Hamilton
03. 01. 2008
0 views

B4 Hamilton

SEC062804
25. 10. 2007
0 views

SEC062804