Introduction to Distributed and Parallel Programming
|
Instructor
Subodh Kumar <subodh@cse.*> |
Contents: Introduction to concurrency, Race conditions, Atomicity, Semantics of concurrent programs, Examples of distributed algorithms, Client-Server paradigm Parallel architecture, Flynn's classification Shared-memory programming with reference to memory consistency, cache in-coherence, false sharing and mutual exclusion Message passing, High level and collective constructs, Point-to-point communication, multicast and broadcast, Blocking versus non-blocking styles for communication, Message buffering Theoretical models of parallel computation and algorithm analysis, Examples of reduction, prefix-sum Performance metrics: Time, work, Scalability. Task/Communication Dependence graphs, Task decomposition, Data-parallel decomposition, Pipelining Synchronization, barriers, counters/semaphores, Progress, Livelock/Deadlock
| Week | Topic | Reading |
| 01 | Introduction, OS & Architecture 1 | Ch 4.1, 1.4 |
| 02 | OS & Architecture 2, Parallel Architecture 1 | Ch 1.1-1.5 |
| 03 | Parallel Architecture 2, Parallel Programming Models (Intro) | Ch 1.6 Ch 2 |
| 04 | Memory Consistency and Linearization Relaxed Consistency Models | Ch 4.2 |
| 05 | Concurrency and Synchronization (part 1) Concurrency and Synchronization (part 2) | Ch 4.3 |
| 06,07 |
Distributed Synchronization Message Passing | Ch 6.2 |
| 08 | Performance Measures MPI send semantics | Ch 3.4-3.8 Ch 6.2 |
| 09 | MPI data MPI collectives, One-sided communication | Ch 6.2 |
| 10 | Parallel Fie IO Parallel Pogramming Models |
Ch 2 |
| 11 | Models of Parallel Computation | Ch 3.3 |
| 12 | Manycore Programming (Recap slide) | Ch 6.5 |
| 13 | Parallel Algorithms Part 1 | Ch 7.1-2 |
| 14 | Parallel Algorithms Part 2 | Ch 7.2-3 |
This means you must protect your code from access by others. Do not leave it where others can find it. Do not give it to someone for submission on your behalf. Do not use any fragment of code obtained online or from someone else, except what is explicitly authorised as a part of the course. Falsifying program output or results is cheating also.
When in doubt, please ask your professor.
Students who are caught cheating will be given a 0 and a full letter-grade
penalty. Second violation will result in summary failure from the course.
Audit-pass is not an option open to Cheaters.
| Work | Points | Tentative Schedule |
|---|---|---|
| Assignment 0 | 4 | Due Jan 16, 1159p. |
| Assignment 1 | 9 | Due Feb 13, 1159p. |
| Assignment 2 | 9 | Due Mar 10, 1159p. |
| Assignment 3 | 9 | Due Apr 03, 1159p. |
| Assignment 4 | 9 | Due Apr 28, 1159p. |
| Quizzes | 10 | |
| Minors | 25 | |
| Major | 25 |
Late Policy: 0.5 mark/12H of delay (beyond 120H)
Attendance Policy: What institute requires
Audit Policy: At least C grade AND At least 30% each in exams and Quiz+assignment. 75% attendance.