Introduction to Distributed and Parallel Programming
This page will be updated regularly. Please visit (and reload) often.

Introduction to Distributed and Parallel Programming
Course# COL 380 (Sem II, 2022-2023)

Mon, Tue, Fri 1200-1300 LH 108


Subodh Kumar <subodh@cse.*>
Office hour: Mon 12-1


< col380ta@cse.*>

Replace * with

Latest Announcements:

Earlier Announcements:

Other parts of this course:

Course Information

Learn how to program parallel devices efficiently.

Contents: Introduction to concurrency, Race conditions, Atomicity, Semantics of concurrent programs, Examples of distributed algorithms, Client-Server paradigm Parallel architecture, Flynn's classification Shared-memory programming with reference to memory consistency, cache in-coherence, false sharing and mutual exclusion Message passing, High level and collective constructs, Point-to-point communication, multicast and broadcast, Blocking versus non-blocking styles for communication, Message buffering Theoretical models of parallel computation and algorithm analysis, Examples of reduction, prefix-sum Performance metrics: Time, work, Scalability. Task/Communication Dependence graphs, Task decomposition, Data-parallel decomposition, Pipelining Synchronization, barriers, counters/semaphores, Progress, Livelock/Deadlock

Tentative outline

Week Topic Reading
01 Introduction,
OS & Architecture 1
Ch 4.1, 1.4
02 OS & Architecture 2,
Parallel Architecture 1
Ch 1.1-1.5
03 Parallel Architecture 2,
Parallel Programming Models (Intro)
Ch 1.6
Ch 2
04 Memory Consistency and Linearization
Relaxed Consistency Models
Ch 4.2
05 Concurrency and Synchronization (part 1)
Concurrency and Synchronization (part 2)
Ch 4.3
06,07 Distributed Synchronization
Message Passing

Ch 6.2
08 Performance Measures
MPI send semantics
Ch 3.4-3.8
Ch 6.2
09 MPI data
MPI collectives, One-sided communication
Ch 6.2
10 Parallel Fie IO
Parallel Pogramming Models

Ch 2
11 Models of Parallel Computation Ch 3.3
12 Manycore Programming (Recap slide) Ch 6.5
13 Parallel Algorithms Part 1 Ch 7.1-2
14 Parallel Algorithms Part 2 Ch 7.2-3

Academic Integrity Code

Academic honesty is required in all your work. Academic honesty also means not accepting dishonesty from others. You must solve all programming assignments entirely on your own, except where group work is explicitly authorised by the course instructor. This means you must not take, neither show, give or otherwise allow others to take your program code, problem solutions, or other work. You may discuss it only with the professor or TAs.

This means you must protect your code from access by others. Do not leave it where others can find it. Do not give it to someone for submission on your behalf. Do not use any fragment of code obtained online or from someone else, except what is explicitly authorised as a part of the course. Falsifying program output or results is cheating also.

When in doubt, please ask your professor.

Students who are caught cheating will be given a 0 and a full letter-grade penalty. Second violation will result in summary failure from the course. Audit-pass is not an option open to Cheaters.

Assignments & Grading

Work Points Tentative Schedule
Assignment 0 4 Due Jan 16, 1159p.
Assignment 1 9 Due Feb 13, 1159p.
Assignment 2 9 Due Mar 10, 1159p.
Assignment 3 9 Due Apr 03, 1159p.
Assignment 4 9 Due Apr 28, 1159p.
Quizzes 10
Minors 25
Major 25

Late Policy: 0.5 mark/12H of delay (beyond 120H)

Attendance Policy: What institute requires

Audit Policy: At least C grade AND At least 30% each in exams and Quiz+assignment. 75% attendance.

Recommended Text

Further Reading