Lectures Schedule: Slot "D" : Tues (9:00 - 10:00), Wed (8:00-9:00), Thu(9:00-10:00)

TA: Rishi Kumar: e-mail: csd00393@cse.iitd.ernet.in

Objective:

This course is aimed at providing students with a deep knowledge of the techniques and tools needed to understand today's and tomorrow's Supercomputers, and to efficiently program them.
Today's supercomputers range from expensive highly parallel shared/distributed memory platforms down to cheap local networks of standard workstations. But the problems associated with software development are the same on all architectures: the user needs to recast his or her algorithm or application in terms of parallel entities (tasks, processes, threads, or whatever) that will execute concurrently. Parallelism is difficult to detect in an automatic fashion because of data dependencies. In many cases, one needs to perform some form of algorithm restructuring to expose the parallelism. Finally, to realize the restructured algorithm in terms of software on a specific architecture may be quite complicated.

In this course we plan to cover and understand the nuts and bolts of developing engineering  applications. For instance our study of Shared memory parallel architectures and programming with OpenMP and Ptheards, Distributed memory message-passing parallel architectures and programming, portable parallel message-passing programming using MPI. This will also include design and implementation of parallel numerical and non-numerical algorithms for  engineering, applications. In addition we will study performance evaluation and benchmarking on today's Supercomputers.

Topics:

Introduction to Supercomputing. Various supercomputer architectures: Shared memory parallel architectures and programming with OpenMP and Ptheards. Distributed memory message-passing parallel architectures and programming, portable parallel message-passing programming using MPI. Design and implementation of parallel numerical and non-numerical algorithms for scientific and engineering,  applications.

PRE-REQUISITES
Knowledge of C/C++/Fortran/Java programming. Understanding of Unix Operating System.

Text Books

MAIN READING

Kai Hwang and Zhiwei Xu, Scalable Parallel Computing, McGraw Hill New York, 1997.

Vipin Kumar, Ananth Grama, Anshul Gupta, George Karypis, Introduction to Parallel Computing, Design and Analysis of Algorithms, Redwood City, CA, Benjmann/Cummings (1994).

Barry Wilkinson and Michael Allen, Parallel Programming, Pearson Education Asia, 1999.

SUPPLEMENTARY READING

Ian T. Foster, Designing and Building Parallel Programs, Concepts and tools for Parallel Software Engineering, Addison-Wesley Publishing Company (1995).

W. Gropp, E. Lusk, and A. Skjellum, Using MPI: Portable Parallel Programming with the Message-Passing, MIT Press, 1994

Al Geist, Adam Beguelin, Jack Dongarra, Weicheng Jiang, Robert Manchek and Vaidy Sunderam, PVM: Parallel Virtual Machine - A Users' Guide and Tutorial for Networked Parallel Computing, MIT Press, 1994

Rohit Chandra , Ramesh Menon, Leo Dagum, David Kohr, Dror Maydan, and Jeff McDonald,Parallel Programming in OpenMP, Morgan Kaufmann, 2000

  Matrix Computations by Gene H. Golub / Charles F. Van Loan

Computing Systems to be used:

PARAM

IBM RS/6000

SUN Fire V6800

Pentium - Linux Cluster

Lecture Notes 

MPI-Part-I

MPI-Part-II

OpenMP Notes (PPT) 

OpenMP Fortran Spec

OpenMP C Spec

Hands-On

MPI

OpenMP

Pthreads

Assignments

This Course will have three Assignments of 10 Marks each. 

Links

For Information about Sun Performance Library please see SUN Workshop

Unix Tutorials

Stanford Unix Programming Tools Manual (pdf file)
GNU Bash Manual (html file)
GNU Make Manual (html file)

Other Parallel Information Sites

• NHSE - National HPCC Software Exchange
• Netlib Repository at UTK/ORNL
• BLAS Quick Reference Card
• LAPACK
• ScaLAPACK
• GAMS - Guide to Available Math Software
• Center for Research on Parallel Computation (CRPC)
• Supercomputing & Parallel Computing: Conferences
• Supercomputing & Parallel Computing: Journals
• High Performance Fortran (HPF) reports
• High Performance Fortran Resource List
• Fortran 90 Resource List
• Major Science Research Institutions from Caltech
• Message Passing Interface (MPI) Forum
• High Performance Fortran Forum
• OpenMP
• PVM
• Parallel Tools Consortium
• DoD High Performance Computing Modernization Program
• DoE Accelerated Strategic Computing Initiative (ASCI)
• National Computational Science Alliance

Related On-line Textbooks

• Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM Publication, Philadelphia, 1994.
• PVM - A Users' Guide and Tutorial for Networked Parallel Computing, MIT Press, Boston, 1994.
• MPI : A Message-Passing Interface Standard
• LAPACK Users' Guide (Second Edition), SIAM Publications, Philadelphia, 1995.
• MPI: The Complete Reference, MIT Press, Boston, 1996.
• Using MPI: Portable Parallel Programming with the Message-Passing Interface by W. Gropp, E. Lusk, and A. Skjellum
• Parallel Computing Works, by G. Fox, R. Williams, and P. Messina (Morgan Kaufmann Publishers)
• Computational Science Education Project TextBook.
• Designing and Building Parallel Programs. A dead-tree version of this book is available by Addison-Wesley.
• High Performance Fortran (HPF), a course offered by Manchester and North High Performance Computing Training & Education Centre, United Kingdom
• Parallel Processing Laboratory, Colorado School of Mines