About Intermediate Parallel and Distributed Computing¶
Intermediate Parallel and Distributed Computing is written by Elizabeth Shoop. Some material in this book was created by Joel C. Adams, Richard A. Brown, or Suzanne J. Matthews.
Adams, Brown, Matthews and Shoop form the leadership team for CSinParallel, an NSF-funded initiative for broadening the exposure of parallel computing in the undergraduate computing curriculum.
In this book we have made an effort to emphasize classic patterns found in programs for two types of PDC:
Shared memory multicore computing using threads and the OpenMP standard and
Distributed computing using processes and the MPI standard.
We start with an overview of the research done on PDC patterns in Chapter 1 and proceed to provide many programs that illustrate these patterns. These programs can be run directly from this book, enabling learners to experience PDC concepts immediately.
We also include our own new patterns for generating random numbers using a library that is designed to properly generate random numbers whose sequence is repeatable when using shared memory with OpenMP.
Lastly, we have included examples for how to use OpenACC to accelerate examples on both multicore CPUs and GPUs.
NOTE: This book is still a work in progress, however the first eight chapters (patterns, random numbers, examples, and OpenACC) are ready for use.
We hope plan to provide applications that demonstrate further use of the patterns for these various technologies and references to assignments.
How to use this book¶
Intermediate Parallel and Distributed Programming is a free on-line textbook designed as a follow-on to the first book we created as an introduction to parallel and distributed computing, which we affectionately call PDC For Beginners, which you can find at learnpdc.org.
Unlike a traditional textbook, we do not anticipate that this textbook be adopted in full in a particular course; rather, we imagine that components of this textbook can be used as background reading and exploration for various courses. The primary use might be for a PDC course for undergraduates, but could also be used in a scientific computing course or for a high performance computing course.
This book assumes that you have some programming knowledge, and are at the very least familiar with C. For readers unfamiliar with PDC concepts, we encourage you to first check out our Introduction to Parallel Computing, called PDC for Beginners, at learnpdc.org.