Parallel Computing for Beginners

By: Joel C. Adams, Richard A. Brown, Suzanne J. Matthews, and Elizabeth Shoop (CSinParallel) and Contributing Authors: Dorian Arnold, Steven Bogaerts, and John Rieffel

NOTE: THIS BOOK IS A WORK IN PROGRESS! However, most of it has been edited and tested.

Table of Contents

• About Parallel Computing for Beginners, 2nd edition
  • How to use this book
  • Contributing
• CHAPTER 0: Introduction
  • 0.0 Computing
  • 0.1 Computing Hardware
  • 0.2 Parallel Computing
  • 0.3 Parallel Performance
  • 0.4 Parallel Programming Strategies
  • 0.5 Parallel Computing Hardware Platforms
• CHAPTER 1: Shared Memory
  • 1.0 Shared Memory Preliminaries
  • 1.1 First Steps - Programming Shared Memory
  • 1.2 Race Conditions
  • 1.3 Reduction
  • 1.4 Real World Problem - Drug Design
• CHAPTER 2: Message Passing using Python
  • 2.1 First Steps - Programming with Message Passing
  • 2.2 Point to Point Communication
  • 2.3 Splitting loops: the parallel for pattern
  • 2.4 Applying the basic patterns
  • 2.5 Deadlock
  • 2.6 Reduction, a coordination Pattern
  • 2.7 Collective Communication
  • 2.8 Using Scatter and Gather
  • 2.9 Real World Example - Forest Fire Simulation
• CHAPTER 3: Common Algorithmic Patterns
  • 3.1 Parallel Selection
  • 3.2 Shared Queue
• CHAPTER 4: GPU Computing Basics with CUDA C
  • 4.1 The manycore GPU device in a host
  • 4.2 GPU Programming model
  • 4.3 Mapping Threads to Data Elements
  • 4.4 Example: Vector Addition
  • 4.5 Vector Addition example with CUDA Unified Memory
• CHAPTER 5: A Deeper Dive into CUDA
  • 5.1 Vector Addition Example: exploring CUDA by timing and experimenting
  • 5.2 Vector Addition Example: see hardware in action
  • 5.3 Exploring the effect of block and grid size settings
  • 5.4 Vector Addition Example: exploring thread block mappings
• CHAPTER 6: Applications
  • 6.1 AI Applications - Minimax Algorithm
  • 6.2 Matrix Multiplication
• CHAPTER 7: Further Avenues of Exploration

To Cite:

Adams J.C., Brown R., Matthews S.J., Shoop E., and others. PDC for Beginners, 2.0 Edition. Available online: https://dx.doi.org/10.55682/VXWY1300

Acknowledgements

This work is sponsored in part by

U.S. National Science Foundation (NSF) Collaborative Research Grants DUE-1822480/1822486/1855761. Collaborative Research: CSinParallel: Experiential Learning of Parallel and Distributed Computing through Sight, Sound, and Touch.

This work is also sponsored in part by:

United States Military Academy (USMA) Award Number W911NF-23-2-0044. USMA, as the Federal awarding agency, reserves a royalty-free, nonexclusive and irrevocable right to reproduce, publish, or otherwise use this software for Federal purposes, and to authorize others to do so in accordance with 2 CFR 200.315(b).

License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

You have attempted of activities on this page