The Team: CSinParallel Project

Libby Shoop Macalester College	Dick Brown St. Olaf College	Joel Adams Calvin University	Suzanne Matthews U.S. Military Academy
shoop@macalester.edu

Motivation

Curricular Guidelines: Parallel Computing needs to be included in courses at different levels

• Introductory Programming
• Data Structures and Algorithms
• Computer Systems
• Operating Systems
• PDC Course

Instructors need

• Different Software: OpenMP, MPI, CUDA, OpenACC
• Modular: add to a class session or two
• No hardware or software setup

Students benefit from

• Interactive experience
• Working code examples
• Test of understanding

First steps with OpenMP

Use in early courses

As you see fit: as little or as much as you want to try

PDC for Beginners

• Prerequisite: CS 1 in C/C++ or Python
• Systems Course: a module emphasizing shared memory using C

Outside reading: Chapter 0.1 - 0.3 as background.Demo

Highlight parts of Chapter 0 reading in a mini lecture.

During one class:
students complete Chapter 1.0 – 1.3 as an activity. Demo

Assign a homework problem that uses OpenMP.

(Optional) During a second class: explore speedup with an activity.

• Keep going: explore GPU computing with CUDA

During class:
Chapter 4 for the basics with vector addition Demo

Chapter 5 for a deeper dive, adding timing

Chapter 6 matrix multiply example

• CS2, Algorithms: If students know Python
Use Chapter 2 for distributed computing with message passing Demo

Use in a PDC Course, emphasizing Patterns and Scalability

As above, Ch. 0, 1, 4, 5, 6 of Beginners, plus more from Intermediate PDC

Mini-lecture presentations given before activities from book or GitHub repo.

Potential Course Plan

Wk	HW/SW Topic	Reading Before Class	In-Class Activity
1	Shared Mem OpenMP	Beginners Ch 0.0 - 0.2, 0.4 Intermediate Ch.1.0-1.1 Demo	Intermediate Ch 2 program structure Demo Fork-join, conductor-worker, barrier
2	Shared Mem OpenMP	Beginners Ch 1.2 - 1.3, Demo 0.3.1 - 0.3.3 Demo	Intermediate Ch 2: parallel for loop, race condition, reduction; trapezoidal rule scalability activity (GitHub)
3	Shared Mem OpenMP	Intermediate Ch 3.0	Intermediate Ch 3.1-2 (random number generation) Demo Can use random numbers in a homework problem
4	Shared Mem OpenMP		Intermediate Ch. 2, rest of patternlets default(none), mutual exclusion of critical region Demo
5	Distributed Mem MPI		Intermediate Ch. 4, Program Structure Patterns Message passing Patterns Demo
6	Distributed Mem MPI		Intermediate Ch. 4, Collective Communication Synchronization, code timing
7	Distributed Mem MPI		Intermediate, Ch. 5, Broadcast, scatter, gather Intermediate, Ch. 6, Monte Carlo and trapezoidal rule examples Demo Additional MPI examples (to be added)
8	Performance Ahmdal, Gustafson	Beginners Ch 0.3.4-0.3.5	Catch up review, deeper look at scalability
8	GPU CUDA		Beginners, Ch. 4 CUDA Basics
9	GPU CUDA		Beginners Ch. 5, deeper dive Demo Ch 6.2, Matrix Multiply Demo
9	GPU OpenACC		Intermediate Ch. 7 OpenACC Basics Demo
10	GPU OpenACC		Intermediate Ch. 8 OpenACC next steps Matrix Multiply

Additional Resources

• LearnPDC.org
• All code from books on GitHub CSinParallel repo
• GitHub CSinParallel Activities/Exemplars
• GitHub CSinParallel PDC Assignments
• Two upcoming book chapters from volume 3 of the CDER Book Project

---

---