CHAPTER 2: Message Passing using Python

We introduce a form of parallel programming called message passing with Python code, using a library called mpi4py. There are code examples that you can run as we introduce common parallel programming patterns.

The code shown here and C versions of it can be found in a GitHub repo for the CSInParallel Project. Look for the folder called Patternlets and the subfolder MPI for the C version and mpi4py for the mpi4py version. The example for the last section (forest fire simulation) can be found in Exemplars/mpi4py-examples/fire.

If you are interested in trying similar examples in C, please look at Chapter 4 of our Intermediate PDC book.

Background Needed

To follow the examples here, we assume that you have some experience writing Python programs that contain loops and that you have worked with lists and dictionaries.

Suggested Uses of this chapter

Sections 1 through 4 can be used as an initial starting point during a class period or a lab. Section 4 ends with an exercise that enables you to try a posed problem using the techniques shown so far.

Sections 5 - 8 go into further details of MPI that are used in many applications. If you want to spend more time learning about MPI, you can add another hour or so studying this material. Section 7 ends with an exercise and section 8 is also a suggested problem to solve.

Section 9 provides a real-world example that combines the patterns and techniques shown, and likely requires an hour or so of study on its own if you really get into the details. But you can try it to see that using parallel ism can improve the time the program takes and let you run larger problems.

• 2.1 First Steps - Programming with Message Passing
  • The SPMD Pattern
  • The Conductor-Worker Pattern
• 2.2 Point to Point Communication
  • Send and Receive Hello World
• 2.3 Splitting loops: the parallel for pattern
• 2.4 Applying the basic patterns
  • Trapezoidal Rule Integration with Message Passing
  • A Brief Aside: the numpy library
  • Exercise - Populate an Array
• 2.5 Deadlock
  • Deadlock Example
  • Fixing Deadlock
• 2.6 Reduction, a coordination Pattern
  • Reduce all values using sum and max
• 2.7 Collective Communication
  • Broadcast
  • Scatter and Gather
• 2.8 Using Scatter and Gather
  • A common example: Array Addition
• 2.9 Real World Example - Forest Fire Simulation
  • The fire functions
  • A single fire burning
  • A full simulation on one process
  • The parallel MPI version
  • Try some other cases to observe how it scales

You have attempted of activities on this page