2.8 Using Scatter and Gather

A common example: Array Addition

Let’s look at the problem of array addition, where we start with 2 arrays, A and B, and add them to get a new array C. To add A to B, take each element of A, A[i], add it to the corresponding element of B, B[i], and place the result in C[i].

We will use simple arrays that enable us to check our work:

  • Array A will contain a 1 at every location.

  • Array B will contain a 2 at every location.

  • Array C should contain a 3 at every location when the addition is complete.

Steps of the Program

After the usual MPI setup, we check for array length divisible by number of processes (code given). Next comes steps described with text and diagrams below.

  1. Create numpy arrays A, B, and C on the conductor with initial values. Worker procesesses have ‘None’ for these. Also create local_A, local_B, and local_C on all processes as empty numpy arrays.

  2. Scatter equal parts of A to all processes from the conductor.

Step 1. create arrays

Step 2. Scatter

../_images/arrayAdd1.png ../_images/arrayAdd2.png
  1. Add each local_A array element to its corresponding local_B array element and place it in the same location in local_C.

  2. Gather local_C from each process back into the conductor.

Step 3. perform local add

Step 4. Gather

../_images/arrayAdd3.png ../_images/arrayAdd4.png

At the end, we perform some debugging checks to ensure that it worked.

Try it yourself

Fill in the code below by adding what is needed for each step where it says TODO in the comments.

Hints: - Look at the middle of section 2.4 or the previous scatter-gather example for the simplest way to create an empty array with a given length with numpy. - Refer to the previous section for the syntax of the Scatter and Gather commands (be sure to use capital S and capital G).


If you have errors in your code, you may simply get output that looks like this:


You will need to try a fix. It is best to work on each step and try running to be sure you haven’t introduced an error.


The following program demonstrates how to implement the array addition program. Make sure you try to solve the exercise yourself before looking at the solution!

You have attempted of activities on this page