6.1. Collective Communication: reduce function

The type of reduction of many values down to one can be done with different types of operators on the set of values computed by each process.

6.1.1. Reduce all values using sum and max

In this example, every process computes the square of (id+1). Then all those values are summed together and also the maximum function is applied.

Program file: 12reduction.py

Example usage:

python run.py ./12reduction.py N

Here the N signifies the number of processes to start up in mpi.

run.py executes this program within mpirun using the number of processes given.

Exercises:

  • Run, using N = from 1 through 8 processes.

  • Try replacing MPI.MAX with MPI.MIN(minimum) and/or replacing MPI.SUM with MPI.PROD (product). Then save and run the code again.

6.1.1.1. Explore the code

Find the place in this code where the data computed on each process is being reduced to one value. Match the prints to the output you observe when you run it.


from mpi4py import MPI

def main():
    comm = MPI.COMM_WORLD
    id = comm.Get_rank()            #number of the process running the code
    numProcesses = comm.Get_size()  #total number of processes running
    myHostName = MPI.Get_processor_name()  #machine name running the code

    square = (id+1) * (id+1)

    if numProcesses > 1 :
        #initiate and complete the reductions
        sum = comm.reduce(square, op=MPI.SUM)
        max = comm.reduce(square, op=MPI.MAX)
    else :
        sum = square
        max = square

    if id == 0:        # conductor/root process will print result
        print("The sum of the squares is  {}".format(sum))
        print("The max of the squares is  {}".format(max))

########## Run the main function
main()

6.1.2. Reduction on a list of values

We can try reduction with lists of values, but the behavior matches Python semantics regarding lists.

Program file: 13reductionList.py

Example usage:

python run.py ./13reductionList.py N

Here the N signifies the number of processes to start up in mpi.

run.py executes this program within mpirun using the number of processes given.

Exercises:

  • Run, using N = from 1 through 4 processes.

  • Uncomment the two lines of runnable code that are commented in the main() function. Observe the new results and explain why the MPI.SUM (using the + operator underneath) behaves the way it does on lists, and what the new function called sumListByElements is doing instead.

6.1.2.1. Explore the code

In this code, try to explain what the function called sumListByElements does. If you are unfamiliar with the zip function, look up what it does.

from mpi4py import MPI

# Exercise: Can you explain what this function returns,
#           given two lists as input?
def sumListByElements(x,y):
    return [a+b for a, b in zip(x, y)]

def main():
    comm = MPI.COMM_WORLD
    id = comm.Get_rank()            #number of the process running the code
    numProcesses = comm.Get_size()  #total number of processes running
    myHostName = MPI.Get_processor_name()  #machine name running the code

    srcList = [1*id, 2*id, 3*id, 4*id, 5*id]

    destListMax = comm.reduce(srcList, op=MPI.MAX)
    destListSum = comm.reduce(srcList, op=MPI.SUM)
    #destListSumByElement = comm.reduce(srcList, op=sumListByElements)

    if id == 0:        # conductor/root process will print result
        print("The resulting reduce max list is  {}".format(destListMax))
        print("The resulting reduce sum list is  {}".format(destListSum))
        #print("The resulting reduce sum list is  {}".format(destListSumByElement))

########## Run the main function
main()

Note

There are two ways in Python that you might want to sum a set of lists from each process: 1) concatenating the elements together, or 2) summing the element at each location from each process and placing the sum in that location in a new list. In the latter case, the new list is the same length as the original lists on each process.

You have attempted of activities on this page