3.1. Parallel for Loop Program Structure: equal chunks

Navigate to: ../02.parallelLoop-equalChunks/

Make and run the code:


mpirun -hostfile ~/hostfile -np N ./parallelLoopEqualChunks

Here the N signifies the number of processes to start up in MPI.


3.1.1. Explore the code

In the code below, notice the use of the variable called REPS. This is designed to be the total amount or work, or repetitions, that the for loop is accomplishing. This particular code is designed so that if those repetitions do not divide equally by the number of processes, then the program will stop with a warning message printed by the conductor process.

Remember that because this is still also a SPMD program, all processes execute the code in the part of the if statement that evaluates to True. Each process has its own id, and we can determine how many processes there are, so we can choose where in the overall number of REPs of the loop each process will execute.

#include <stdio.h> // printf()
#include <mpi.h>   // MPI

int main(int argc, char** argv) {
    const int REPS = 8;                      // repetitions in a loop
    int id = -1, numProcesses = -1;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &id);
    MPI_Comm_size(MPI_COMM_WORLD, &numProcesses);

    // In this example, ensure that the REPS can ben evenly divided by the
    // number of processors and that the number of processes doesn't exceed REPS.
    // If either is the case, stop.
    if ((REPS % numProcesses) == 0 && numProcesses <= REPS) {

      int chunkSize = REPS / numProcesses;      // find chunk size
      int start = id * chunkSize;               // find starting index
      int stop = start + chunkSize;             // find stopping index

      for (int i = start; i < stop; i++) {      // iterate through our range
          printf("Process %d is performing iteration %d\n", id, i);

    } else {
      if (id == 0) {
          printf("Please run with -np divisible by and less than or equal to %d\n.", REPS);

    return 0;
You have attempted of activities on this page