6. Collective Communication: reduction pattern¶
There are often cases when every process needs to complete a partial result of an overall computation. For example if you want to process a large set of numbers by summing them together into one value (i.e. reduce a set of numbers into one value, its sum), you could do this faster by having each process compute a partial sum, then have all the processes communicate to add each of their partial sums together.
This is so common in parallel processing that there is a special collective communication function called reduce that does just this.