7. Collective Communication: scatter and gather patterns¶
There are often cases when each process can work on some portion of a larger data structure. This can be carried out by having the master process maintain the larger structure and send parts to each of the worker processes, keeping part of the structure on the master. Each process then works on their portion of the data, and then the master can get the completed portions back.
This is so common in message passing parallel processing that there are two special collective communication functions called scatter and gather that handle this.