2. Program Structure Patterns¶
This initial set of pattern examples illustrates how many distributed processing programs are structured. For these examples, it is useful to look at the overall organization of the program and become comfortable with the idea that multiple processes all running this code simultaneously, in no particular guaranteed order. The lack of a guaranteed order is also known as non-determinism.
We will start with the most basic pattern, the single program, multiple data or SPMD pattern. It is used in every example you will see here in this tutorial. Next, we will look at conductor-worker pattern, which is often used in tandem with the SPMD pattern.