Question

What are the differences between the following:

-Collective synchronization (barrier) with Collective communication (reduction)

-Master-worker and fork join

Answer 1

In Collective communication, all the processes are reach to a specific point before executing. It acts as a reduction since once process of the communicator collects data from all other processes and performs an operation to find the result. It uses MPI_Reduce() function.

In Collective Synchronisation, it blocks all the processes until a specific synchronisation point is being reached. It acts as a barrier since it blocks the process until all other processes are being reached to synchronisation point successfully. It uses MPI_Barrier() function.

In master-worker pattern, a master or main process is being divided into small chunks which inturn being distributed to several worker processes wherein in Fork-Join pattern which is used to execute parallel light weight processes and threads.