Link to this document's Jupyter Notebook

In order to successfully complete this assignment you must do the required reading, watch the provided videos and complete all instructions. The embedded survey form must be entirely filled out and submitted on or before 11:59pm on Tuesday March 23. Students must come to class the next day prepared to discuss the material covered in this assignment.

Pre-Class Assignment: MPI Programming Syntax¶

1. Synchronizing Threads¶

In OpenMP we synchronize threads using #pragma omp barrier; in CUDA we can synchronize threads between blocks using the __synchronize() function.

✅ QUESTION: What is an equivalent process synchronizing function in MPI?

Put your answer here.

2. Blocking vs Non_blocking Communication¶

The MPI_send and MPI_recv functions are "blocking" meaning they do not complete until the message has been passed. MPI_send waits for a conformation from the receiving core and MPI_recv will wait until it gets the message it is looking for. These effectively are types of locks and can cause MPI programs to DEADLOCK.

An alternative to these function are MPI_isend and MPI_irecv. These functions will not block execution of the program. Meaning that MPI_isend will send off a message and assume the recipient will get it. And MPI_irecv will look to see if a message is coming in and if not continue the program while returning a kind of message not found error.

MPI_send
MPI_recv
MPI_isend
MPI_irecv
MPI_Bcast

✅ QUESTION: Give an example, when you might want to use MPI_isend and/or MPI_irecv instead of the typical MPI_send and MPI_recv.

Put your answer here.

3. Broadcasting¶

Think of broadcasting as syncing a variable on one machine with all other machines. A kind of global copy.

int MPI_Bcast( void *buffer, 
               int count, 
               MPI_Datatype datatype, 
               int root, 
               MPI_Comm comm )

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
        int rank;
        int buf;
        const int root=0;

        MPI_Init(&argc, &argv);
        MPI_Comm_rank(MPI_COMM_WORLD, &rank);

        if(rank == root) {
           buf = 777;
        }

        printf("[%d]: Before Bcast, buf is %d\n", rank, buf);

        /* everyone calls bcast, data is taken from root and ends up in everyone's buf */
        MPI_Bcast(&buf, 1, MPI_INT, root, MPI_COMM_WORLD);

        printf("[%d]: After Bcast, buf is %d\n", rank, buf);

        MPI_Finalize();
        return 0;
}

✅ QUESTION: Give an example about when you might want to use the MPI_Bcast function?

Put your answer here.

4. Assignment wrap up¶

Please fill out the form that appears when you run the code below. You must completely fill this out in order to receive credits for the assignment!

Direct Link to Google Form

If you have trouble with the embedded form, please make sure you log on with your MSU google account at googleapps.msu.edu and then click on the direct link above.

✅ Assignment-Specific QUESTION: What is an equivalent process synchronizing function in MPI?