In order to successfully complete this assignment you must do the required reading, watch the provided videos and complete all instructions. The embedded survey form must be entirely filled out and submitted on or before 11:59pm on Sunday March 21. Students must come to class the next day prepared to discuss the material covered in this assignment.

Pre-Class Assignment: The Message Passing Interface (MPI)¶

1. Distributed Memory Systems¶

✅ QUESTION: In a Distributed Memory System we no longer can pass information though memory. How is data passed in a distributed memory system?

Put your answer here.

2. Quick Send / Recieve Example¶

A GPU does not do automatic memory caching like a CPU. Instead you need to do all of work for memory management yourself. The following video gives a brief overview of the concept of tiling.

✅ QUESTION: In the example showed in the video there are four (4) processors. When the program is done, which processor has the final answer?

Put your answer here

3. Initializing MPI job¶

The following code initializes a typical MPI job. Read the MPI documentation and answer the questions.

int rank, size;

MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Rank (ID) of this process
MPI_Comm_size(MPI_COMM_WORLD, &size); // Total size of MPI job

✅ QUESTION: What is MPI_COMM_WORLD?

Put your answer here.

✅ QUESTION: Where is the MPI_COMM_WORLD Defined?

Put your answer here.

✅ QUESTION: It is possible to use something besides MPI_COMMM_WORLD, provide an example when this might be useful in your program?

Put your answer here.

4. Basic Send/Recieve¶

Read the following blog post provides a good descrtiption of how to use MPI Send/Recieve

http://mpitutorial.com/tutorials/mpi-send-and-receive/

int MPI_Send( void* data,
              int count,
              MPI_Datatype datatype,
              int destination,
              int tag,
              MPI_Comm communicator) 

int MPI_Recv( void* data,
              int count,
              MPI_Datatype datatype,
              int source,
              int tag,
              MPI_Comm communicator,
              MPI_Status* status)

Example Send and recieves:

MPI_Send(&processor_name,str_length,MPI_INT,0,1,MPI_COMM_WORLD);

MPI_Recv(&hostname[0][proc],str_length,MPI_INT,proc,1,MPI_COMM_WORLD,&status);

The types of data (and their C equivelents) are as follows.

MPI datatype	C equivalent
MPI_SHORT	short int
MPI_INT	int
MPI_LONG	long int
MPI_LONG_LONG	long long int
MPI_UNSIGNED_CHAR	unsigned char
MPI_UNSIGNED_SHORT	unsigned short int
MPI_UNSIGNED	unsigned int
MPI_UNSIGNED_LONG	unsigned long int
MPI_UNSIGNED_LONG_LONG	unsigned long long int
MPI_FLOAT	float
MPI_DOUBLE	double
MPI_LONG_DOUBLE	long double
MPI_BYTE	char

✅ QUESTION: In the above example code, the MPI_INT data type is incorrect, which data type should be used instead?

Put your answer here.

✅ QUESTION: Although MPI_INT is an error, why doesn't it cause a bug in the code?

Put your answer here.

5. Finishing a job¶

At the end of the program you need to include the MPI_Finalize function to close down all of the communication channels and clean up the program.

MPI_Finalize();

6. MPI Example¶

✅ DO THIS: Use the getexample command to download the HelloMPI program and get it to compile and run on the HPCC.

✅ QUESTION: Analyze the code (it is short). See if you can find the MPI related commands and try to figure out what they do. What are their inputs? What are their outputs? What do they do?

Put your observations here.

✅ QUESTION: How far did you get with the MPI Example? Where did you get stuck?