Fundamentals of Parallel Programming#

This course by The Molecular Sciences Software Institute (MolSSI) teaches the fundamentals of parallel programming techniques, with an emphasis on MPI and OpenMP parallelization. It assumes basic familiarity with Python programming, which is the subject of the MolSSI Python Scripting for Computational Molecular Science lessons. Episodes 4 and 6 assume basic familiarity with C++. To see the full MolSSI’s education mission statement, please see here. This lesson is under continual development, please report issues to the workshop repository.

If you see a subject you would like to contribute to, submit a pull request!

Prerequisites

Students should be familiar with opening the Terminal window and creating and navigating files in bash.

Workshop Lessons#

Set-Up#

Lesson Title

Questions

Objectives

Set-up

How do I set up my environment?

Download files required for the lesson

Introduction#

Lesson Title

Questions

Objectives

Introduction to Parallel Programing

What is parallelization and how does it work?

  • Understand the motivation for parallelizing code.

  • Understand how machine architecture affects the ability to parallelize code.

  • Be aware of the types of parallelization common in computational chemistry.

Distributed-Memory Parallelization#

Lesson Title

Questions

Objectives

Introduction to Distributed-Memory Parallelization

What is distributed-memory parallelization?

Understand the advantages and limitations of distributed-memory parallelization.

MPI Hands-On - mpi4py

How can I use MPI to parallelize a Python code?

  • Learn how to prepare an environment that includes mpi4py.

  • Learn the basics of writing an MPI-parallelized code.

  • Explore point-to-point and collective MPI operations

MPI Hands-On - C++

How can I use MPI to parallelize a compiled code?

  • Compile and run C++ codes that are parallelized using MPI.

  • Use proper MPI error handling.

  • Learn how to use non-blocking communication methods.

  • Use a debugger with an parallelized code.

Shared-Memory Parallelization#

Lesson Title

Questions

Objectives

Introduction to Shared-Memory Parallelization

What is shared-memory parallelization?

Understand the advantages and limitations of shared-memory parallelization.

OpenMP Hands-On

How can I use OpenMP to parallelize a code?

  • Use OpenMP to implement shared-memory parallelization.

  • Learn how to identify and fix race conditions.

  • Optimize the performance of an OpenMP code.