Parallel Computing Notes

These are my notes for parallel computing. More specifically, the first part of the notes will be for the course CSC367 at the University of Toronto. There may be also more notes from my personal experience after the course, or notes about my project TBD.

I will do my best to update notes within 24 hours of the lecture. If you have any feedback, questions, clarifications, or if you see a typo or a mistake, feel free to email me at jonah.chen@mail.utoronto.ca.

Cheers!
Jonah

Credit for the template and notetaking format goes to my friend QiLin Xue

Table of Contents

1. Part 1 (Lecture 1): Why Parallel Computing, Moore's Law, and How to Write Fast Code.
2. Part 2 (Lectures 2-4): Single Processor Optimization and Memory Hierarchy
3. Part 3 (Lectures 5-7, 14, 19): Designing Parallel Algorithms
4. Part 4 (Lectures 8-10): Shared Memory Architectures, pthreads, OpenMP
5. Part 5 (Lectures 11-13): Distributed Systems and Message Passing, MPI
6. Part 6 (Lectures 15-18): GPU Programming, CUDA