Lesson 13

Date: 4/19/2017
High Performance Computing (part I: OpenMP)
Linux System Administration

Parallel Computing Paradigms

Shared memory

Distributed memory

  • All the processors or cores have access to the same global address space. 
  • Most of the main stream commodity based x86 desktops and laptops have CPUs with multiple cores.
  • Relatively easy to program applications, for example, with OpenMP.
  • Performance scalability with the number of cores may be low due to CPU-RAM traffic increase, high latency of RAM, and maintaining of the cache coherency.
  • The CPUs have access only to their own memory space.
  • Data is exchanged via messages passed over the network.
  • Message passing function calls have to be used in a code development, for example, MPI.
  • Good performance scalability may be achieved with the number of nodes increase.

  • Take me to the Course Website