Distinguishes multi-processor computer architectures according to how they can be classified along the two independent dimensions of Instruction Stream and Data Stream.
Each of these dimensions can have only one of two possible states: Single or Multiple.
All processors can have access to all memory as a global address space.
Multiple processors can operate independently but share same memory resources.
Uniform Memory Access (UMA)
Non-Uniform Memory Access (NUMA)
- User-friendly global address space
- Fast and Uniform data sharing between tasks
- Lack of scalability between memory and CPUs relationship
- Synchronization relies on programmer
Requires a communication network to connect inter-processor memory.
- Memory is scalable with number of CPUs
- Each CPU can rapidly access its own memory
- Cost effectiveness: can use commodity, off-the-shelf processors and networking
- Data communication is mostly responsability of the programmer
- Global memory data structures can not easily map to this memory organization
- Non-uniform memory access times
Hybrid Distributed-Shared Memory
The largest and fastest computers in the world today employ both shared and distributed memory.
- Whatever is common to both shared and distributed memory architectures.
- Increased scalability is an important advantage
- Increased programmer complexity is an important disadvantage
Parallel Programming Pattern's Definition
In the book of Patterns for Parallel Programming from Massingill, Sanders and Mattson, there's a pattern language that helps on the process of understanding and designing parallel programs.
Programmers should start their design of a parallel solution by analyzing the problem within the problem's domain to expose exploitable concurrency.
Is the problem large enough and the results significant enough to justify?
Our goal is to refine the design and move it closer to a program that can execute tasks concurrently by mapping the concurrency onto multiple UEs running on a parallel computer.
The key issue at this stage is to decide which pattern or patterns are most appropriate for the problem.
We call these patterns Supporting Structures because they describe software constructions or "structures" that support the expression of parallel algorithms.
Create the set of UEs.
Manage interactions between them and their access to shared resources