Challenges of Concurrent Memory Access in Parallel Processing

Multiple processors/cores accessing shared memory simultaneously creates conflicts due to true/false sharing. Key challenges:

1. Race Conditions & Data Inconsistency

Two CPUs read-modify-write same location x without synchronization:

P1: R x(5) -> +1 -> W x(6)
P2: R x(5) -> +1 -> W x(6) // lost update, expected 7

Without atomicity, final value non-deterministic.

2. Cache Coherence

Each core has private L1/L2 cache. If P1 caches x=5 and P2 updates to 6, P1 still sees stale 5. Need coherence protocol (MSI/MESI) with invalidation/update, adding bus traffic and latency.

3. Memory Consistency Model

Order in which writes by one CPU become visible to others. Sequential consistency vs weak consistency. Without fences/barriers, reordering by compiler/CPU leads to subtle bugs.

4. Synchronization Overhead

Need locks, semaphores, barriers, atomic CAS. Lock contention serializes execution, creates critical section bottleneck, reduces speedup (Amdahl). Lock granularity trade-off: fine-grain = overhead, coarse-grain = contention.

5. False Sharing

Two unrelated variables a,b in same cache line (64B). P1 writes a, P2 writes b -> line bounces between caches via invalidations despite no true sharing, degrades performance by 10-100x.

6. Memory Contention & Bandwidth

Shared bus/memory controller saturates. In UMA, all accesses via single bus -> bottleneck. In NUMA, remote memory access latency 2-3x local -> non-uniform performance, need locality optimization.

7. Deadlock / Livelock / Priority Inversion

Circular wait on locks -> deadlock. Livelock when both back-off repeatedly. Requires deadlock avoidance protocols.

8. Scalability & Coherence Traffic

Snoopy protocol broadcasts limit to ~32 cores; directory protocols add memory overhead and indirection latency for large systems.

Solution directions: MESI/MOESI + directory, atomic primitives (LL/SC), transactional memory, lock-free data structures, cache-line padding, NUMA-aware allocation.