Hierarchical Memory Structure

Memory is organized as a pyramid/hierarchy based on trade-off between speed - cost - capacity - access time. Faster memory per bit is costlier and smaller.

              ↑ Speed, Cost/bit
              |  Capacity ↓
    +-----------------+
    |  CPU Registers  |  < 1 ns, <1KB, fastest, costliest
    +-----------------+
    |     Cache       |  L1: 2-5 ns, 32-64KB
    |  L1 | L2 | L3   |  L2: 5-10 ns, 256KB-8MB  SRAM
    +-----------------+
    |   Main Memory   |  50-100 ns, 4-32 GB, DRAM
    |   (Primary)     |
    +-----------------+
    |  Secondary      |  5-10 ms, 256GB-4TB
    |  SSD / HDD      |
    +-----------------+
    |  Tertiary /     |  sec-min, TB-PB, Optical Tape
    |  Off-line       |  Archival, slowest, cheapest
    +-----------------+
              ↓ Slower, Cheaper, Larger

Levels in Detail

1. Registers (Top)

• Inside CPU, <32-64 words, flip-flops, 0.5-1 ns, directly accessed by ALU.

2. Cache Memory (SRAM)

• Between CPU and RAM. Holds copies of frequently used blocks (locality of reference).
• L1 split I/D, L2 unified, L3 shared. Hit ratio ~ 90-95%. Block size 32-128B.
• Managed by hardware.

3. Main Memory (DRAM)

• Primary memory, volatile, 4-32 GB. Byte-addressable. Managed by OS + MMU.

4. Secondary (Magnetic/Flash)

• Non-volatile, virtual memory backing store, page file on SSD/HDD.

5. Tertiary/Offline

• Tape libraries, optical jukebox for backup.

Key Principles

• Inclusion: Data at level $i$ is subset/copy of level $i+1$ (Cache contains copy of RAM block).
• Locality: Temporal (recently used reused) & Spatial (neighbors used) → hierarchy works.
• Access time: $T_{eff}= H\times T_{cache} + (1-H)\times (T_{cache}+T_{main})$ where $H$ is hit rate.
• Cost: $C_{avg}= \frac{\sum C_i S_i}{\sum S_i}$

Advantages: Gives illusion of large, cheap memory with speed near registers; transparent to programmer via memory management; exploits locality.

Modern example PC: Registers ~1KB, L1 64KB, L2 512KB, L3 8MB, RAM 16GB, SSD 1TB, HDD/Cloud TB.