Effect of Instruction Set, Compiler, CPU Implementation, Cache/Memory Hierarchy on CPU Performance - CPI Analysis (7M)

Fundamental Performance Equation

$CPU_{time} = IC \times CPI \times T_{c} = IC \times CPI / f$

Where $IC$ = dynamic Instruction Count, $CPI$ = cycles per instruction (average, effective), $f$ = clock frequency, $T_c$ = clock period.

Also $MIPS = f / (CPI \times 10^6)$; $CPI_{effective} = \sum_i (CPI_i \times F_i) + Stall_{memory}+Stall_{pipeline}$ ; with memory: $CPI = CPI_{base} + MissRate \times MissPenalty$.

Every factor influences one or two of those three terms; optimal performance balances them.

1. Instruction Set Architecture (ISA) Effect

ISA defines operations, formats, addressing modes, registers.

• Program Length (IC):

  • CISC (x86): Complex instructions (REP MOVS, MUL, CALL) do more work per instruction => fewer instructions ($IC$ down) but longer and variable-length (1-15 bytes) => higher $CPI$ per instruction (5-12). Code dense, good for small memory.
  • RISC (ARM, MIPS): Simple load-store, fixed 4-byte, 3-operand => more instructions needed to do same task ($IC$ up ~30%) but each executes in 1 cycle ideal, easy pipelining.

  Example: Memory copy: x86 REP MOVSB = 1 instruction; MIPS needs loop: LW, SW, ADDI, BNE = 4 instructions. IC difference.

• Clock Rate ($f$):

  • CISC decode complex → longer critical path → lower max $f$. RISC decode trivial → higher $f$ (can pipeline deeper).

• Effective CPI:

  • ISA determines $CPI_i$ per instruction class. Variable CISC CPI varies widely; RISC keeps near 1 (plus load-use stalls). Also ISA addressing mode complexity adds extra µ-cycles.

Trade-off: $IC \downarrow$ vs $CPI \uparrow$ and $f \downarrow$.

2. Compiler Technology Effect

Compiler maps high-level code to ISA sequence; determines actual $IC$, instruction mix $F_i$, and how well CPI hides hazards.

• IC:

  • Optimization -O2 removes dead code, common subexpression elimination, loop unrolling, inlining → $IC \downarrow$ 10-50%.
  • Poor code: extra spills, redundant loads → $IC \uparrow$.

• CPI:

  • Instruction scheduling: Reorders to avoid RAW stalls, fills branch delay slots, uses forwarding effectively → stalls ↓ → $CPI_{effective} \downarrow$ from 1.5 to 1.1 (30% gain).
  • Register allocation: Keeps variables in registers → fewer loads/stores (memory CPI high) → $CPI \downarrow$ and $IC$ of memory ops ↓.
  • Choice of instructions: Uses cheap sequences (shift instead of multiply by 8) → $CPI_i$ lower.

• Clock Rate: Little direct effect, but better code may allow better branch prediction locality.

Example: Without scheduling: LW R1; ADD R2,R1 → 1 stall; scheduled: insert independent instruction between → 0 stalls → CPI saved.

3. CPU Implementation & Control Effect

Implementation is microarchitecture realizing same ISA (e.g., 486 vs Pentium for same x86 ISA).

• Clock Rate ($f$):

  • Technology: Smaller transistor, deeper pipeline (15-20 stages) enables higher $f$ (3 GHz vs 100 MHz).
  • Control: Hardwired CU → faster decode → higher $f$; microprogrammed → lower $f$ but more complex instruction support.

• CPI:

  • Pipelining: Converts $CPI$ from ~5 (non-pipelined multicycle) to ~1 + stall; $Speedup \approx n_{stages}$.
  • Superscalar / Superpipelined: Issue 2-8 instructions per cycle → $CPI <1$ (IPC >1). E.g., 4-wide processor can have $CPI_{effective}=0.25$.
  • Hazards & Stalls: Deeper pipeline increases branch mispredict penalty (15 cycles vs 3) → CPI penalty up if prediction poor.
  • Multicycle vs Single-cycle: Single-cycle CPU $CPI=1$ but long $T_c$ (slowest instr), multicycle average $CPI=3-4$ but short $T_c$ plus pipeline gives best both.

Equation with pipeline: $T_c = max(T_{stage}) + T_{latch}$. Better implementation reduces $T_{stage}$ variance.

• IC: Generally none (same ISA → same dynamic instruction count if ignoring micro-ops). But micro-op translation (x86 translates 1 CISC to 2-4 RISC µops) can affect internal IC count but not programmer visible IC.

Result: Implementation trades $CPI$ ↔ $f$. e.g., Deep pipeline: $f \uparrow 2x$ but branch miss CPI penalty rises, golden point exists.

4. Cache and Memory Hierarchy Effect

Memory hierarchy bridges CPU (ns) ↔ DRAM (100 ns) gap; determines stall component of CPI.

• Clock Rate: Cache access time is part of cycle: L1 hit ~ 1 cycle keeps $f$ high; larger cache slower but reduces miss. Design chooses split.

• Effective CPI - Dominant:

  • No cache: Every $LW/SW$ suffers 50-100 cycles miss → $CPI_{effective} = CPI_{base} + Memory\_access\_per\_instr \times (1 - h) \times MissPenalty$.
  • With hierarchy: $T_{eff} = h_1 T_{L1} + (1-h_1)[h_2 T_{L2} + (1-h_2)T_{RAM}]$.
  • Example: Base CPI=1.0, 30% loads/stores, miss rate 5%, penalty 20 cycles → $CPI =1.0 +0.3*0.05*20 =1.30$ (+30% time). Improving hit rate from 95% to 98% via larger cache → CPI → 1.12.

• Program Length (IC):

  • Larger block/cache with prefetch can reduce instruction fetch IC stalls but adds overhead of coherence/ TLB misses. Prefetch / O-O-O hides.

• Interactions:

  • Larger cache → lower miss rate → $CPI\downarrow$ but larger access time → may limit $f$ and higher power/cost.
  • Write policy: Write-back reduces bus traffic → less stall.

Overall $CPU_{time}$ with hierarchy: $CPU_{time} = IC \times (CPI_{execution} + Stall_{mem}) \times T_c$.

Summary Matrix (Mark-Scoring Table)

Factor	Primary effect on	Direction	Example justification
ISA - CISC complexity	$IC \downarrow$	fewer instr	REP MOVS vs loop
	$CPI \uparrow$, $f\downarrow$	each CISC takes micro-steps, complex decode	CPI 5 vs 1
Compiler optimization	$IC \downarrow$	eliminate	DCE -20%
	$CPI \downarrow$	schedule to avoid stalls	reorder LW-USE
CPU Implementation (pipeline, superscalar, hardwired)	$f \uparrow$	deeper pipe, faster tech	3 GHz vs 33 MHz
	$CPI \downarrow$ (~1 to <1)	overlap, parallel issue	4-wide IPC=3
Cache Hierarchy	$CPI_{eff} \downarrow$ greatly	hit reduces stall	$h$=95% saves 20 cycles
	$f$ slightly ↓ if large	size ↔ speed	L1 32KB 2 cycles vs L2 1MB 10 cycles

Combined Formula Justification:
$Performance \propto 1 / CPU_{time} \propto f / (IC \times CPI_{base+stall})$

To improve performance: Reduce IC via RISC? Actually need product minimal. Recent trends: Keep ISA fixed (x86), aggressively improve Compiler, Implementation (O-O-O, branch prediction, speculative), and Hierarchy (multi-level cache, prefetch, TLB) to hide $CPI_{stall}$ while pushing $f$ via scaling. Amdahl emphasizes bottleneck - e.g., even infinite $f$ but 20% memory stalls caps speedup.

For 7M answer: Write equation first, then 4 subsections each stating effect on IC, f, CPI with 2-line justification + numeric example, conclude with trade-offs matrix.