SIMD Basics
18 min
SIMD (Single Instruction, Multiple Data) is a different kind of
parallelism than threads or processes: a single CPU instruction operates
on several data elements packed into one wide register at once — e.g.
adding 8 numbers to 8 other numbers in ONE instruction, instead of 8
separate additions. This is exactly what libraries like NumPy exploit
under the hood for vectorized operations, and why arr + arr beats a
Python for loop so dramatically.
import math
def vectorized_speedup(n_elements, lane_width, scalar_cycles_per_element, vector_overhead_cycles):
scalar_cost = n_elements * scalar_cycles_per_element
n_vector_ops = math.ceil(n_elements / lane_width)
vector_cost = n_vector_ops * (scalar_cycles_per_element + vector_overhead_cycles)
return round(scalar_cost / vector_cost, 4)
The theoretical maximum speedup equals lane_width (process 8 elements per instruction -> up to 8x) -- but real speedup is always lower, because each vector instruction still carries fixed overhead that a pure lane_width multiplier ignores. This gap between theoretical and real speedup is exactly why real-world SIMD gains rarely hit the 'obvious' number.
Write `vectorized_speedup(n_elements, lane_width, scalar_cycles_per_element, vector_overhead_cycles)`: scalar cost is `n_elements * scalar_cycles_per_element`. Vector cost is `ceil(n_elements / lane_width)` vector instructions, each costing `scalar_cycles_per_element + vector_overhead_cycles`. Return `scalar_cost / vector_cost`, rounded to 4 decimal places.
What does SIMD (Single Instruction, Multiple Data) actually parallelize, and how is that DIFFERENT from Concurrency's multi-threading?
You can quantify SIMD's realistic speedup accounting for per-instruction overhead, and distinguish data-level parallelism (SIMD) from thread-level parallelism (Concurrency).