9.8 SIMD and Compiler Auto Vectorization

Modern CPUs are incredibly wide. They don't just process one piece of data at a time. Through a technology called SIMD (Single Instruction, Multiple Data), a single CPU core can perform the exact same mathematical operation on multiple data elements simultaneously.

Let's look at vector registers, how the compiler automatically optimizes your loops, and how to write code that hardware can accelerate.

Traditionally, if you want to add two arrays of 4 integers together, the CPU executes 4 separate add instructions. This is scalar processing.

Modern processors (x86_64 or ARM) have 128-bit, 256-bit (AVX2), or 512-bit (AVX-512) vector registers. A 256-bit register can hold eight 32-bit integers at once. A single SIMD add instruction can update all eight lanes in one issue, but real speedups depend on memory bandwidth, alignment, loop overhead, and whether the core can sustain one vector op per cycle. Treat "up to 8×" as a theoretical upper bound for independent element-wise work, not a guarantee for every loop.

You rarely need to write raw SIMD assembly. Modern C++ compilers (GCC, Clang, MSVC) contain an Auto Vectorizer. When you compile with optimization flags (like -O3 or -mavx2), the compiler analyzes your loops and automatically converts them into SIMD vector instructions.

However, the auto vectorizer is conservative. It will refuse to vectorize a loop if it detects data dependencies (where one iteration depends on the previous one) or complex branching:

// The iterations are independent.
void addArrays(const float* a, const float* b, float* result, int size) {
    for (int i = 0; i < size; ++i) {
        result[i] = a[i] + b[i];
    }
}

// SLOW: The compiler CANNOT auto-vectorize this loop.
// Iteration 'i' depends on the result of iteration 'i-1'.
void runningSum(const float* a, float* result, int size) {
    result[0] = a[0];
    for (int i = 1; i < size; ++i) {
        result[i] = result[i-1] + a[i]; // Data dependency!
    }
}

To ensure the compiler can use SIMD instructions on your loops, follow these rules:

1. Keep it simple: Avoid if statements or complex function calls inside the loop body.

2. Contiguous memory: Use flat arrays or std::vector. SIMD registers load memory in contiguous blocks. If you use a std::list (linked list), the CPU cannot load the scattered data into a vector register efficiently. (The auto vectorizer will take one look at a linked list and just walk away).

3. No aliasing: If the compiler suspects that your input pointer and output pointer might overlap in physical memory, it will refuse to vectorize to prevent data corruption. (In C, you can use the restrict keyword; in C++, standard library algorithms or compiler specific pragmas often handle this).

If the compiler fails to auto vectorize a critical loop, systems engineers use SIMD intrinsics, compiler-provided functions in headers such as <immintrin.h> that map directly to vector instructions. Lesson 9.9 shows how to read vectorization reports and write a small intrinsic kernel.

Contiguous memory and minimal branching still matter: intrinsics express what you want the CPU to do; they do not fix a scattered data layout.