9.6 False Sharing and CPU Cache Line Bouncing

Atomics can avoid a mutex for a narrow operation, but they can still suffer from contention and false sharing. The impact ranges widely by workload and hardware, so treat a dramatic slowdown as a hypothesis to benchmark rather than a promise.

Let's look at cache coherency protocols, the physical layout of cache lines, and how to prevent cache line bouncing.

Processors transfer memory in cache-line-sized chunks. 64 bytes is common on contemporary x86 and ARM systems, but it is not a C++ or universal hardware constant. If neighboring variables share a line, independent writes can still contend.

Modern processors use a hardware Cache Coherency Protocol called MESI (Modified, Exclusive, Shared, Invalid) to ensure all cores see the same data.

If Core A modifies a variable, its L1 cache marks that cache line as 'Modified'. The hardware then broadcasts an invalidate signal on the coherence interconnect. If Core B has a copy of that same cache line, Core B's cache marks it as 'Invalid'. If Core B reads it again, it suffers a cache miss, the line may be forwarded from another core's cache or loaded from memory.

Imagine you have two independent threads. Thread 1 updates counter1, and Thread 2 updates counter2. They share no logical state. However, because they are declared next to each other, they reside on the exact same 64 byte cache line.

#include <atomic>

// These variables sit right next to each other in physical RAM.
// They will be loaded into the exact same 64-byte CPU cache line!
struct SharedCounters {
    std::atomic<int> counter1{0};
    std::atomic<int> counter2{0};
};

SharedCounters counters;

When Thread A updates counter1, the CPU invalidates the entire cache line for Thread B. When Thread B updates counter2 a microsecond later, it incurs another cache miss, possibly refilling from Core A's cache or memory, and invalidates the line for Thread A.

The cache line bounces back and forth between cores over the on-chip coherence interconnect. Even though the variables are independent, the hardware fights over the memory block. Performance plummets.

To reduce false sharing, separate frequently written variables with alignas and the C++17 constant std::hardware_destructive_interference_size. It is a portable interference-size hint, not a guaranteed measurement of the target CPU's cache line, so confirm the effect with a benchmark.

#include <atomic>
#include <new> // Required for hardware_destructive_interference_size

// We force each variable to start on its own dedicated 64-byte boundary.
// They will now load into completely separate CPU cache lines!
struct OptimizedCounters {
    alignas(std::hardware_destructive_interference_size) std::atomic<int> counter1{0};
    alignas(std::hardware_destructive_interference_size) std::atomic<int> counter2{0};
};

* The Desk Metaphor: False sharing is like two accountants (threads) trying to write in two different ledgers (variables) that are glued to the exact same clipboard (cache line). When Accountant A pulls the clipboard to write, Accountant B is forced to stop. By padding the structure, we unglue the ledgers and give them their own clipboards, allowing both accountants to work at max speed simultaneously.