12.3 Multithreading, Lock-Free & Concurrency

Let's look at concurrency on the hot path. Lock-free code sounds heroic. Sharing less often wins harder.

A mutex can be the correct tool when contention is low and the code is not latency critical.

Lock-free code can be slower or less safe when it introduces:

• Spinning under contention
• Excessive atomic operations
• False sharing
• Complex memory reclamation
• ABA problems
• Weak-memory-order bugs
• Poor observability

For many HFT pipelines, the best architecture minimizes sharing instead of making sharing more clever. Clever atomics are not a substitute for clear ownership.

A common topology is single-producer, single-consumer:

Each stage owns its mutable state. Communication happens through bounded queues.

This gives you:

• Clear ownership
• Fewer locks
• More predictable synchronization
• Easier testing
• Better cache locality

A bounded SPSC queue is a useful building block when exactly one producer thread and one consumer thread access it.

#include <array>
#include <atomic>
#include <cstddef>
#include <type_traits>

template <typename T, std::size_t Capacity>
class SpscRingBuffer {
    static_assert(Capacity > 1);
    static_assert((Capacity & (Capacity - 1)) == 0,
                  "Capacity must be a power of two.");
    static_assert(std::is_trivially_copyable_v<T>);

public:
    bool try_push(const T& value) noexcept {
        const auto write = write_index_.load(std::memory_order_relaxed);
        const auto next = (write + 1) & mask_;

        if (next == read_index_.load(std::memory_order_acquire)) {
            return false;
        }

        buffer_[write] = value;

        write_index_.store(next, std::memory_order_release);
        return true;
    }

    bool try_pop(T& output) noexcept {
        const auto read = read_index_.load(std::memory_order_relaxed);

        if (read == write_index_.load(std::memory_order_acquire)) {
            return false;
        }

        output = buffer_[read];

        read_index_.store((read + 1) & mask_, std::memory_order_release);
        return true;
    }

private:
    static constexpr std::size_t mask_ = Capacity - 1;

    alignas(64) std::array<T, Capacity> buffer_{};

    alignas(64) std::atomic<std::size_t> write_index_{0};
    alignas(64) std::atomic<std::size_t> read_index_{0};
};

The basic synchronization idea is:

The producer's release store makes earlier writes visible before the consumer sees the updated write index. The consumer's acquire load completes that handshake.

Important limitations

This queue is not a multi-producer, multi-consumer queue.

It assumes:

• Exactly one producer
• Exactly one consumer
• Capacity is a power of two
• T is trivially copyable
• Queue-full behavior is handled by the caller
• The producer does not overwrite unread elements
• The consumer does not read unpublished elements

One more detail: the alignas(64) on write_index_ and read_index_ is the false-sharing fix that matters. Aligning the whole buffer_ array does not by itself solve producer/consumer ownership contention on the indexes.

In real systems, benchmark and validate the queue under the target architecture and compiler. Correctness comes before nanoseconds.

False sharing is two neighbors slamming the same hallway door. The variables are different. The cache line is not.

For example:

struct BadLayout {
    std::atomic<std::uint64_t> producer_counter{0};
    std::atomic<std::uint64_t> consumer_counter{0};
};

If producer and consumer run on separate cores, each update can bounce cache-line ownership even though they touch different variables.

Separate hot, independently written data:

struct BetterLayout {
    alignas(64) std::atomic<std::uint64_t> producer_counter{0};
    alignas(64) std::atomic<std::uint64_t> consumer_counter{0};
};

The cache-line size is often $64$ bytes on modern x86 systems, but this is hardware-dependent. C++ provides:

std::hardware_destructive_interference_size

Where supported, it can express intent more clearly:

#include <new>

struct Counters {
    alignas(std::hardware_destructive_interference_size)
    std::atomic<std::uint64_t> producer{0};

    alignas(std::hardware_destructive_interference_size)
    std::atomic<std::uint64_t> consumer{0};
};

std::hardware_destructive_interference_size is not guaranteed to be usable on every target and toolchain. Prefer it when available, and keep a platform-configured cache-line constant (often $64$) as a fallback for portable builds.

Measure the effect. Padding raises memory use and can worsen cache utilization if applied indiscriminately.

This is the Blank Flag Hazard: it looks like a signal between threads, and it is not:

volatile bool ready{false};

volatile tells the compiler that reads and writes are observable side effects. It does not give you:

• Atomicity
• Mutual exclusion
• Memory ordering
• A happens-before relationship
• Correct publication between threads

Use atomics:

std::atomic<bool> ready{false};

Producer:

data = compute_data();
ready.store(true, std::memory_order_release);

Consumer:

if (ready.load(std::memory_order_acquire)) {
    use(data);
}

This example is intentionally tiny. Real shared-state designs often need more than one flag and a carefully specified protocol.

volatile can still matter for memory-mapped I/O or specialized hardware interactions, but it is not a substitute for std::atomic.

The C++ memory model provides several orderings. The most common for message passing are:

For an SPSC publication pattern:

producer_data[index] = event;
published_index.store(next_index, std::memory_order_release);

And:

const auto published =
    published_index.load(std::memory_order_acquire);

const auto event = producer_data[index];

Using relaxed everywhere may work on one CPU architecture and fail on another. Using seq_cst everywhere may be correct but can impose unnecessary constraints.

The goal is not to collect weak memory orderings like trading cards. Use a clearly correct ordering, then measure only if the synchronization cost actually matters.

Minimize sharing first. Use SPSC queues when they fit, pad hot counters apart, never use volatile as a lock, and treat memory ordering as a correctness tool before a speed hobby.

Practice: draw the producer/consumer ownership boundary for a three-stage pipeline and mark which indexes need release/acquire.