12.2 Memory Control & Zero-Allocation Hot Paths

Let's look at memory on the hot path. In earlier chapters we treated the heap like a cafeteria tray system that can get expensive. Here the rule gets stricter: on the runway, do not stop mid-race to build a new lane.

Heap allocation can look fine in isolation. In a per-message loop it can inject jitter from:

• Allocator locks or contention
• Thread-local allocator refills
• Cache misses
• Memory fragmentation
• Page faults
• Deallocation work
• Allocator implementation changes

Avoid this in a per-message loop:

auto event = std::make_unique<MarketEvent>();

Prefer automatic storage when you can:

MarketEvent event{};

For bounded object lifetimes, use fixed-capacity pools or arenas.

template <typename T, std::size_t Capacity>
class ObjectPool {
public:
    T* acquire() noexcept {
        if (next_ == Capacity) {
            return nullptr;
        }

        return &storage_[next_++];
    }

    void reset() noexcept {
        next_ = 0;
    }

private:
    // Put the hot scalar before the large array so next_ and storage_[0]
    // tend to share early cache use instead of straddling the array.
    std::size_t next_{0};
    std::array<T, Capacity> storage_{};
};

Member order is part of the layout story: you always read next_ before touching storage_[0], so keep them close. This simple pool is appropriate only when:

• Objects do not need arbitrary individual deallocation
• The maximum capacity is known
• Reset timing is safe
• Object lifetime requirements are simple

Fancier pools need careful ownership and reuse rules. Reusing memory while another thread still reads it is a data race with a fancy resume.

Standard-library containers are useful. The issue is uncontrolled growth in latency-sensitive paths.

This can allocate:

std::vector<MarketEvent> events;

events.push_back(event);

Avoid surprise reallocations by reserving capacity during initialization:

std::vector<MarketEvent> events;
events.reserve(65'536);

But reserve is not a lifetime warranty. Grow past the reserved capacity and the vector may allocate again. That is the Reallocation Pitfall.

For strict bounded behavior, use fixed-capacity containers or enforce an explicit capacity check:

template <typename T, std::size_t Capacity>
class FixedVector {
public:
    bool push_back(const T& value) noexcept {
        if (size_ == Capacity) {
            return false;
        }

        data_[size_++] = value;
        return true;
    }

    std::size_t size() const noexcept {
        return size_;
    }

private:
    // Same layout rule as ObjectPool: hot size_ before the payload array.
    std::size_t size_{0};
    std::array<T, Capacity> data_{};
};

Now the problem shifts from “unexpected allocation” to “what happens when capacity is exhausted?” That behavior must be explicit:

• Reject the event
• Trigger recovery
• Drop noncritical telemetry
• Stop quoting
• Raise an operational alert
• Fail closed for risk-sensitive operations

Exceptions are valuable in many C++ applications. In a latency-critical event loop they are often avoided because the exceptional path is hard to bound and unwinding is expensive.

Use explicit error results:

enum class DecodeError {
    none,
    truncated,
    invalid_type,
    invalid_length,
    invalid_value
};

struct DecodeResult {
    DecodeError error{DecodeError::none};

    bool ok() const noexcept {
        return error == DecodeError::none;
    }
};

For functions expected not to throw:

void process_market_event(const MarketEvent& event) noexcept;

Be precise with noexcept. If a function marked noexcept throws, the program calls std::terminate(). Use it when termination is the intended response or when you can prove the function cannot throw.

A common pattern is:

• Hot path: explicit error codes or status types
• Initialization and configuration: exceptions may be acceptable
• Administrative services: normal application error handling
• Fatal invariant violations: controlled fail-safe behavior and alerts

Preallocate, bound capacity, and keep exceptions off the runway. When something is full, decide the policy before the market decides it for you.

Practice: rewrite a tiny event buffer once with std::vector::reserve and once with FixedVector, then write down what happens at capacity for each.