Introducing SPMC-Queue - a highly efficient and thread-safe Single Producer Multiple Consumer Queue (SPMC) designed for high-performance applications. Inspired by the enlightening CPPCon 2022 talk Trading at Light Speed, this implementation leverages the power of atomic operations for a lock-free design that significantly reduces latency and contention compared to traditional locking mechanisms.

Why Choose SPMC-Queue?

The SPMC Queue enables multicast message fanning, allowing multiple threads to seamlessly access data produced by a single producer. This makes it an ideal solution for applications where concurrency management is critical. Here are some key benefits of using SPMC-Queue:

• Lock-Free Design: By utilizing atomic operations, SPMC-Queue avoids the pitfalls of locks and mutexes, minimizing costly context switching and improving overall performance.
• Enhanced Cache Performance: The queue is designed to enhance cache utilization, ensuring that related data remains close together, leading to fewer cache misses and faster access times.

Architecture Overview

SPMC Version 1

In its initial version, the SPMC Queue uses two globally shared atomic indices, which synchronize reading and writing operations. However, this comes with increased contention as all threads compete for the same resources.

SPMC Version 2

The second iteration of the SPMC Queue introduces a novel approach by giving each queue element its own version number, significantly boosting performance:

• Reduced Contention: Localized atomic indices minimize competition among threads, allowing concurrent operations on different blocks.
• Better Cache Locality: Threads work on data that is spatially close to its corresponding metadata, enhancing cache efficiency.

struct Block
{
    std::atomic<BlockVersion> mVersion; // Local block versions reduce contention for the queue
    std::atomic<MessageSize> mSize; // Size of the data
    alignas(64) uint8_t mData[64]; // 64 byte buffer
};

Synchronization Logic:

• Initially, all block versions are set to 0 (no reads allowed).
• On writing, if it’s the first write, the data is written, and the version is incremented to indicate that reading is permitted.
• For subsequent rewrites, the version toggles between odd and even to control read access effectively.

Performance Metrics

The SPMC Queue has been rigorously tested against traditional locking mechanisms and other lock-free implementations like boost::lockfree::queue. The results, represented in the histogram below, showcase the SPMC Queue's superior performance.

In conclusion, the SPMC-Queue stands out as a cutting-edge solution for applications requiring efficient data processing with multi-threading capabilities. Explore the code and benchmarking data available in the src folder, and optimize your production workloads with this powerful multi-threaded queue!