Scaling Stateful Network Services on Multicore Architectures

Abstract

In recent years, network speeds have surged while CPU speeds have plateaued, making kernel-based networking stacks increasingly impractical, particularly for multicore servers in datacenters. Consequently, kernel-bypass networking stacks and multicore applications have become necessary to keep up with faster datacenter networks. A large number of applications still require TCP for interoperability with legacy applications. However, TCP processing is resource-intensive compared to microsecond-scale applications, and its per-connection state complicates scheduling across multicore architectures. The interaction between network stack and application scheduling, whether on shared or different workers (e.g., cores), strongly impacts performance by affecting cache usage, CPU pipeline efficiency, and memory access patterns. Given the demands of high-speed networks, optimizing these architectures is essential, yet many existing systems fail to address the performance of TCP and its interactions with the application under real-world conditions.

One of the goals of this thesis is to investigate the effective scheduling of a TCP stack alongside applications on multicore architectures, emphasizing the trade-offs involved in allocating workers for both TCP and application processing. This thesis takes a principled look at the relationship between a stateful network protocol with strong guarantees for scheduling such a stack with multicore applications. To allow fair comparisons, we design and implement Demieagle, a benchmark framework that includes (i) a flexible, kernel-bypassing, microsecond-scale TCP stack that schedules network processing and application requests with different multicore architectures and (ii) a benchmark suite with workloads across a range of characteristics that stress different trade-offs in multicore scheduling. Demieagle allows the execution of "apples-to-apples" experiments to uncover the trade-offs of different multicore scheduling policies and architectures; thus, guiding application programmers toward an ideal scheduling policy for their workload.

In this thesis, we also address the complexity of scaling network functions. Network Function Virtualization (NFV) promises better utilization of computational resources by dynamically scaling resources on demand. However, most network functions are stateful and require per-packet state updates. During a scaling operation, workers need to synchronize access to a shared state to avoid race conditions and to guarantee that network functions process packets in arrival order. Unfortunately, the classic approach to control concurrent access to a shared state with locks does not scale to today's throughput and latency requirements. To address these challenges, we design, implement, and evaluate Dyssect, a system that enables dynamic scaling of stateful network functions by disaggregating their states. By carefully coordinating actions between workers and a central controller, Dyssect migrates shards and flows between workers for load balancing or traffic prioritization without using locks or reordering packets. Also, Dyssect's state disaggregation allows the offloading of stateful network functions to programmable NICs and makes it easier to explore hardware-software trade-offs that better suit specific network functions and traffic loads. Our experimental evaluation shows that Dyssect reduces tail latency up to 32.04% and increases throughput up to 19.36% compared to state-of-the-art competing solutions.