Building a Zero-Allocation Network Engine for Tellstone

Introduction

Go is famous for its simple concurrency model using goroutines. When I started building Tellstone, an ultra‑fast, cloud‑native memory store, I quickly realized that the standard net package wasn't going to cut it. While Go's standard net package scales surprisingly well, extremely high connection counts can still introduce scheduler overhead and additional allocation pressure.

Any allocation at the networking boundary cascades down to the database shards, hurting overall throughput. To prevent this, I decided to engineer a custom networking layer from scratch.

By pairing an edge‑triggered epoll event loop using the fantastic panjf2000/gnet library with precise, allocation‑free byte slicing, microbenchmarks on an AMD Ryzen 9 9950X show frame decoding at roughly 1.46 ns/op with 0 B/op and 0 allocations, demonstrating that the decoding path itself remains allocation-free.

The complete, open‑source project is available on GitHub: Saxy/Tellstone.

TL;DR

Table of Contents

The Protocol Blueprint {#the-protocol-blueprint}

To eliminate allocations, I needed a protocol where I know exactly how many bytes to expect. I designed a rigid, highly predictable binary wire‑format layout:

+----------------+----------------+-----------------+
| uint32 length  | uint8 type     | []byte payload  |
+----------------+----------------+-----------------+

Zero-Alloc Decoding via Ring-Buffer Slicing {#zero-alloc-decoding}

The core principle to my zero‑allocation network parsing is straightforward: Never copy memory unless you absolutely have to. Instead of allocating a fresh byte slice for every incoming request, Tellstone taps directly into the live underlying ring‑buffer window provided by the gnet reactor. I parse the frame length inline using manual bit-shifting:

// Required imports for the snippet
import (
    "errors"
)

// Minimal type definitions used in the example
type MessageType uint8

type Message struct {
    Type    MessageType
    Payload []byte
}

var (
    errShortRead = errors.New("short read")
    errZeroLength = errors.New("zero length")
)

func Decode(data []byte, out *Message) (int, error) {
    if len(data) < 5 {
        return 0, errShortRead
    }

    // Inline manual big‑endian decoding for 0‑alloc execution
    length := uint32(data[0])<<24 | uint32(data[1])<<16 | uint32(data[2])<<8 | uint32(data[3])
    if length == 0 {
        return 0, errZeroLength
    }
    if uint32(len(data)) < length+4 { // length does NOT include the 4‑byte header
        return 0, errShortRead
    }

    out.Type = MessageType(data[4])
    payloadLen := int(length) - 1 // subtract the type byte

    if payloadLen > 0 {
        out.Payload = data[5 : 5+payloadLen] // slicing the live window!
    } else {
        out.Payload = nil
    }
    return payloadLen, nil
}

Why this matters for Tellstone: out.Payload references the underlying ring-buffer window exposed by gnet, avoiding additional copies and heap allocations. The resulting slice can be fed directly into the SQL state machine without touching the allocator.

Efficient Writes with Scatter-Gather I/O {#efficient-writes}

Naively issuing separate writes can increase syscall overhead and may result in additional packets being emitted.

To send both memory regions without concatenating them into a newly allocated buffer, I leveraged Go's net.Buffers. This translates the operation directly into a single system‑level writev scatter‑gather syscall:

import (
    "io"
    "net"
)

type MessageType uint8

func Write(w io.Writer, msgType MessageType, payload []byte) error {
    total := 1 + len(payload) // type byte + payload
    var hdr [5]byte
    hdr[0] = byte(total >> 24)
    hdr[1] = byte(total >> 16)
    hdr[2] = byte(total >> 8)
    hdr[3] = byte(total)
    hdr[4] = byte(msgType)

    // net.Buffers creates a slice header on the stack; the buffers themselves are not copied.
    bufs := net.Buffers{hdr[:], payload}
    _, err := bufs.WriteTo(w)
    return err
}

Handshaking with gnet {#handshaking-with-gnet}

To scale across multiple cores with minimal synchronization overhead, I wrapped this protocol logic into an edge‑triggered gnet multi‑reactor architecture.

A critical detail when managing multi‑reactor ring buffers is preventing hot loops. If a packet boundary is hit and a frame is incomplete, I break the execution loop and yield until epoll signals the next read event:

import (
    "errors"
    "github.com/panjf2000/gnet"
)

func (s *Server) OnTraffic(c gnet.Conn) gnet.Action {
    for {
        buf, _ := c.Peek(-1)
        var msg Message

        payloadLen, err := Decode(buf, &msg)
        if err != nil {
            if errors.Is(err, errShortRead) {
                // Incomplete frame – yield back to epoll.
                break
            }
            return gnet.Close // malformed frame
        }

        totalPacketLen := 5 + payloadLen // 4‑byte length header + type + payload

        if s.handler != nil {
            respPayload, respType, err := s.handler(&msg)
            if err != nil {
                return gnet.Close
            }
            if respPayload != nil {
                if err = Write(c, respType, respPayload); err != nil {
                    return gnet.Close
                }
            }
        }

        // Discard the processed bytes from the ring buffer.
        _, _ = c.Discard(totalPacketLen)
    }
    return gnet.None
}

Note: Edge‑triggered epoll only notifies when new data arrives, reducing spurious wake‑ups compared to level‑triggered epoll.

Closing the Loop: The Zero-Alloc Client {#closing-the-loop-the-zero-alloc-client}

A fast server is useless if your client libraries choke on allocations during cross‑service communication. To close the execution pipeline, I built a matching synchronous, low‑latency Go client.

Following Tellstone's GetInto storage pattern, the client requires the caller to pass a pre‑allocated scratchpad buffer for responses. This shifts memory ownership entirely to the application layer, allowing loops to reuse memory over and over:

import (
    "io"
    "net"
)

func (c *Client) Call(msgType MessageType, reqPayload []byte, buf []byte, out *Message) error {
    // 1. Transmit request via the optimized writev pipeline.
    if err := Write(c.conn, msgType, reqPayload); err != nil {
        return err
    }
    // 2. Read response directly into the caller‑provided buffer.
    if err := Read(c.conn, buf, out); err != nil {
        return err
    }
    return nil
}

(A minimal Read helper that mirrors Decode but reads from a net.Conn into the supplied buffer would be defined alongside the other utilities.)

Production Benchmarks {#benchmarking}

I profiled the performance of Tellstone's network package natively on my AMD Ryzen 9 9950X machine (16‑Core / 32‑Threads) running Linux:

goos: linux
goarch: amd64
pkg: github.com/Saxy/Tellstone/internal/network
cpu: AMD Ryzen 9 9950X 16‑Core Processor            

BenchmarkReadMessageZeroAlloc-32         856,039,898        1.464 ns/op        0 B/op        0 allocs/op
BenchmarkGnetServerHandlerParallel-32        522,372         2297 ns/op      149 B/op        6 allocs/op

Disclaimer: The ns/op figures are synthetic micro‑benchmarks that run after aggressive compiler optimizations; real‑world latency will be higher, but the relative zero‑allocation nature remains.

Key Takeaways

Conclusion {#conclusion}

Achieving true high performance in Go requires moving away from implicit object generation and adopting rigorous buffer‑ownership models. Tellstone demonstrates that careful buffer ownership and an event-driven architecture allow Go to deliver extremely competitive networking performance while retaining Go's development ergonomics.