Why should a drop decision for redundant packets stay in the kernel instead of escalating every retry to user space, and how should a busy BPF ring buffer be sized and watched?

When a socket layer is receiving thousands of redundant transport retries per second, the first question is where the "is this packet a duplicate?" decision belongs. The drop of an already-seen transport frame is a deterministic, bounded-state lookup, and it belongs in the kernel: escalating every redundant retry to user space pays a context switch, a wake, and scheduler jitter on the very packets you are trying to filter out, which is the exact cost the filter exists to avoid. The alternative that actually works is a small in-kernel dedup table plus a single ring-buffer export path, and the whole design reduces to two numbers you must watch.

The mechanism

The cost people misjudge is the verifier. A eBPF program is checked once at load time: the verifier walks the program, proves it terminates and touches only bounds-checked memory, and that work is a fixed load-time complexity budget. The BPFConf 2025 talk on BPF's one-milion-instruction limit frames it that way: the limit is on verifier complexity at load time, not on per-packet work, and the budget is spent on unrolled loops and always-inlined helpers that inflate the program shape. A program that fits under that limit costs the same to verify whether it runs ten times a day or ten million times a second. So the runtime cost of the in-kernel dedup path is not the program shape; it is a handful of map operations.

That matters because the only bounded-state dedup you can run in-kernel is a map. A socket filter that drops a duplicate keeps a small table of in-flight frame identifiers, does a lookup on arrival, increments a counter, and drops when the key is present. Every one of those operations is a map access that runs in the same context as the packet path, with no process wake between them. The decision stays deterministic and bounded, and its cost is a constant number of map operations per packet.

The export path is a ring buffer. The kernel ring-buffer implementation is a power-of-2 shared buffer with multiple producers and one consumer, and its data area is memory-mappable so user space reads samples without a second copy. A producer reserves a fixed-size record; the reservation needs a size that is a compile-time constant so the verifier can prove the program cannot read outside the record. When the ring has no space left, the reservation fails and returns NULL without blocking: the busy producer must count the miss and continue, so a full ring degrades observability rather than stalling the datapath. On the consumer side, a single query helper reports the amount of unconsumed data and the current producer and consumer positions, which is the watermark you watch for backpressure. The consumer can also choose how the producer wakes it, from no wakeups to a forced one, so a busy ring can batch notifications instead of waking on every record. Each reserved record also carries a small header with the record length and a busy bit, and in NMI context the reservation can fail even when the ring is not full, because it takes a spin lock that an interrupt may not acquire. A record that does not fit a fixed size has to use the copy-based output helper instead of the reserve path, since the reserve only accepts a constant size.

Verification and debugging path

1. Count drops on the BPF side. Keep a per-CPU dropped counter next to the dedup table and bump it every time a ring-buffer reservation returns NULL. A climbing counter means the ring is smaller than the sample rate; a flat counter with steady traffic means the ring is keeping up.
2. Watch the watermark on the user side. The same query helper that reports the amount of unconsumed data also reports the producer and consumer positions. The gap between those two is your live backpressure gauge: a gap that grows toward the ring size is the drop counter about to climb.
3. Find the steady state. Run at full retry rate, let the counters settle, and read the dropped counter next to the available-data watermark. When both stop changing while traffic continues, you have an equilibrium ring size; raise max_entries and re-test if the counter still climbs.
4. Measure the trade-off you are deciding. Timestamp a sample at the point of drop in the kernel and at the point it is processed in user space, then compare the two distributions. The difference is the context-switch and wake cost of escalating every retry to user space; that gap is the number that justifies keeping the drop in the kernel.

The limitation

• In-kernel dedup state is bounded by the map's entry limit. A full hash map rejects new inserts while updates to existing keys still succeed, so the dedup table can silently stop tracking new in-flight frames and keep dropping the old ones: check the map's full condition before blaming the ring buffer.
• A ring-buffer drop is silent unless you count it. The reservation failing when the ring is full is the designed behavior, not an error; without the per-CPU dropped counter, a busy ring simply produces no sample and you have no signal that observability has degraded.
• The reserve path needs a constant record size. If your samples are variable length, the fixed-size reservation will not take them; use the copy-based output helper, which accepts a runtime length at the cost of an extra copy.
• Wakeups are a policy you set, not a property of the ring. Leaving the consumer at its default wake behavior under heavy load means a notification per record; use the no-wake and force-wake flags to batch, or the consumer burns CPU on wakeups rather than on the samples.

References

• Linux kernel documentation: BPF ring buffer — power-of-2 shared ring, bpf_ringbuf_reserve()'s constant-size requirement and its non-blocking failure when full, the BPF_RB_AVAIL_DATA, BPF_RB_RING_SIZE, BPF_RB_CONS_POS, and BPF_RB_PROD_POS query flags, the BPF_RB_NO_WAKEUP and BPF_RB_FORCE_WAKEUP wake control, the per-record header, and the NMI-context reservation caveat.
• libbpf ring-buffer API — the user-side libbpf_ring_buffer_query() and the BPF_RB_AVAIL_DATA, BPF_RB_RING_SIZE, BPF_RB_CONS_POS, and BPF_RB_PROD_POS flag values the consumer reads.
• BPFConf 2025: Beyond 1M BPF instructions — frames the one-milion-instruction verifier complexity budget as a load-time limit spent on unrolled loops and always-inlined helpers.

Community discussion today

The selected question came from an opt-in archive: a thread asking how to benchmark kernel-space drop latency against user-space context-switch cost under thousands of concurrent socket-layer retries per second, with a heavily loaded ring buffer behind it. The thread was open-ended: it asked for the boundary where a drop stays in the kernel versus where it escalates to user space, and how to run and size the ring buffer under that load, without a public primary source or a decisive boundary attached. This page anchors that boundary to the public kernel ring-buffer and verifier documentation: a deterministic, bounded in-kernel dedup decision stays in the kernel, and the ring buffer is sized against a watched watermark plus a counted drop.

Other threads this day: a re-post of the OBI Kubernetes cache address environment variable being ignored under a Config v2 document, now with a helm-charts pull request and an in-thread confirmation that it is a real issue plus a manual workaround — already published, so not republished. A pull request recording the Agent Skills available to an agent at invocation start so a trace can tell "never offered" from "offered but a weaker match won" — also already published.

Channel coverage for this run: the two opt-in archives provided eight messages, all covered above. The visible-browser-only sources (Discord, the eunomia-bpf and sched-ext communities, the bpf mailing list, and r/eBPF) could not be reviewed in this run, since no visible-browser session was available, so they are marked uncovered, not quiet.