The Trout's Question: Backpressure as an Upstream-Swimming Problem
A young Viktor Schauberger noticed that trout holding station in a fast Alpine stream did not appear to be swimming. They were doing something else — using the structure of the water around them to remain stationary. Ninety years later, the design instinct survives intact: the right way to think about an async pipeline under bursty load is not as a queue to be drained but as a flow to be shaped, and the inference stack the team builds around betterFANN is the artefact in which we think about the problem most carefully.
The standard textbook description of stream backpressure is some variant of: the consumer signals to the producer how many items it is ready to receive, and the producer respects that signal. This is not wrong, but it is a description of the protocol, not of the underlying physics. The interesting question is what the producer does with the time during which it is not allowed to produce, and what the consumer does with the time during which it has not yet been asked to consume. The answer chosen by most runtimes is: nothing. They idle. We think the underlying physics rewards a richer answer.
What Schauberger Saw
From the manuscript notes later compiled by Callum Coats as The Water Wizard:
The trout, hanging motionless in the violent current, demonstrated to me by its very passivity that it was not the fish that was working but the water around it. The fish had organized the flow such that the flow was moving past it in the form that the fish required, and the fish merely had to occupy that form.Schauberger, manuscript notes c. 1929, in The Water Wizard
Schauberger's claim — which his contemporaries dismissed as romantic but which subsequent fluid-dynamics work on trout swimming (Liao, Phil. Trans. R. Soc. B, 2007) has substantially confirmed — was that the trout exploits the small-scale vortex shedding behind upstream rocks to convert a hostile flow into a hospitable one. The fish does not fight the current. It rearranges the part of the current it occupies until the current is no longer fighting it. This requires positioning, not exertion.
The Async Pipeline Analogy
Consider an event-processing pipeline of the form Producer → Stage₁ → Stage₂ → … → Stageₙ → Sink. The producer emits events at a rate λ. The slowest stage in the pipeline — call it Stageₖ — has a service rate μₖ < λ. Standard backpressure says: the sink signals upstream, and through k hops of token propagation, the producer's emission rate is throttled to μₖ.
This works. It is also catastrophic in a specific failure mode that production systems hit constantly: when the producer is bursty. A producer that emits 10λ for one second and 0 for nine seconds will be permitted, on average, to emit at μₖ — but the burst will be either dropped at the producer (data loss), buffered to disk (latency spike), or buffered in memory (out-of-memory crash). All three are first-derivative failures of the standard model.
The Schauberger reading: the burst is the fast water. The pipeline is the trout. The question is not how do we make the trout work harder but how do we shape the flow so the burst flows past the slow stage rather than into it.
Three Schauberger-Derived Primitives
Three primitives that the standard reactive-streams protocol does not include but that the trout observation suggests are worth exploring — and that the team has been working through inside the inference pipeline that sits in front of betterFANN's typed-tensor SIMD-AVX2 backend:
- Vortex buffering at upstream stages. When Stageₖ signals saturation, the upstream stages do not immediately signal further upstream. Instead, each upstream stage uses the time it has been granted (the difference between its own service rate and Stageₖ's service rate) to pre-process the next batch — sorting it, deduplicating it, or transforming it into a form the slow stage can ingest in larger units. The burst is not fought; it is reshaped en route.
- Multi-channel laminar splitting. Where the workload permits, the runtime splits the flow into parallel laminae upstream of Stageₖ and merges them downstream. The split is not a load balancer; it is a deliberate decision to expose Stageₖ to a flow whose statistical properties (mean, variance, autocorrelation) are easier to service than the original flow. This corresponds to Schauberger's observation that a flow split into multiple sub-streams and recombined downstream often does work that the unsplit flow could not do.
- Eddy reuse. Where Stageₖ rejects an item — because it is malformed, duplicate, or below a quality threshold — the item is not dropped. It is returned to a small upstream eddy buffer where a parallel low-priority worker can repair, deduplicate, or score it. The repaired items rejoin the main flow with a lower priority, and the rate of irrecoverable losses falls in proportion to the eddy capacity.
Why the Pattern Matters for Inference
The reason this matters specifically for an inference pipeline is that the slow stage is almost always the model itself, and the cost of an out-of-memory event on the model tier is the cost of failing all in-flight requests, not the cost of failing one of them. A standard backpressure topology in front of an inference engine treats the burst as a problem to absorb at the queue. The Schauberger reading treats the burst as a resource to occupy, and the eddy buffer becomes the place where a request can be deduplicated against the in-flight set, normalised to the model's tokenizer, scored for priority, or even partially evaluated against a smaller local-LLM tier — before it ever reaches the slow stage. The Akka pipeline pattern is not badly designed. It is designed correctly under the standard model, which treats the burst as a problem to absorb. The trout treats the burst as a resource to occupy.
The Schauberger Lineage
The geometric observation we borrow is precise: a fish-shaped object held in a complex flow does work that an inert object of the same drag profile does not. The trout's body shape, fin angle, and station-keeping behaviour collectively constitute a transducer that converts disorder upstream into order downstream — a phenomenon now well-documented in the modern hydrodynamics literature on fish swimming in altered flows. Alick Bartholomew's Hidden Nature and Olof Alexandersson's Living Water together provide the most reliable English-language chronicle of the manuscript record from which the trout observation is drawn.
An async pipeline stage can do the same kind of work the trout does. Most do not. The published implementation work continues in the open at github.com/VRIL-LABS/betterFANN; the design vocabulary it borrows from is the one this essay describes.