Noise-floor measurement

Our own programme — we measured what our own research produces when there is nothing to find

Sector: Quantitative research · Year: 2026

Summary

A systematic search for short-horizon directional signal in the mechanics of a futures market. It found nothing tradeable, which was the less interesting half of the result. The useful half is that we measured, rather than assumed, how convincing a search of that size looks when it is run on data with the answer deliberately removed — and every result we had found sat inside that range.

Challenge

The hardest problem in this work is not building the signal. It is that a search over hundreds of candidate rules will always return a best one, and the best of hundreds of coincidences looks indistinguishable from a discovery. The standard response is to apply a textbook correction and proceed. That treats the size of the search as a nuisance to be adjusted for, when it is actually a measurable property of the specific dataset, the specific feature set and the specific grid — and it differs enormously between them.

Approach

  • Recorded a week of one instrument at full depth — 5.7 million trades and 5.8 million order-book snapshots — and computed the standard microstructure indicators from the literature on a volume-time decision clock, with backward-only joins so no figure can carry information published after the moment it is stamped.
  • Built the null by rotating the forward-return block against the feature block: every distribution, every autocorrelation and every overlap relationship preserved exactly, and only the correspondence between signal and outcome destroyed. Rotation was chosen over a block bootstrap specifically because it introduces no free parameter, in a study about the cost of free parameters.
  • Re-ran the entire 854-cell search inside each of a thousand scrambled worlds and kept only the extremes, so the comparison is like for like — a distribution of maxima against an observed value that was itself chosen as a maximum.
  • Corrected the statistics for overlapping holding periods before comparing anything, collapsing entries that describe the same forward window into one observation. A headline t-statistic of 5.16 turned out to be 194 restatements of 27 independent events.
  • Scored every result against five separate cost floors rather than one, because the venue’s own published fee was inconsistently reported and we would rather a conclusion not depend on which source was right.
  • Ran the same null against our own most promising finding, and withdrew it when it failed.

Outcome

In worlds where prediction was impossible by construction, the best of 854 rules still returned a median overlap-corrected t of 3.405, reaching 5.955 at the 95th percentile. Every real result was inside that distribution — the strongest at the 88th percentile of pure noise. That threshold is now the standing floor any future study here is checked against, and it is worth more than the strategy it retired. Three supporting measurements came out of the same work: the price path is a random walk in volume time to within 8% across a fortyfold range of horizons; the round-trip cost of a fast trade exceeds the entire average five-minute move, making one whole class of strategy impossible by arithmetic rather than by evidence; and one well-known indicator that appeared to predict volatility in perfect monotone order was withdrawn when the null reproduced that ordering 14% of the time. No capital was committed at any stage, and roughly 24 days of recorded tape remain sealed — not for want of ideas, but because no specification survived its own null, so there is nothing worth confirming out of sample.

Stack

Python, NumPy, pandas, Apache Arrow, Parquet, WeasyPrint

Published papers

  • Measuring the Noise Floor — What a systematic search returns when there is nothing there — and why we published the number instead of the failure. 9 pages, PDF.
  • The Same Knob Twice — How a parameter repair and a threshold change turned out to be the same action, and the test that told them apart. 7 pages, PDF.
  • Impossible Before We Started — The cost floor, the accuracy it implies and the latency budget — three constraints that can be settled with arithmetic before any research begins. 8 pages, PDF.

More work