Crypto factor search

We let an AI model search crypto futures for an edge, then tested whether anything it found was real.

Sector: Quantitative research · Year: 2026

Summary

An in-house research programme on Binance perpetual futures. An AI model proposed trading signals, a strict statistical test banked the survivors, and 488 made it in. Then we asked the question that matters: how many ideas did we really try, and does anything still hold up on data that did not exist when it was picked? Counted honestly, none of the banked signals held up. Eight signals, named in advance, were then tested on three months of fresh data. None passed.

Challenge

A machine that proposes thousands of trading ideas will always find some that look excellent on past data. The safeguard is the deflated Sharpe ratio, a test that asks how good a result has to be given how many ideas were tried. It is only as honest as that count, and a search that runs for weeks makes the count easy to get wrong. The programme needed an answer it could trust before any money was involved.

Approach

  • Built a test bed from 18 months of hourly Binance USDT-margined perpetual futures data across about 490 coins, with the set of coins re-chosen at each step using only what was known at the time.
  • Had an AI model propose signal formulas in bulk, and banked a candidate only if it cleared a deflated Sharpe test with a floor fixed before the search began. 488 candidates were banked.
  • Found the flaw in our own count. The search ran in rounds, and the tally of ideas tried restarted at every round, so each round faced an easier bar than the whole search deserved. Later rounds were also steered by what earlier rounds had already rejected, so they were not independent. Counted across the whole search, about 1,930 genuinely different ideas had been tried.
  • Re-priced every banked signal at the honest count. The best fell from 0.95 to 0.77 on the deflated scale, against a floor of 0.80, and none of the 81 signals that had passed the strictest gate still cleared it. We stopped the search.
  • Ran a 51-hour paper-trading soak of four of the top signals, pricing every fill from the real order book instead of an assumed fee. Across 114 fills the book finished down 0.016%, the expected result for signals that no longer qualified, and order-book pricing came in 80% below a flat 0.075% assumption.
  • Before looking at any fresh data, wrote down eight signals to test, the best from each of eight families of ideas, and the pass mark: a t-statistic of at least 1.645 over about three months of new hourly data. Picking winners out of 488 afterwards would have repeated the original mistake.

Outcome

None of the eight passed. All eight earned a positive return on the new data, but the best reached a t-statistic of 1.43 against the 1.645 required, and the weakest was close to flat (0.22). That bar needed an annualised Sharpe near 3.3 over three months, so the test was demanding as well as strict. Across all 488 banked signals, 10 cleared the bar, fewer than the roughly 24 that pure chance would produce, and none of the ten had been named in advance. Most of the 488 (420) read the exchange’s funding rate, so the bank was closer to one idea told many ways than to 488 separate ones. The programme stays on research and paper trading. No client capital is involved, and no signal, formula or system design is published.

Stack

Python, pandas, NumPy, SQLite, Parquet

Published papers

  • Counting Honestly — An AI model banked 488 crypto trading signals. Counted properly, none held up. Eight signals named in advance then met three months of new data. 8 pages, PDF.

More work