Is the number real?

The code ranked assets and bought the best ones. When two assets tied, something had to break the tie — and the program broke ties differently each time it was started. Different assets went into the portfolio on different runs.

Ties weren't some rare edge case. About 55% of this exchange's funding payments sit at exactly the same floor value, so any two assets resting on that floor looked identical to the ranking code.

In the periods where the runs disagreed, the typical gap was 1.57 percentage points. That isn't rounding. That's a different portfolio.

That single day fed into thirty separate monthly averages, and produced a result of +192.66% a year. The pool held two of the largest and most closely linked crypto assets, where a return like that isn't merely unlikely — it can't happen.

Every safety check passed. The checks all looked at the averages, and the bad number got in before the averages were built. On screen it looked like an exciting outlier, not like broken data.

The research already contained the right test. It had been run — on two pools, on one day. That's a spot check. Run across everything, it catches the bad day immediately.

The effect was backwards on its face: spreading the same pot of money across more assets made it look more expensive to trade. It can't. Splitting a fixed amount across more positions makes each position smaller. We checked the arithmetic across all 590 periods of three sets of records.

Corrected, the strategies looked transformed. On the standard risk-adjusted score — where below zero means it lost money — one went from −8.73 to +0.34, another from −2.83 to +1.39.

All three stayed shut. Those flattering new numbers came from the very data the strategies were built on, and the margin of error was wide enough to include zero — so the improvement might be nothing at all. The only honest test was data the strategy had never seen. Against that, the best of the three failed outright, losing money both before and after the stretch it was built on. It was over-fitted, not worn out.

A data migration quietly froze 65 files — they simply stopped updating. Because the filter counted rows instead of reading dates, those frozen assets stayed eligible for about 26 days. The strategy's holdings changed completely from one week to the next, which read like the strategy responding to the market. It was the data breaking underneath it.

Then a second freeze was found inside the restarted clock: three more assets stopped updating on the same day, and one was bought into a portfolio a full week after it had already gone dead.

Underneath that sat the worst of it. One position the strategy held had not traded at all — zero out of 1,441 minutes with a single trade in them. A full-size position, booked at a profit, describing a trade nobody on earth could have made.

The price on your screen is the price for a tiny trade. Put a real order through and you eat into worse and worse prices as you go. On this basket the screen price suggested a cost of 0.004%–0.033%, while getting in and back out actually cost 0.28%–0.78%. A measurement taken at the screen price cannot see that, at all.

So we went back and audited every cost assumption in the research. It found this:

A standing study had assumed the worst possible round trip on that exchange was 0.31%. Filling real orders, 40 of 122 assets cost more than that at $10,000 apiece — the worst of them 0.92%. And those were the easiest-to-trade assets available.

That study's own follow-up note had already tried to fix it with a textbook formula — and overshot by about 1.8× the other way, exactly as the note itself suspected it might. Measured properly, the truth sat between the two. The original conclusion survived at the size it was written for, and failed at ten times that size.

The strategy is event-driven and holds a fixed maximum number of positions. On the worst days far more signals arrive at once than there are slots — the busiest single hour had 194. Something has to choose which ones get in, and that choice was being made by the arbitrary order the sorting step happened to leave the data in. 72% of all events shared their hour with at least one other.

Two of our own programs, same data, same settings, produced annual returns 1.4 percentage points apart. Both were running correctly. So we re-ran it 300 times, shuffling only within each tied hour: the standard deviation of the answer was 0.76 points, and the best ordering beat the worst by 4.2.

Worse than the number itself: a cross-check had already been run and had passed. One study cited the other's agreement as proof its simulator reproduced the original. The two figures were 1.4 points apart, and the gap was hidden inside a rounding in the summary. A verification step read as independent confirmation when it was two programs sharing one defect.

Measuring how confident you are entitled to be usually means reshuffling the daily results thousands of times and seeing how wide the outcomes spread. That assumes each day is independent of the last. This rule held the same position for days at a time, so consecutive days moved together — and treating them as independent counts the same evidence over and over, making the range far too narrow.

Reshuffling in blocks of ten days instead, which keeps that clumping intact, every one of the nine configurations had a range that included zero. Splitting the history in half pointed the same way: performance fell in every version, one from a risk-adjusted score of 0.88 to 0.09.

It is worth saying what the problem was not. Trading costs barely mattered: the score went 0.98 before fees, 0.95 at normal fees, 0.85 at four times normal fees. This failed on uncertainty, not on costs — which is exactly the wrong place to have been looking.

The strategy only fires during rare market-wide episodes, so its trades arrive in a small number of clusters — and every trade inside a cluster is the same event happening to different assets. Counting distinct weeks instead of trades, the strongest result in the study rests on six weeks in six and a half years. About one independent observation a year.

The trap that sets is not theoretical. The obvious improvement — sit out the small isolated events, save room for the broad ones — tested better on data held back for the purpose (0.88 → 1.46) and worse on the data it was built with (1.16 → 0.70). A rule that improves on unseen data is the opposite of the usual warning sign and reads like a discovery. Its entire advantage rested on two crash weeks. We did not adopt it.

The strategy places a ladder of buy orders below the price and sells above. Each completed round trip books a small gain — but when the price falls and keeps falling, the buys fill and the matching sells never do, so the position sits there losing. At the end of a test period, what do you do with the lots still open? Value them at the current price, or leave them out because they never closed?

We ran all eight configurations both ways across 20,032 test windows. The accounting choice flips the sign in every configuration. The gap tracks exactly what you would expect: the configurations holding the most unsold lots show the widest gap, one going from +2.4% counting only closed trades to −5.1% counting everything.

One note on how we treated our own claim. The original study made this point in a single line and asserted the counterfactual without ever computing it — the code tracked both figures and saved only their sum. We measured it before publishing this, and the assertion turned out to be conservative: it said “most configurations”, the answer is all of them. Being right by luck is not the same as having checked.