The idea has real intuitive pull. A network that is too small for its task must, somewhere inside, be mapping inputs it should distinguish onto the same internal representation. Those collisions are measurable. So: train a seed network, find the layer where irrecoverable collisions concentrate, grow new capacity exactly there, initialised specifically to separate the colliding pairs, and repeat. Capacity delivered like medicine, to the site of the injury, instead of scattered uniformly the way a wider architecture scatters it.
This is the story of that idea surviving one experiment in a way that should already have worried us, and then dying completely in the confirmation run. It is the cleanest illustration in this ledger of why fresh seeds exist.
Act one: the encouraging pilot
Five seeds, a ~6.5k-parameter seed network on CIFAR-10, three growth arms (random growth, gradient-guided growth in the GradMax style, and collision-guided growth) plus a dense-from-scratch control at matched final size.
| arm | accuracy | params |
|---|---|---|
| seed only (floor) | 67.75 ± 0.44 | 6,474 |
| grow random | 71.31 ± 0.77 | ~11,136 |
| grow gradient-guided | 71.37 ± 0.52 | ~12,785 |
| grow collision-guided | 72.23 ± 0.53 | 9,606 |
| dense from scratch | 73.22 ± 0.45 | 11,286 |
v1, mean ± std over 5 seeds. Collision-guided growth was the best growth arm with 14–25% fewer parameters.
The preregistered gate required paired t > 2.24 against both baselines. Against gradient-guided: t = 3.76, all five seeds, pass. Against random growth: t = 2.19. Fail, by 0.05 of a t-statistic. We recorded it as a fail, because the alternative, rounding a preregistered threshold in your own favor, is how ledgers rot. But the ablations looked promising: decomposing placement from initialisation showed the collision-trained filter itself carrying +0.60 points at t = 3.19, the largest and most reliable effect among the growth arms. A near-miss with a mechanistically clean story. Any reasonable person runs the confirmation.
Act two: the confirmation
| comparison | v1 (5 seeds) | v2 (10 fresh) |
|---|---|---|
| filter effect | +0.60pp, t=3.19 | +0.34pp, t=1.61 · FAIL |
| growth vs dense, matched arch | — | −1.91pp, t=6.84, 10/10 · FAIL |
Both preregistered gates failed. The pilot effect did not confirm; growth lost to scratch training in every single seed.
Both gates failed, and the second failed in the worst possible way for the idea: at identical architecture and parameter count, a network trained from scratch beat the grown network by 1.9 points in 10 of 10 seeds. Growth wasn't merely failing to beat a fair baseline. The growth process itself was leaving accuracy behind.
The placement signal turned out to be worse than useless. Given a free choice, the collision metric chose the deepest layer in 163 of 180 growth events, and following that choice cost 3.2 points (t = −16) compared to just forcing all growth into layer 0. Collisions really do concentrate deep, the measurement was correct, and acting on it was harmful: deep growth builds architectures that are bad even when you train them from scratch. The signal was pointing at where the error is visible, which is not where capacity helps.
What this bought us
Respect for a 0.05 margin. If we had rounded t = 2.19 up to a pass in act one, the ledger would today contain a false positive with a plausible mechanism attached, which is the most dangerous kind, and the confirmation run that killed it might never have been scheduled. The preregistered threshold did its job twice: once by refusing the claim, and once by forcing the run that settled it.