The inference-cost cliff — what's investable
The floor under token prices is falling faster than the models are getting bigger. The margin is leaving who trains and moving to who serves — and the market hasn't repriced it yet.
The story the tape keeps telling is about training — bigger clusters, bigger raises, bigger names. That story is priced. The one underneath it is about serving, and it is moving in the opposite direction: the marginal cost of a token has fallen roughly two-thirds in a year, and it is still falling.
That collapse is not bad news for everyone. It is a transfer. When inference gets cheap, the value doesn't evaporate — it relocates to whoever owns the workload, the distribution, and the last mile of latency. The setup is a classic picks-and-shovels rotation, except the shovels are getting cheaper and the ground is getting bigger at the same time.