nanoSAE, and the neuron that means five things at once
Open a language model, pick a neuron, and look at what makes it fire. You will not find a tidy answer. The same unit lights up for legal boilerplate, for DNA sequences, and for something about Japanese railway timetables. It is not broken. It is doing what the architecture asks of it.
That property has a name, polysemanticity, and the work of undoing it has
one too: monosemanticity, the goal of finding a representation in which one
direction means one thing. nanoSAE came out of us trying to do that on models
we actually run.
Why a neuron holds several meanings
A model has far more things to represent than it has dimensions to represent them in. Faced with that, it does the sensible thing and packs features into the same space, relying on the fact that most of them are rare and rarely co-occur. The interpretability literature calls this superposition, and it is the reason a single neuron reads as a jumble: you are looking at a sum, and trying to name one of its terms.
The consequence is practical, not philosophical. If you cannot say what a direction in the model stands for, you cannot say why an answer came out the way it did, you cannot tell whether a behaviour you dislike is one feature or twenty, and you certainly cannot steer it without collateral damage.
Sparse autoencoders, briefly
The standard move is to stop looking at neurons and go looking for features instead. Train a small autoencoder on the model's activations, give it more hidden units than the layer it is reading, and force it to use only a handful at a time. The width gives the features room to stop overlapping; the sparsity penalty is what stops the model from simply learning the identity.
What comes out is a dictionary: a long list of directions, each of which fires on something you can usually name. Not perfectly, and not always, but far closer to one-feature-one-direction than the raw layer was.
Why we wrote our own
Sparse autoencoders are not a large idea. The training loop fits on a page, and the interesting work is in everything around it: which activations you collect, how you normalise them, what sparsity you ask for, and how you decide a dictionary is any good.
The existing implementations were built for frontier-scale interpretability work and carry the machinery that implies. We wanted the opposite: something small enough to read in a sitting, fast enough to run a sweep over lunch, and easy to bend when an experiment needs a different loss or a different way of sampling. So we wrote it, the way we wrote okflib, because the problem was ours and the library did not need to be anyone else's.
What it is for, in a client project
This is research, and we keep doing it alongside client work rather than instead of it. It is also the kind of research that pays back quickly. When a model classifies a document the way it does, being able to point at the features that moved is the difference between a system you can correct and one you can only retrain and hope. It is the same argument as lineage: an answer nobody can trace is an answer nobody signs.