I reproduced Anthropic's recent interpretability research

Follow the full discussion on Reddit.
Not that many people are paying attention to LLM interpretability research when capabilities research is moving as fast as it currently is, but interpretability is really important and in my opinion, really interesting and exciting! Anthropic has made a lot of breakthroughs in recent months, the biggest one being "Towards Monosemanticity". The basic idea is that they found a way to train a sparse autoencoder to generate interpretable features based on transformer activations. This allows us to look at the activations of a language model during inference, and understand which parts of the model are most responsible for predicting each next token. Something that really stood out to me was that the autoencoders they train to do this are actually very small, and would not require a lot of compute to get working. This gave me the idea to try to replicate the research by training models on my M3 Macbook. After a lot of reading and experimentation, I was able to get pretty strong results! I wrote a more in-depth post about it on my blog here: