I'm a research engineer working on language model post-training. Most of my time goes into reinforcement learning for LLMs on non-verifiable tasks and into interpretability: understanding what training actually changes inside a model, not just what it changes on a benchmark.

Most recently, I was a core contributor to Pipsqueak 3, Character.AI's flagship and highest-traffic model. I developed the main techniques used in its training, including new approaches for cross-tokenizer distillation and an interpretability-inspired method that decomposes a reward model into hand-picked, aligned features for RL.

Research interests

• RL for language models: reward modeling, policy optimization, and what goes wrong in between
• Preference learning: new methods such as distributional preference learning
• Mechanistic interpretability: parameter decomposition and causal transcoders

Currently

I'm working on post-training at Disney, following its acquihire of Character.AI. You can reach me at arnavkartikeya@gmail.com.