Asher Labovich

Researchers introduce a fixed-point framework for analyzing looped transformers, which use iterative computation at test time to tackle harder problems. The work proves that certain architectural choices, specifically recall combined with outer normalization, enable these networks to generalize to harder problems rather than memorize training solutions. Empirical validation on chess, sudoku, and prefix-sums tasks confirms the framework's predictions, and a novel internal recall placement variant outperforms standard approaches on some benchmarks.