Researchers at Google have introduced Titans and MIRAS, a new architecture and theoretical framework designed to give AI models effective long-term memory without the computational explosion of Transformers. While current alternatives like Mamba-2 trade expressiveness for speed by compressing context into fixed sizes, Titans employs a deep neural network as a dynamic memory module that updates in real-time based on a “surprise metric”—effectively learning context on the fly rather than compressing it into a static state. This approach, grounded in the MIRAS framework’s move beyond standard Mean Squared Error optimization, reportedly allows Titans to outperform GPT-4 on the BABILong benchmark with context windows exceeding 2 million tokens.