The Transformer era
After 2017, Transformer architectures spread from translation into language, vision, speech, and multimodal systems. Attention-based scaling changed the dominant training stack without making recurrence or convolution obsolete for every task.
Overview. After 2017, Transformer architectures spread from translation into language, vision, speech, and multimodal systems.
Context. Attention-based scaling changed the dominant training stack without making recurrence or convolution obsolete for every task.
Claims and evidence
Claim ledger, 2After 2017, Transformer architectures spread from translation into language, vision, speech, and multimodal systems.
Attention-based scaling changed the dominant training stack without making recurrence or convolution obsolete for every task.