Language Models are Few-Shot Learners
The GPT-3 paper studies autoregressive language-model scaling and task performance from instructions or examples in context. The work reports few-shot behavior without gradient updates for each evaluated task.
Overview. The GPT-3 paper studies autoregressive language-model scaling and task performance from instructions or examples in context. The work reports few-shot behavior without gradient updates for each evaluated task.
Mechanism and context. The study scales autoregressive language models and supplies task instructions or examples inside the context at inference time. The resulting few-shot behavior changes the conditional continuation without a gradient update for each evaluated task.
Practical use. Preserve the exact prompt, number and order of examples, decoding settings, task split, and dated model identity when interpreting an in-context result. Those details define the evaluated system and can matter even when the parameter weights stay fixed.
Related reading. Continue with Foundation model for broad reuse, Large language model for the sequence objective, Prompting for inference-time conditioning, and Neural scaling laws for the empirical planning relationship studied in this research era.
Claims and evidence
Claim ledger, 2The GPT-3 paper studies autoregressive language-model scaling and task performance from instructions or examples in context.
The work reports few-shot behavior without gradient updates for each evaluated task.