BERT
BERT pretrains bidirectional Transformer encoders using masked-token and sentence-level objectives. Its architecture is oriented toward learned representations for downstream tasks rather than autoregressive generation.
Overview. BERT pretrains bidirectional Transformer encoders using masked-token and sentence-level objectives.
Context. Its architecture is oriented toward learned representations for downstream tasks rather than autoregressive generation.
Claims and evidence
Claim ledger, 2BERT pretrains bidirectional Transformer encoders using masked-token and sentence-level objectives.
Its architecture is oriented toward learned representations for downstream tasks rather than autoregressive generation.