Context Window

AI research moves quickly. We explain the ideas behind it and put our favorite papers on shirts.

Learn

A bigram model uses one context word; a trigram model uses two

N-gram models

Next-word probabilities after we sat on the bank of the

Next-word prediction

Words split into subword tokens

Tokenization

Shop

Attention Is All You Need on UT0112 — back

Attention Is All You Need$40

Language Models are Few-Shot Learners on UT0112 — back

Language Models are Few-Shot Learners$40

Generative Pre-training on UT0112 — back

Generative Pre-training$40

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding on UT0112 — back

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding$40