Depth

Language models quiz

All 10 questions. After you submit, each question shows the right answer and why. Back to the quick check

ExplorerQuestion 1 of 10

A chatbot that sounds very confident is always telling the truth.

True

False

ExplorerQuestion 2 of 10

What is a large language model mainly doing when it writes a sentence?

Predicting a likely next word (or piece of a word), again and again

Looking up the one correct answer in a list of facts

Asking a human to type the answer

Copying a sentence from a dictionary

InvestigatorQuestion 3 of 10

A model gives 'mat' probability 0.6 after 'The cat sat on the', then gives '.' probability 0.5 after 'The cat sat on the mat'. What is the probability of the continuation 'mat.'?

Your answer

InvestigatorQuestion 4 of 10

How many parameters did GPT-3 have, in billions?

Your answer

ScientistQuestion 5 of 10

According to its own authors, GPT-3 retains the biases of the data it was trained on, which may lead it to generate stereotyped or prejudiced content.

True

False

ScientistQuestion 6 of 10

Why do language models split rare words into smaller subword units?

To hide the meaning of words from users

Because computers cannot store whole words

So they can handle words they have never seen whole, using a fixed vocabulary

To make every word exactly three letters long

AdvancedQuestion 7 of 10

What did Bengio and colleagues (2003) propose to help language models generalise to word sequences never seen in training?

Using only one-word contexts

Learning a distributed vector representation for each word

Removing all rare words from the language

Storing every possible sentence in a table

AdvancedQuestion 8 of 10

In Ouyang et al. (2022), which model's outputs did human evaluators prefer?

A 1.3-billion-parameter InstructGPT model fine-tuned with human feedback

The original 175-billion-parameter GPT-3

A model trained without any human input at all

Neither — they were judged identical

ExpertQuestion 9 of 10

Kaplan et al. (2020) reported that language-model loss scales with model size, dataset size and compute in what functional form?

A step function

An exponential decay to exactly zero

No consistent relationship

A power law

ExpertQuestion 10 of 10

State the range of hallucination rates the AI Index 2026 reports across 26 models on the AA-Omniscience benchmark, and give one reason such a figure should not be quoted as 'the' hallucination rate of language models.

Your answer