Language models quiz — ScienceVerse

ScientistQuestion 1 of 3

According to its own authors, GPT-3 retains the biases of the data it was trained on, which may lead it to generate stereotyped or prejudiced content.

True

False

AdvancedQuestion 2 of 3

What did Bengio and colleagues (2003) propose to help language models generalise to word sequences never seen in training?

Storing every possible sentence in a table

Learning a distributed vector representation for each word

Using only one-word contexts

Removing all rare words from the language

AdvancedQuestion 3 of 3

In Ouyang et al. (2022), which model's outputs did human evaluators prefer?

Neither — they were judged identical

A model trained without any human input at all

The original 175-billion-parameter GPT-3

A 1.3-billion-parameter InstructGPT model fine-tuned with human feedback

Language models quick check