Transformers quiz — ScienceVerse

AdvancedQuestion 1 of 3

In the Transformer decoder, the prediction for a position may attend to later positions in the output so it can see what comes next.

True

False

AdvancedQuestion 2 of 3

How does the per-layer computational cost of self-attention grow with sequence length n?

Linearly, O(n·d²), like a recurrent layer

It does not depend on n

Exponentially

Quadratically, O(n²·d)

AdvancedQuestion 3 of 3

What distinguishes BERT's pre-training from a left-to-right language model?

It conditions on both left and right context in all layers

It uses no attention at all

It is trained only on labelled data

It reads text right-to-left only

Transformers quick check