Guess the Picture from Its Features

No adult supervision needed: pictures and sticky notes only.

Protective equipment and “do not substitute” warnings have not been recorded for this experiment yet. They are added when its safety review is completed; until then, follow the supervision notes above.

Disposal: Reuse the sticky notes or recycle them with the paper.

Materials

• Five or six printed or drawn pictures of everyday things (a zebra, a bicycle, a cat, a car, a tree)
• About 12 sticky notes per picture
• A partner
• Paper and a pen for scoring

Steps

1. Cover one picture completely with sticky notes, without your partner seeing it.
2. Remove one sticky note at a time. After each one, your partner may make one guess.
3. When they guess correctly, write down how many notes were removed and which small piece gave it away (stripes, a wheel, an ear, a leaf).
4. Swap roles and repeat with the other pictures.
5. Compare: which small pieces gave the picture away fastest, and what kind of feature were they?

What you should see

• Some pictures are guessed from one or two small pieces, others need many.
• The give-away pieces are usually distinctive edges, textures or colours (zebra stripes) or recognisable parts (a wheel, an ear).
• Guessers combine several small clues into a part, and parts into a whole object.

Why it works

Computer-vision systems built from convolutional neural networks also recognise images by building up features. When researchers looked inside one trained network, its early layers responded to simple features such as edges and colour combinations, middle layers to textures and more complex patterns, and later layers to object parts and whole objects.

This activity is a loose human analogy, not a description of how the network works inside: you combine small clues into parts and parts into objects, in roughly the same order. The network learns which features are useful from many training images rather than from experience.

Learn the ideas behind it

• Computer vision
• Machine learning
• Neural networks

Explore it further

The same ideas, as something you can change and watch on screen:

• Neural Network Playground (simulation)
• Neural Network Explorer (3D)

Sources

• Visualizing and Understanding Convolutional Networks (Zeiler & Fergus, ECCV 2014) — Springer (Computer Vision – ECCV 2014, LNCS 8689)