The world that learns what it cannot see
Two small learners live in two worlds built from the same pieces. In both worlds every piece, and every pair of pieces, turns up equally often. Only the way three pieces go together differs: in world A, matching eyes come with a smile; in world B, matching eyes come with a frown. Nobody tells the learners which pieces belong together. Let them live in their worlds, then show both the same face with something covered.
Learner A
Lives in world A: matching eyes, smile.
What its experience connected
Waiting for its first experience.
The same scene, shown to both
Click a piece to change it.
Same experience, links two at a time
A delta-rule linear map trained on the very same scenes. It can only learn how pieces relate in pairs, and in these worlds pairs say nothing.
Learner B
Lives in world B: matching eyes, frown.
What its experience connected
Waiting for its first experience.
How often each one gets a covered mouth right
How a learner learns
Each learner has 24 small branches. A branch is wired to two or three of the nine pieces and keeps a handful of snapshots of what those pieces looked like together (competitive learning: only the closest snapshot moves toward each new experience).
Before learning from an experience, every branch tests itself. It hides one of its own pieces, predicts it from the others, and checks. A branch that keeps guessing at chance is cut and regrows on other pieces. A branch that predicts reliably is kept. Pairs never pass the test here, because in these worlds no piece says anything about any other single piece. Only the branch that happens to wire the two eyes and the mouth together does, and it holds the opposite rule in the two worlds.
To fill a covered piece, every kept branch that touches it weighs its snapshots against what is visible (and against any pixels you peek at). How sure it is depends on how much the evidence agrees. A scene unlike anything it has lived through, or a peek that contradicts it, takes its say away.
What to try
- Before any experience, both learners paint a grey average. Start experience and watch the wiring search until one branch locks onto the eyes and the mouth.
- Change one eye. Both learners flip their answer, in opposite directions.
- Cover an eye and the mouth. Neither can tell. Slide the peek: as the mouth shows, the hidden eye appears too, and not the same eye for both learners.
- Cover only the mouth of a world A face and peek: learner B's certainty collapses as the evidence disagrees with it.
- Cut the links: every stored piece is still there, only re-paired. The answers become wrong until the self-tests notice and the learner rebuilds.
- Move learner A into world B and watch its own experience overrule what it knew.
What this is, and what it is not
Measured with this page's own code in node (world_receipt.mjs): 20 learners, ten seeds
per world, 3,000 experiences each. All 20 found the eye, eye and mouth wiring by themselves, after 62 to 1,121
experiences (median 250), and none kept any other wiring. A covered mouth: every learner 100%; links two at a time
49.9%; erased 50%; never shown a mouth 50%; links cut 46% on average (9 to 100%, because a random re-pairing
sometimes lands on an equivalent one).
It is a toy: nine pieces in fixed places, two glyphs per piece, a hand-set pixel noise level. The world was built so that only the three-way relation carries the answer. The search over wirings is a random search over 84 possible triples; in real images the number of possible groupings explodes, and that is the open problem. The parts are old: competitive learning (Rumelhart & Zipser 1985), Barlow's suspicious coincidences, masked self-prediction, synaptic turnover, mixture-model completion.
From the Oct 22 thread with Astra, Sol and Gemini: ClaudeRepo1. The webcam memory toy is Living Paper.