FLA · Experiment 1 teachers · 16 Nov 2026 · all reports · summary

The two teachers

Before a camera can learn from the sole, the sole's labels have to be good. Two step-level classifiers act as teachers: one reads only the sole, one reads only joints and IMU. Here is how well each names the floor, on gaits it was trained on, on a gait it never saw, and under other sensor models.

Simulation only. Booster K1 in MuJoCo, simulated sole. Six floor classes, three gaits.

In short

  1. On gaits they saw, the two teachers tie. Sole 0.804, joints and IMU 0.805, macro-F1 per step over six floor classes.
  2. On a gait they never saw, only the sole still works. With SONIC held out: sole 0.399, joints and IMU 0.095. The other two hold-outs point the same way.
  3. The joints and IMU teacher does worse than guessing on the new gait. It calls almost every SONIC step "edge". The independent re-check notes that this is below a random guesser (about 0.17), so "chance" undersells the failure.
  4. Coarse sole features survive new sensor parameters, not a new sole. Shifted parameters cost 2 points on seen gaits; a differently built sole or a foot-outline mask costs 19-22 points.
  5. For support state the sole is clearly better. Sole 0.719, joints and IMU 0.445 on seen gaits.
1

What was done

2

Seen and unseen gait

Macro-F1 per step, trained on G113 and G80
Sole, seen gaits
0.804
Joints + IMU, seen gaits
0.805
Sole, SONIC (unseen)
0.399
Joints + IMU, SONIC (unseen)
0.095
Bars run from 0 to 1. Shuffled-label control: 0.057.
TeacherSensor and test setSeen gaits (G113, G80)Unseen gait (SONIC)
Sole (coarse)step-1 test, stored sensor0.804 ± 0.0030.399 ± 0.004
Joints + IMUstep-1 test, stored sensor0.805 ± 0.0020.095 ± 0.015
Sole, shuffled labels (control, one seed)step-1 test0.057

Reading the table. On gaits both teachers saw in training they are equally good, about 0.80. On SONIC the sole teacher drops to 0.40 but still carries information; the joints and IMU teacher collapses. It calls almost every SONIC step "edge". The gap is at least ten times the seed spread. The sole's SONIC labels are biased too: it almost never says hard_grip (F1 0.05) and says slope too often.

The independent re-check confirms both numbers but adds: the joints and IMU teacher is a degenerate "edge" predictor that scores below a random guesser (about 0.17), so calling it "at chance" undersells how badly it fails. The shuffled control re-computes to 0.052 on seen gaits and 0.070 on SONIC; it collapses to nearly one class.
3

Per class

TeacherGaitshard_griphard_slipsoftroughslopeedge
Soleseen0.760.961.000.570.940.61
Joints + IMUseen0.760.960.990.610.970.54
Soleunseen0.050.680.730.280.380.27
Joints + IMUunseen0.010.160.080.000.170.16

Reading the table. F1 per class, step-1 test set, mean of 3 seeds. On seen gaits both teachers are near perfect on soft, slippery and sloped floors and weak on rough, edge and grippy floors, which they confuse with each other. Part of that error is in the label itself: an "edge" step often never touches an edge. On SONIC the sole keeps slippery and soft floors partly, and the joints and IMU teacher keeps nothing.

4

Other held-out gaits

The same experiment with each gait held out in turn: train on the other two, test on all three.

Held-out gaitTeacherSeen gaitsHeld-out gait
SONICSole0.804 ± 0.0030.399 ± 0.004
SONICJoints + IMU0.805 ± 0.0020.095 ± 0.015
G113Sole0.747 ± 0.0010.627 ± 0.024
G113Joints + IMU0.730 ± 0.0070.279 ± 0.029
G80Sole0.779 ± 0.0060.485 ± 0.010
G80Joints + IMU0.762 ± 0.0010.264 ± 0.021

Reading the table. Every hold-out points the same way: on the gait it never saw, the sole teacher is clearly ahead (0.40 to 0.63 against 0.095 to 0.279). The sole labels on a new gait are noisy but informative.

5

Other sensor models

A real sole will not match the simulated one. So the same sole teacher (trained on the stored sensor, SONIC held out) is scored on 288 separate test simulations (96 per gait), read out in different ways. The pad sole is a differently built sole that changes the dynamics, so it needs its own simulation of the same tasks (11,552 steps against 11,556).

TeacherSensorSeen gaitsSONIC
Solenominal sensor0.806 ± 0.0040.437 ± 0.017
Soleshifted parameters0.784 ± 0.0110.429 ± 0.009
Solepad sole (own simulation)0.586 ± 0.0120.325 ± 0.003
Solefoot-outline mask0.621 ± 0.0040.419 ± 0.019
Joints + IMUnominal sensor run0.805 ± 0.0040.112 ± 0.020
Joints + IMUpad sole run0.789 ± 0.0050.105 ± 0.020

Reading the table. Shifted sensor parameters (blocks moved, twice the spread, noise and compliance, crosstalk) cost the sole 2 points on seen gaits (0.806 to 0.784, about twice the spread, borderline) and nothing on SONIC (0.437 to 0.429). The pad sole and the foot-outline mask cost 19 to 22 points. Coarse-feature labels survive a change of sensor parameters, not a change in how the sole is built. The joints and IMU teacher does not read the sole; its rows are there for reference.

On the replayed episodes of the vision dataset (617 of them labelled for this check), the same sole teacher on the stored / nominal / shifted readout scores 0.778 / 0.781 / 0.761 on seen gaits and 0.394 / 0.402 / 0.395 on SONIC. Its shifted labels agree with its stored-sensor labels on 88 % of steps.

6

Support-state teachers

A second question for the same teachers: is the foot fully supported, partly supported (over a gap or a step edge), or not in contact? Truth is the simulator's support state 0.1 s after touchdown. None means no contact (force along the sole normal of 2 N or less). Partial means contact with fewer than 90 % of the sole's 24 probe points over supporting ground; tilt and bumps of 2 cm or less do not count. Full is the rest. Same steps, inputs and recipe, with a three-way output weighted 1 / 3 / 1.

SplitG113G80SONIC
Train245 / 3,590 / 49,927692 / 3,361 / 48,649295 / 1,791 / 32,757
Validation43 / 513 / 7,606156 / 455 / 7,41341 / 271 / 4,722
Test85 / 998 / 18,478203 / 959 / 17,961134 / 434 / 13,614

Reading the table. Steps per support state, none / partial / full. "None" is rare (0.9 % of steps, mostly on slippery floor). Partial support occurs almost only on edge terrain: 12,365 of 12,372 partial steps. The source report says 12,364; the independent re-check counted 12,365, and its count is used here.

Held-out gaitTeacherSeen gaitsHeld-out gaitPartial F1, seenPartial F1, held out
SONICSole0.719 ± 0.0150.486 ± 0.0120.695 ± 0.0210.254 ± 0.046
SONICJoints + IMU0.445 ± 0.0030.305 ± 0.0350.363 ± 0.0160.042 ± 0.023
G113Sole0.693 ± 0.0080.581 ± 0.0400.608 ± 0.0090.442 ± 0.096
G113Joints + IMU0.417 ± 0.0170.346 ± 0.0080.265 ± 0.0310.120 ± 0.007
G80Sole0.597 ± 0.0310.513 ± 0.0070.626 ± 0.0050.509 ± 0.032
G80Joints + IMU0.418 ± 0.0130.357 ± 0.0120.281 ± 0.0410.112 ± 0.028

Reading the table. Macro-F1 over the support states present in the test split, mean ± sd over 3 seeds. Unlike the floor class, here the sole is clearly ahead even on gaits both teachers saw. With three states, the chance reference is 0.333, and the shuffled-label control scores 0.324. The joints and IMU teacher is close to that on held-out gaits.

How vision learns support state from these labels: support state from vision.

7

Independent re-check

Full list: independent re-check of experiment 1.

8

Limits

Scores: macro-F1 per step against simulator truth on the step-1 test split, mean ± sd over 3 seeds. Related: the dataset, the vision head these teachers train, support state from vision, independent re-check. Teachers of experiment 2 (two classes, one gait): teachers of experiment 2. Step 1: Terrain from Touch.