FLA · Experiment 2: teachers · 16 Nov 2026 · all reports · summary

Teachers for hard against soft floor

Before a camera can learn from the sole, the sole has to label the steps. Three teachers label every footstep of the Booster K1 as hard or soft floor after the step: one reads only the sole, one only the joints and IMU, one both. Every step is labelled by a model that never trained on its episode.

Simulation only. One gait, two floor classes.

In short

  1. The task is almost solved by every teacher. Sole 0.997, joints and IMU 0.990, both together 0.995, on 22,773 steps. Chance is 0.500.
  2. The sole is slightly better than the joints. +0.007 on all steps, real by the rule used here, but both error rates are already below 1 %.
  3. Reading both does not help over the sole alone.
  4. Standing and stopping is the only weak spot. On steps commanded below 0.05 m/s, the joints teacher drops to 0.899, the sole to 0.975.
  5. The consequence: the three label sets agree with truth on 99.0 to 99.7 % of steps, so a vision head trained on any of them cannot show which teacher is better on this task.
1

What was done

2

How well each teacher labels

TeacherAll stepsTest splitSingle-floor episodesStrips
Sole (coarse)0.997 ± 0.001
[0.996, 0.997]
0.997 ± 0.001
[0.996, 0.999]
0.997 ± 0.001
[0.996, 0.998]
0.995 ± 0.001
[0.992, 0.997]
Joints + IMU0.990 ± 0.001
[0.988, 0.992]
0.993 ± 0.001
[0.990, 0.995]
0.990 ± 0.001
[0.989, 0.992]
0.987 ± 0.002
[0.980, 0.992]
Sole + joints + IMU0.995 ± 0.001
[0.994, 0.996]
0.997 ± 0.001
[0.996, 0.999]
0.995 ± 0.001
[0.994, 0.996]
0.991 ± 0.002
[0.987, 0.995]
Sole, shuffled labels0.377 ± 0.045
[0.357, 0.397]
0.365 ± 0.039
[0.326, 0.400]
0.378 ± 0.042
[0.354, 0.397]
0.363 ± 0.072
[0.338, 0.384]
Scored steps22,7735,01620,3012,472

Reading the table. Macro-F1 against truth, mean ± sd over three seeds, with a 95 % interval from resampling episodes. All three teachers are right on about 99 % of steps or more, everywhere. Accuracy is the same as F1 to three decimals (0.997, 0.990, 0.995); per class, the sole scores 0.997 on hard and 0.996 on soft. On the test split, with stiffness and friction values no model trained on (soft stiffness 8.3e3 to 1.6e4 against 1.1e4 to 1.1e5 in training; friction up to 1.30 against 1.10), all three keep 0.993 to 0.997.

The shuffled control sits at or below chance (seeds 0.429, 0.349, 0.353): it collapses toward one class, as the constant majority answer (0.350) does. Its accuracy is 0.515. The independent re-check reproduced the main scores but did not verify the stiffness and friction ranges of the test split.

3

By walking speed

TeacherStand / stop, below 0.05 m/s0.05 to 0.300.30 to 0.400.40 and above
Sole (coarse)0.975 ± 0.0030.996 ± 0.0010.999 ± 0.0010.999 ± 0.000
Joints + IMU0.899 ± 0.0110.992 ± 0.0010.998 ± 0.0000.998 ± 0.000
Sole + joints + IMU0.952 ± 0.0070.996 ± 0.0010.998 ± 0.0000.999 ± 0.001
Sole, shuffled labels0.367 ± 0.0090.390 ± 0.0340.358 ± 0.0440.388 ± 0.067
Scored steps1,4477,1779,0395,110

Reading the table. Speed is the commanded forward speed at touchdown. Above 0.05 m/s every teacher scores 0.992 to 0.999. When the robot stands or stops, the joints barely move and the joints teacher falls to 0.899; the sole still feels the floor directly and keeps 0.975. This is the only subset where the teachers differ, and the fixed-patch results use it to try to separate them.

4

Is the sole really better?

Differences in seed-mean macro-F1 with a paired 95 % interval from resampling episodes. A difference counts as real only if the interval excludes 0 and the difference is larger than twice the seed sd.

ComparisonStepsDifference95 % intervalReal?
Sole minus jointsall+0.0067[+0.0053, +0.0083]yes
Both minus soleall−0.0020[−0.0027, −0.0014]no
Both minus jointsall+0.0048[+0.0034, +0.0062]yes
Sole minus jointstest split+0.0045[+0.0022, +0.0069]yes
Both minus soletest split−0.0003[−0.0014, +0.0008]no
Both minus jointstest split+0.0042[+0.0019, +0.0069]yes
Sole minus jointsstrips+0.0080[+0.0027, +0.0147]yes
Both minus solestrips−0.0032[−0.0062, −0.0004]no
Both minus jointsstrips+0.0047[−0.0001, +0.0103]no

Reading the table. The sole beats the joints by a small margin that passes the rule everywhere: +0.007 [+0.005, +0.008] on all steps, +0.005 [+0.002, +0.007] on the test split. Both error rates are below 1 %, so for labelling the gap does not matter. Reading both sensors does not help over the sole: on all steps both minus sole is −0.002, with an interval that excludes 0 but a size within twice the seed sd, so it does not count.

5

A shifted sole readout

The same teacher weights, applied to a second, shifted model of how the simulated sole turns contact into readings. Two sets: 159 of these episodes replayed bit-identically for the earlier vision experiment (cross-fitted as above), and 32 separate G113 runs on hard and soft floor that no model saw.

TeacherReplays, usual readoutReplays, shiftedSeparate runs, usualSeparate runs, shifted
Sole (coarse)0.997 ± 0.0010.997 ± 0.0001.000 ± 0.0001.000 ± 0.000
Joints + IMU0.989 ± 0.0010.989 ± 0.0010.997 ± 0.0000.997 ± 0.000
Sole + joints + IMU0.996 ± 0.0010.995 ± 0.0011.000 ± 0.0001.000 ± 0.000
Sole, shuffled labels0.375 ± 0.0420.367 ± 0.0330.356 ± 0.0000.356 ± 0.001
Scored steps6,7616,7611,3171,317

Reading the table. The coarse sole teacher loses nothing under the shifted readout: 0.997 to 0.997 on the replays (+0.0002 [−0.0004, +0.0008]), 1.000 to 1.000 on the separate runs. The coarse features were chosen for this reason; step 1 showed that the fine pressure map does not survive a change of sensor model.

The independent re-check did not recompute the shifted readout; only the original check covers it.

6

At a floor change

On strip episodes: how many steps after the boundary each teacher needs to name the new floor (0 = on the boundary step itself). "Never" means not before the next boundary. 60 strips, one boundary each, always hard to soft; pooled over three seeds.

TeacherCases01234+NeverMedian90th pctMean
Sole (coarse)180179100000.00.00.01
Joints + IMU180178002000.00.00.03
Sole + joints + IMU180180000000.00.00.00
Sole, shuffled labels1801602261540.023.54.96

Reading the table. The teachers name the new floor on the boundary step in 178 to 180 of 180 cases. That is expected: each teacher reads one step's own stance window and keeps no memory, so nothing carries over from the old floor. The shuffled control is censored in 154 of 180 cases, so its median of 0 describes only the 26 cases it got right.

7

What it means and its limits

Scores are macro-F1 per step against simulator truth, cross-fitted, mean ± sd over three seeds. Related: dataset of experiment 2, fixed-patch results, re-check of experiment 2, teachers of the six-class experiment, step 1, second simulation run.