FLA · Experiment 1 support state · 16 Nov 2026 · all reports · summary

Support state from vision: a weak result

Will the next foot be fully supported, half over a gap, or not in contact at all? A second small head on the same ViT features tries to say so before the step. It barely beats answering the most common state every time.

Weak result. Vision's accuracy equals a trivial "always full" answer. Simulation only.

In short

  1. Vision reads support state poorly. Macro-F1 0.53-0.55 before the step, against 0.736 for the sole teacher after the step.
  2. Its accuracy is a trivial baseline. Vision is right on 0.94-0.95 of steps, and always answering "full" scores 0.943.
  3. On hidden gaps, vision is below a trivial answer. Where a sheet hides the gaps between planks, always answering "partial" scores 0.685 accuracy. Vision scores 0.652, the sole teacher 0.715.
  4. Vision calls plank floors "partial" from their look. That is why it finds partial support more often than the sole on honest planks (recall 0.864 against 0.673), and why it loses that edge when the gaps are hidden.
  5. Sole labels teach this head better than joints and IMU labels, and as well as truth: 0.548 against 0.411 and 0.529.
1

What was done

2

How rare the states are

Test subsetNonePartialFull
All honest test steps (6,269)543035,912
Edge-class test steps (689)5303381
Hidden gaps, no_support (726)2497227

Reading the table. "None" barely exists (under 1 % of steps) and is not a usable class. Partial support exists almost only on edge terrain: in the whole step-1 data, 12,365 of 12,372 partial steps (the source report says 12,364 of 12,371; the independent re-check's count is used). The no_support scenes are plank floors under a sheet that exists only for the camera: it lies 2 mm below the plank tops, hides the gaps, looks solid and gives no support.

3

All test steps

Labels for the vision headMacro-F1Accuracy
(a) sole teacher0.548 ± 0.0200.950 ± 0.011
(a) joints + IMU teacher0.411 ± 0.0410.945 ± 0.002
(a) simulator truth0.529 ± 0.0110.940 ± 0.007
(b) sole teacher0.504 ± 0.0220.927 ± 0.012
(b) joints + IMU teacher0.264 ± 0.0730.652 ± 0.322
(b) simulator truth0.537 ± 0.0040.945 ± 0.002
Shuffled labels (control)0.341 ± 0.0200.913 ± 0.031
Always "full" (trivial baseline)–0.943
Sole teacher, after the step0.736 ± 0.0170.961 ± 0.003
Three bar charts of support-state macro-F1 for the vision heads and the sole teacher: on all test steps, on edge-class steps and on hidden-gap scenes; the sole teacher after the step is highest in each
Support-state macro-F1 by label source, for all test steps, edge-class steps and hidden-gap scenes. Dark bar: the sole teacher after the step.

Reading the table. Accuracy says almost nothing here: nearly every step is fully supported, so the (a) heads land at 0.94-0.95, the same as always answering "full" (0.943). Macro-F1 shows the difference. Sole labels beat joints and IMU labels, really, in both settings ((a) 0.548 against 0.411, (b) 0.504 against 0.264), and are as good as truth (neither 0.548 against 0.529 nor 0.504 against 0.537 is a real difference). The sole teacher after the step is really above every vision head. Chance for three states is 0.333; the shuffled control sits at 0.341.

From the independent re-check: vision's accuracy (0.94-0.95) equals the "always full" baseline of 0.943. The "none" state has only 54 test steps, so the macro-F1 leans on a tiny class. On SONIC steps the "sole teacher after the step" is the teacher trained with G113 held out, which did see SONIC.
4

Planks and hidden gaps

SourceEdge steps, macro-F1Edge steps, accuracyHidden gaps, macro-F1Hidden gaps, accuracy
Vision, (a) truth labels0.462 ± 0.0130.691 ± 0.0180.423 ± 0.0030.652 ± 0.012
Vision, (a) sole labels0.460 ± 0.0180.690 ± 0.0250.388 ± 0.0080.589 ± 0.015
Vision, (a) joints + IMU labels0.332 ± 0.0440.598 ± 0.0250.191 ± 0.0410.343 ± 0.038
Always "partial" (trivial baseline)–––0.685
Sole teacher, after the step0.693 ± 0.0470.802 ± 0.0090.466 ± 0.0070.715 ± 0.009

Reading the table. On honest plank floors the sole after the step is far ahead (0.693 against 0.462). Under a sheet that hides the gaps, everything drops, and the sole is really better than vision by 4 points of macro-F1 and 6 of accuracy. But answering "partial" every time already gives 0.685 accuracy on these scenes: vision is below that trivial answer, and the sole teacher is only 3 points above it.

The trivial "always partial" baseline was found by the independent re-check; the original report did not have it. It turns the hidden-gap result from "the sole is better" into "neither source does much better than guessing the most common state".
5

Finding partial support

Share of truly partial steps that each source calls partial.

SourceEdge steps (303 partial)Hidden gaps (497 partial)
Vision, (a) truth labels0.864 ± 0.0080.642 ± 0.051
Vision, (a) sole labels0.860 ± 0.0250.525 ± 0.038
Vision, (a) joints + IMU labels0.177 ± 0.0950.050 ± 0.067
Sole teacher, after the step0.673 ± 0.0170.702 ± 0.005

Reading the table. On honest planks vision finds more partial steps than the sole does, because it calls plank ground "partial" from its look. When a sheet hides the gaps, vision's recall falls (0.864 to 0.642 with truth labels, 0.525 with sole labels), and the sole keeps 0.702. Vision fails where the sole succeeds in direction, and moderately in size. It does not fail completely, because the planks are still visible.

6

Independent re-check

Full list: independent re-check of experiment 1.

7

What it means and limits

Scores: macro-F1 over the support states present in each subset, against simulator truth on held-out textures, mean ± sd over 3 seeds; accuracy where stated. Related: the dataset, the two teachers, the vision head, independent re-check. Model: DINOv3 ViT-B/16. Textures: ambientCG, CC0.