FLA · Step 1, second simulation run · 14 Nov 2026 · all reports · summary

Terrain from Touch: the second simulation run

The Booster K1 in MuJoCo with a pressure-map sole: 19 × 8 points per foot, read on top of rigid contact. Seven questions, fixed in advance with one rule for what counts as a real difference: map against four pressure blocks, the value of shear, unseen gaits, a mixed-terrain course, partial support, and whether any of it survives a change in how the sole is simulated.

Simulation only. The most important answer does not survive a change of sensor model.

In short

  1. Read this first. The point map's advantage over four pressure blocks is large in this simulation and does not survive a change in how the map is produced. Point-map models trained on one sole setting and tested on another lose most of their accuracy; block models lose much less.
  2. On the main data the map wins on every metric: terrain F1 0.877 against 0.829, partial-support F1 0.802 against 0.471, long edge events missed 2.0 % against 26.6 %.
  3. Shear adds little. Block shear on top of the map: no difference with joints and IMU. Per-point shear here has no content of its own.
  4. Sole-only models transfer better to an unseen gait for all five sole inputs: mean 0.629 against 0.532; on the human-like gait 0.526 against 0.071.
  5. On a mixed course the sole model beats joints and IMU alone (accuracy 0.842 against 0.785) and hand-written rules (0.311).
  6. A real recording is needed before a fine pressure map can be trusted over block sums.
1

The seven answers

Inputs: P is joints and IMU (no walking command). The five sole inputs, each with and without P: C1 point map plus per-point shear; C2 point map plus block shear; C3 point map only; C4 four pressure blocks per foot plus block shear; C5 four pressure blocks only. All come from one recording. Walking, test split, mean of three seeds unless stated. The metric definitions and the rule for calling a difference real were fixed before the dataset existed.

#QuestionAnswer by the ruleKey numbers
1Does the pressure map beat four pressure blocks?Yes on the main dataset, on every metric; not robust (question 7).P+C3 against P+C5: terrain F1 0.877 against 0.829, support F1 0.929 against 0.810, partial-support F1 0.802 against 0.471, friction MAE 0.156 against 0.174, long edge events missed 2.0 % against 26.6 %
2Does block shear add anything on top of the map?No difference with joints and IMU. Without them, terrain F1 is better and nothing else.P+C2 against P+C3: 0.889 against 0.877 terrain F1, inside the rule's 2-point floor. C2 against C3: 0.881 against 0.856
3Does per-point shear add anything over block shear?No, with two exceptions with joints and IMU (friction better, partial support worse), and the simulator cannot test this question.P+C1 against P+C2: friction MAE 0.141 against 0.158 (better), partial-support F1 0.801 against 0.827 (worse); all else no difference. Per-point shear here is the foot's total shear shared out by pressure
4Do sole-only models transfer to an unseen gait better than models with joints and IMU?Yes, for all five inputs.Mean terrain F1 on the held-out gait, C2 against P+C2: 0.629 against 0.532. On held-out SONIC: 0.526 against 0.071
5On a mixed course, how much better is the sole model than joints and IMU, and than rules?Better than both on accuracy, F1 and latency.Selected model P+C2: accuracy 0.842, boundary latency 0.62 s. P: 0.785 and 0.71 s. Rule-based: 0.311 and 3.57 s, half the boundaries missed
6Partial support on edge events of 0.3 s or longer?The point map detects them; blocks and joints miss a quarter to a third.Missed long events: P 37.3 %, block inputs 26.6–28.0 %, point-map inputs 2.0–2.6 %. Shear does not change the detection of long events; its one real effect is the lower partial-support F1 of P+C1
7Do the answers survive a change of simulation detail?Answers 2 and 3 survive on the three metrics tested (terrain F1, support F1, friction MAE). Answer 1 does not.Point-map models trained on the nominal sole fall from terrain F1 0.86–0.88 to 0.53–0.60 with shifted sensor parameters and to 0.20–0.27 on the pad sole and the foot-outline mask; block models fall to 0.59–0.76

Reading the table. On the main data the map wins and the size of the win is in support state, not terrain class. But answer 7 undercuts answer 1: point-map models trained on one sole setting lose most of their accuracy on another (up to 22 points lost for block models on the pad sole, 2 to 9 elsewhere, far more for map models). The simulation supports "a pressure map carries useful geometry about what is under the foot"; it cannot yet tell how much of that will hold on the real sole.

2

What changed since the first run

First runSecond run
Sole2 × 4 rigid zones per foot, forces taken per zone from the contact solverContact: four rigid blocks (front and back, each split into left and right) and a middle raised 1 mm. Reading: 19 × 8 points at 10 mm per foot (152), rebuilt from the foot's total force and moment
Sensor viewszones: normal force N, shear Smap and per-point shear; four blocks with block shear; all from one recording
Sensor parametersone fixed settingrandom per episode (boundaries, clearance, layer compliance, neighbour spread, crosstalk, gain, offset, noise, quantisation), plus fixed settings for the sensitivity test
Joints and IMU (P)joints, IMU, commandjoints, IMU; no command (one diagnostic run with it)
Modelstemporal CNNsame temporal CNN; point maps first pass a small masked 2D encoder per foot and frame. Weights of all 156 models saved
Dataset24.7 h, 4,572 episodes33.2 h, 6,139 episodes (train 21.9 h, validation 3.2 h, test 8.1 h), plus a test-only hard-mid set (friction 0.36–0.7), per-foot ground labels, sensitivity sets and 120 course runs
New testsleave-one-gait-out with and without P; a mixed-terrain course against P and a rule-based detector; sensitivity of the conclusions to the sole model
Gatethree pass/fail thresholdsno gate; seven questions with one rule, fixed in advance, for a real difference

Reading the table. The first run asked one pass/fail question about zone shear (first run). This run changes the sole model, removes the command from P, adds a third more data and asks seven narrower questions, including whether the answers depend on how the sole is simulated.

3

Setup, with what is approximate

Why the map is rebuilt from the foot's total force

The plan assumed the rigid simulator gets each region's total force right. Measured on G113 walking on flat ground, it does not: one block carries over 95 % of a foot's load in 33–38 % of loaded physics steps, and the whole load can move to another block within one 2 ms step. For flat parts of one rigid body the split is statically indeterminate, and the solver's choice is an artefact, the same one that put the load in one corner zone in the first run. What the dynamics do determine is the foot's total force, its centre of pressure and the twist about it. The map is rebuilt from those at every physics step: a thin elastic layer on a rigid plane, pressure at a point proportional to how far the plane presses into the layer given the ray-cast gap to the ground under that point, with the plane's height and two tilts solved so that total force and centre of pressure match the simulator exactly. Then the sensor model: neighbour spread, crosstalk, per-point gain and offset, noise, a 60 N per-point limit, quantisation, 150 Hz to 50 Hz.

Consequences, stated as limits of this simulator

Sole checks, 17 of 17 passed

The six checks of the first run on the new geometry; standing, no point above 1.0 % of the foot's load (limit 15 %) and the middle at 0.00 % (limit 5 %); centre of pressure from the map within 0.006 mm of the simulator's (limit 5 mm); a 5 mm bump under the middle raises its share to 80 %; half over an edge, points over the void read exactly zero before noise and the support fraction matches ground truth; the sum over points equals the foot force within 0.0009 % (limit 1 %). "Load shared by many points" had no number in the plan; the rule used is "at least half the points of the front and of the back carry at least half the uniform share" (measured 100 %), which replaced a first rule that no even spread could meet.

The simulated sole on a test rig: the foot is pressed down. Colour is pressure, arrows are shear. Front and back load, the middle stays empty.
Test rig: the foot is set down half on an edge. Only the supported part of the sole reads pressure.

Data, training and the rule-based detector

4

Main results

Walking, test split, mean of three seeds. Edge events are partial-support events on edges; long means 0.3 s or more.

InputTerrain F1Support F1Partial F1Friction MAEGrip/slip F1Strip latency, sLong edge missedShort edge missedLifted delay, s
P0.8170.7600.3580.1810.9480.640.3730.8450.30
P+C1 (map + point shear)0.8720.9290.8010.1410.9490.350.0220.7430.11
P+C2 (map + block shear)0.8890.9380.8270.1580.9780.360.0260.7550.11
P+C3 (map)0.8770.9290.8020.1560.9570.380.0200.7440.10
P+C4 (blocks + block shear)0.8360.8130.4770.1760.9690.490.2800.8160.11
P+C5 (blocks)0.8290.8100.4710.1740.9450.510.2660.8200.12
C10.8770.9290.8020.1410.9670.330.0270.7570.10
C20.8810.9340.8160.1560.9770.390.0220.7510.10
C30.8560.9230.7860.1700.9570.470.0260.7470.10
C40.8170.7950.4280.1750.9630.500.2780.8260.11
C50.7800.7920.4190.1930.9190.570.2860.8540.10
Block shear alone0.7930.7460.3130.1720.9470.580.3660.8540.35
Per-point shear alone0.8570.8790.6740.1650.9610.410.0500.7580.20

Reading the table. The clearest line runs between map inputs (C1 to C3) and block inputs (C4, C5): partial-support F1 0.786 to 0.827 against 0.419 to 0.477, share of long edge events missed 0.020 to 0.027 against 0.266 to 0.286. Terrain F1 differs much less. Strip latency in steps is 2.00 for every input without the point map and 1.33–2.00 with it; with three seeds of integer medians that is not a real difference by the rule.

Two bar charts of terrain macro-F1 and support-state macro-F1 for the five sole inputs C1 to C5, each with and without joints and IMU, with a dashed line for joints and IMU alone
Terrain and support-state F1 for the five sole inputs, sole only (light) and with joints and IMU (hatched). Dashed line: joints and IMU alone.
5

Map, block shear, point shear (questions 1 to 3)

Question 1: map against blocks

A real difference on all five metrics, with and without joints and IMU. The size is in support state, not terrain class: partial-support F1 goes from 0.47 to 0.80 and long edge events missed from 27 % to 2 %, while terrain F1 gains 5 points. The per-foot labels show where: on edge terrain, windows in which a foot was actually on an edge are recognised 94 % of the time with the map and 59–65 % without. A window of edge or rough terrain in which no foot touched an edge or a bump is recognised much less often by every input (edge 28–43 %, rough 8–18 %), which supports what the first run could only assert: part of the class error is in the label, not in the sensing. The per-foot labels are for analysis only.

Question 2: block shear on top of the map

With joints and IMU: no difference on terrain F1 (+1.3 points, under the 2-point floor), support F1, friction MAE, strip latency or grip/slip F1. Without them, terrain F1 is really better (+2.6 points) and nothing else. The first run's friction effect of shear is not there: 0.158 against 0.156.

Question 3: per-point shear over block shear

No difference, except friction MAE with joints and IMU (0.141 against 0.158, real by the rule) and partial-support F1, which is really worse (0.801 against 0.827). Without joints and IMU the friction difference is the same size (0.141 against 0.156); it passes the spread test and misses the 10 % floor by a hair (0.0152 against 0.0156), so by the rule it is no difference. Given how per-point shear is built here, this is an effect of feeding the same information in a different encoding. It says nothing about the real sensor's per-point shear.

6

An unseen gait (question 4)

Train on two gaits, test terrain F1 on the third.

InputG113 held outG80 held outSONIC held outMeanSame input, gait seen
P+C10.7980.6730.1200.5300.872
P+C20.8410.6860.0710.5320.889
P+C30.7880.6330.0980.5060.876
P+C40.4880.5540.0790.3740.834
P+C50.4980.4810.0660.3480.828
C10.7790.7150.5160.6700.876
C20.7170.6440.5260.6290.881
C30.6530.6290.4180.5670.855
C40.6230.5780.3630.5210.816
C50.4660.4900.2400.3990.780

Reading the table. Sole-only models are really better on the held-out mean for every input. Most of the gain comes from SONIC: with joints and IMU in the input, a model that never saw SONIC scores 0.07–0.12 on it, the level of the shuffled-label control; without them it scores 0.24–0.53. Between the two velocity policies, which move alike, the picture is mixed: with the map, joints and IMU help on held-out G113 (P+C2 0.841 against C2 0.717); with block pressure and block shear they hurt there (P+C4 0.488 against C4 0.623); with block pressure alone they make little difference (P+C5 0.498 against C5 0.466). Every model still loses 21 to 48 points against a gait it has seen. The first run's conclusion stands and is sharper: the controller that will run on the real robot must be in the training data, and joints and IMU are the part of the input that ties a model to a gait.

Bar chart of mean terrain macro-F1 on a held-out gait for C1 to C5, sole only and with joints and IMU, with black lines for the score on a seen gait
Leave-one-gait-out, mean over the three held-out gaits. Black lines: the same input on a gait seen in training.
7

The mixed course (question 5)

120 recorded runs, 40 courses per walking source, test only: six types of ground chunk in random order, planned length 60–69 s, completed runs 70–80 s. No source reached the planned 80 % of completed courses: every chunk was crossed in 19 of 40 runs by G113, 22 by G80 and 6 by SONIC (SONIC stays up in 12, but in six of those it does not reach the last chunk: its clips turn and wander as well as walk slowly). All runs are scored up to the fall, as planned. The sole model was selected on validation data before any course result: P+C2 (validation terrain F1 0.884; runner-up P+C3 0.880).

DetectorAccuracyMacro-F1Support F1Boundary latency, sStepsBoundaries missed
Sole model P+C20.8420.8410.9280.6220.7 %
P only0.7850.7830.7760.7121.3 %
Rule-based0.3110.2720.6333.57849.9 %

Reading the table. The sole model is really better than P on accuracy (+5.7 points), macro-F1 (+5.8) and median latency (0.09 s earlier); in steps both detect the new ground on the second step in the pooled data (P needs three steps on SONIC). The largest gap is on SONIC (accuracy 0.812 against 0.699, latency 0.62 s against 0.89 s). The rule-based detector is far behind both: its tracking-error rule fires on SONIC's low-gain controller for the whole run, and it has no rule that separates soft ground. A learned model on signals the controller already has is a much stronger baseline than hand rules.

Three bar charts for the mixed course: accuracy, macro-F1 and median boundary latency for the sole model, joints and IMU only, and the rule-based detector
Mixed course, pooled over the three walking sources and scored up to the fall.
One mixed course run with the flat-foot policy G113.
8

Partial support and unseen friction (question 6)

InputLong events missedMedian delay, sPartial-support F1Short events missed
P37.3 %0.120.35884.5 %
P + blocks (C4, C5)26.6–28.0 %0.12–0.130.47–0.4881.6–82.0 %
P + point map (C1–C3)2.0–2.6 %0.100.80–0.8374.3–75.5 %

Reading the table. The point map catches almost every edge contact that lasts 0.3 s or more; blocks and joints miss a quarter to a third of them. Brief contacts are missed by everything. Shear makes no difference to any of these numbers.

Two bar charts for all inputs: partial-support F1, and the miss rate of edge events lasting 0.3 s or more
Partial-support F1 (left) and missed long edge events (right); markers are seeds.

hard-mid: friction 0.36–0.7, never seen in training

Friction MAE: P 0.140, P+C5 0.159, P+C3 0.128, P+C2 0.109, C1 0.107. Models assign these windows to hard-grip in 53–64 % of cases and to hard-slip in 27–39 %. The friction estimate interpolates into the unseen range about as well as it works inside the trained ranges, and shear inputs give the lowest errors here: the one place in this run where shear shows a consistent benefit. It was not one of the comparisons fixed in advance, so it is an observation.

9

Does simulation detail pay off? (question 7)

Models trained on the fixed nominal sole, tested on the same test simulations read in five ways. Terrain F1; support F1 behaves alike.

InputNominalShifted parametersPad soleFoot-outline maskNo saturation
P+C10.8650.5790.2730.2040.786
P+C20.8760.5970.2260.2090.788
P+C30.8600.5300.2090.2410.779
P+C40.8050.7370.5880.7430.670
P+C50.7780.6880.5900.7550.650

Reading the table. Answer 1 flips: on shifted parameters, the pad sole and the outline mask, the block models are really better than the point-map models on terrain F1, and on the pad sole and the outline mask also on support F1. Answers 2 and 3 hold on all five test sets: no difference. The 60 N limit does not decide answers 1 and 2: trained and tested without saturation, the verdicts are the same as nominal (P+C3 0.864 against P+C5 0.776). Questions 3 to 6 were not rerun without the limit. A model trained with saturation and tested without loses 8–14 points, so the per-point range of the real sensor must be known before training for it.

Grouped bar chart of terrain macro-F1 for P+C1 to P+C5 trained on the nominal sole and tested nominal, shifted, on pads, with the foot-outline mask and without saturation
Trained on the nominal sole, tested five ways. Pink: foot-outline mask; yellow: trained and tested without saturation.

On the point "a fine pressure map beats four blocks", the simulation cannot be trusted until real recordings exist. A point-map model learns the reading method along with the ground. Block sums are much less sensitive to it, and that robustness is itself an argument for them, or for training on real data.

Exploratory, after the verdict: why the point-map models fail there

Not part of the seven answers. The main models, trained on per-episode random sensor parameters, tested on the same five sets (terrain F1):

ModelTrainingNominalShiftedPad soleOutline maskNo saturation
P+C2randomised0.8950.8840.2660.1270.827
P+C2nominal only0.8760.5970.2260.2090.788
P+C3randomised0.8840.8690.2120.1330.797
P+C3nominal only0.8600.5300.2090.2410.779
P+C5randomised0.8380.8200.6430.8240.797
P+C5nominal only0.7780.6880.5900.7550.650

Reading the table. Randomising the sensor parameters buys robustness to those parameters: on shifted parameters the point-map models recover from 0.53–0.60 to 0.87–0.88. It does not buy robustness to how the map is formed: on the pad sole and the outline mask the randomised models fail as badly as the nominal ones.

10

First run against second run

Not a like-for-like comparison: the sole, the sensor model, the command input, the dataset size and the network all differ. Both runs use the same three walking sources, classes, bands and metric code.

RunInputTerrain F1Support F1Partial F1Friction MAEStrip latency, sLifted delay, s
FirstP with command0.8020.7550.3530.1860.670.29
FirstP+N (2 × 4 zone Fz from the solver)0.8610.8660.6330.1660.520.07
FirstP+N+S0.8600.8530.5990.1500.550.08
SecondP, no command0.8170.7600.3580.1810.640.30
SecondP with command (diagnostic)0.8180.7650.3690.1800.620.27
SecondP+C5 (four block Fz from the map)0.8290.8100.4710.1740.510.12
SecondP+C4 (plus block shear)0.8360.8130.4770.1760.490.11
SecondP+C2 (point map plus block shear)0.8890.9380.8270.1580.360.11

Reading the table. P reproduces across the two runs, with or without the command. The first run's eight zones per foot scored higher than this run's four blocks. The two differ in resolution and in where the numbers come from (the first run read the solver's per-zone forces, which this run showed to be unreliable while walking), so the gap cannot be assigned to one cause. The first run's friction gain from zone shear (0.166 to 0.150) is not reproduced by block shear here (0.174 against 0.176 for blocks, 0.156 against 0.158 for the map); per-point shear does lower the friction error (0.141 against 0.158, question 3). With joints and IMU in the input, neither run found a shear effect on terrain class or support state; without them, this run found one on terrain class (question 2).

11

A new walking policy (parked)

A fourth walking source was trained in Isaac Lab, warm-started from G113, with generated terrain (rough, slopes to 10°, stepping stones, 3–6 cm platforms) and friction 0.1–1.0. The acceptance rule, fixed before the evaluation that mattered: every chunk crossed in at least 16 of 20 baseline courses, then at least 80 % of 40 fresh courses.

StageWhat changedBest checkpointCrossed, of 20No fall, of 20
Baselineexisting policies G113 / G80, same courses–7 / 147 / 14
StartG113 recipe plus generated terrain, warm start200walked at 0.13 m/s and lost speed tracking
Fix 1command curriculum off, terrain promotion by distance walked500 / 100014 / 1113 / 15
Fix 2more stepping stones and gaps, stricter promotion30002
Fix 3from fix 1 at 1000: its terrain mix with a slightly larger plank share (0.20 against 0.15; fix 2 had 0.24), no gaps, smaller policy step, more weight on forward speed5001016

Reading the table. Three different fixes, none within reach of 16 of 20, and the last within chance of the first: parked. The remaining failures are falls on plank chunks and slowing down on terrain. The evaluated checkpoints are not used anywhere in this run.

12

Integrity, approximations, failures

Verified independently

A correction made after the first tables

Edge events lasting exactly 0.30 s (21 of 2,908 in the main test set) were first split by comparing float32 times, which put 13 of them on the short side. The split now uses the integer frame difference (0.30 s or longer counts as long), which puts all 21 on the long side: 828 short and 2,080 long events. Recomputed from the stored predictions, long-event miss rates moved by at most 0.003 and short-event miss rates by up to 0.042 (one leave-one-gait-out model); no verdict changed. The independent recomputation compares float64 times, which rounds 7 of those 21 events to the short side; its long-event miss rates agree with the tables and its short-event miss rates are about 0.005 lower.

Approximations and placeholders

Failures and things not done

Close-up of the simulated sole while walking on a grippy floor, slowed to 0.25×. The corner loading at heel strike and toe-off is visible.
13

Next steps

  1. Record the real sole before any more simulation of it. Ten minutes of standing, hand presses and walking on two surfaces would answer what this run cannot: how load is spread while walking, the per-point range, and whether per-point shear has content of its own.
  2. Decide the foot. If the real K1 sole really carries load on front and back with a groove between, the simulated foot should get rounded edges so that it stops rocking on box corners; that changes what the walking policies see and needs a decision.
  3. Train for robustness to the reading method, not on one setting, and keep block sums as the fallback input: they lose 5 points of terrain F1 and most of the partial-support detection, and they survive a change of sensor.
  4. Put the deployed controller in the training data, and consider a sole-only model where the gait is not known in advance.

Scores are on walking test windows, mean of three seeds unless stated: macro-F1 for terrain class over six terrains and for support state, F1 for partial support, mean absolute error for friction, latencies in seconds, miss rates as shares of events. Related: step 1, first simulation run, step 1, sensor-layout check, support state, the simulated sole up close, step 1 summary, teachers of the fixed-patch experiment.