Experiments · E231

Does recomputing every training energy at its true lowest point improve the energy model?

Partly. Fits became steadier (seed spread 8.5 to 6.3 meV/atom), but the overall error barely moved and the Mo–Ta miss stayed.

In the log: the E228 relabel, one variable: v5's rows, v5's recipe, corrected labels (pre-registered 2026-09-22 19:5x)

mixedDate 2026-09-22 19:5x, as written in the logrung 2 · hull, MACE4 predictions · 1 result paragraphEXPERIMENTS.md lines 15807–15824, lines 15957–15963
exp E231 diagram
What E231 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E231.svg).

Pre-registration

  1. (1)
    Removing δ's per-frame scatter (sd 7.3–7.7 meV/atom, ≈ 26 % of v5's training MSE) drops the ensemble-mean training RMSE from 13.6 to ≤ 12.5 and validation RMSE from 17.7 to ≤ 16.7.
    falsified(RMSE ≤ 12 …
  2. (2)
    Seed spread on the unbiased set falls from 8.5 to ≤ 7.0 meV/atom.
    confirmed(spread ≤ 7 …
  3. (3)
    The Mo–Ta error does not move: the four Mo–Ta cells' ensemble mean stays within ±7 of +34 (E228: the relabel nets ≈ 0 on Mo₅₀Ta₅₀).
    confirmed(Mo–Ta does not move); …
  4. (4)
    Held-48 under the new labels: v5r ensemble MAE ≤ v5's + 0.5. If
    confirmed
The pre-registration, as written

E231 — the E228 relabel, one variable: v5's rows, v5's recipe, corrected labels (pre-registered 2026-09-22 19:5x)

Labels: every pass-1 row patched to the true lattice-constant minimum (rhea_labels.py --refine-minimum; δ per frame, label lowers by exactly δ; agrees with E228 to 0.0000 meV/atom), the pure frames all refined (Hf and Ti references sit 0.23 Å strained, so no strain limit there), pass 2 with median pure references (E228's winner's curse), the synthetic Zr row carried unchanged. Rows: exactly v5's 4,326 indices — the 0.10 Å strain cut would move with the new a_relaxed and change the set, so it is pinned. Fit: v5's recipe (embedding 3, 16-16), seeds 0, 1, 2 → ece_v5r_seed*. Held-out rows scored under the new labels for both ensembles. Predictions. (1) Removing δ's per-frame scatter (sd 7.3–7.7 meV/atom, ≈ 26 % of v5's training MSE) drops the ensemble-mean training RMSE from 13.6 to ≤ 12.5 and validation RMSE from 17.7 to ≤ 16.7. (2) Seed spread on the unbiased set falls from 8.5 to ≤ 7.0 meV/atom. (3) The Mo–Ta error does not move: the four Mo–Ta cells' ensemble mean stays within ±7 of +34 (E228: the relabel nets ≈ 0 on Mo₅₀Ta₅₀). (4) Held-48 under the new labels: v5r ensemble MAE ≤ v5's + 0.5. If (1) and (2) both come within ±0.5 of v5, δ's scatter was not a material part of the fit error and the relabel is kept for correctness alone. Chain runs/e231_chain.sh, behind E229 (≈ 5 h of MACE at 2 threads, sleeps while a search runs; then three fits).

Results

EXPERIMENTS.md · line 15957

E231 RESULT (12:1x). Relabelled v5 rows (true minimum, median references), v5's recipe, 3 seeds, scored against the three v5 seeds with held-out rows under the new labels: eleven-cell MAE 22.1 (v5 20.6), B2 Mo–Ta +38.3 (+29.5), training RMSE mean 13.3 (13.6), validation 17.6 (17.7), held-48 9.7 (10.5), per-cell seed spread 6.3 (8.5). (1) falsified (RMSE ≤ 12.5 and ≤ 16.7 not met); (2) confirmed (spread ≤ 7.0); (3) confirmed (Mo–Ta does not move); (4) confirmed. The relabel is kept for correctness and steadier fits; it is not the Mo–Ta lever — capacity is (E229).

The full record

This entry is written in 2 separate places in the log, shown here in log order.

EXPERIMENTS.md · lines 15807–15824

E231 — the E228 relabel, one variable: v5's rows, v5's recipe, corrected labels (pre-registered 2026-09-22 19:5x)

Labels: every pass-1 row patched to the true lattice-constant minimum (rhea_labels.py --refine-minimum; δ per frame, label lowers by exactly δ; agrees with E228 to 0.0000 meV/atom), the pure frames all refined (Hf and Ti references sit 0.23 Å strained, so no strain limit there), pass 2 with median pure references (E228's winner's curse), the synthetic Zr row carried unchanged. Rows: exactly v5's 4,326 indices — the 0.10 Å strain cut would move with the new a_relaxed and change the set, so it is pinned. Fit: v5's recipe (embedding 3, 16-16), seeds 0, 1, 2 → ece_v5r_seed*. Held-out rows scored under the new labels for both ensembles. Predictions. (1) Removing δ's per-frame scatter (sd 7.3–7.7 meV/atom, ≈ 26 % of v5's training MSE) drops the ensemble-mean training RMSE from 13.6 to ≤ 12.5 and validation RMSE from 17.7 to ≤ 16.7. (2) Seed spread on the unbiased set falls from 8.5 to ≤ 7.0 meV/atom. (3) The Mo–Ta error does not move: the four Mo–Ta cells' ensemble mean stays within ±7 of +34 (E228: the relabel nets ≈ 0 on Mo₅₀Ta₅₀). (4) Held-48 under the new labels: v5r ensemble MAE ≤ v5's + 0.5. If (1) and (2) both come within ±0.5 of v5, δ's scatter was not a material part of the fit error and the relabel is kept for correctness alone. Chain runs/e231_chain.sh, behind E229 (≈ 5 h of MACE at 2 threads, sleeps while a search runs; then three fits).

EXPERIMENTS.md · lines 15957–15963

E231 RESULT (12:1x). Relabelled v5 rows (true minimum, median references), v5's recipe, 3 seeds, scored against the three v5 seeds with held-out rows under the new labels: eleven-cell MAE 22.1 (v5 20.6), B2 Mo–Ta +38.3 (+29.5), training RMSE mean 13.3 (13.6), validation 17.6 (17.7), held-48 9.7 (10.5), per-cell seed spread 6.3 (8.5). (1) falsified (RMSE ≤ 12.5 and ≤ 16.7 not met); (2) confirmed (spread ≤ 7.0); (3) confirmed (Mo–Ta does not move); (4) confirmed. The relabel is kept for correctness and steadier fits; it is not the Mo–Ta lever — capacity is (E229).

Related entries

Built with PRISMWebsite and visualizations made using Claude