Does the ordering sampler get the right temperature on a model with a known answer?
No. It placed a known 1300-1410 K transition at 612 K, a factor of 2.1-2.3 too low.
In the log: second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)
confirmedDate 2026-09-21 17:4x, as written in the logrung 4 · DFT2 predictions · 2 result paragraphsEXPERIMENTS.md lines 13913–13931, lines 14011–14030, lines 14327–14389, lines 14583–14626
What E215 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E215.svg).
Pre-registration
(1)
E210 had already started its
nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus
a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that
kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so
it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is
calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to
start while any ece mcmc or search is alive.
no verdict written against it
(2)
The stale e215_failed marker the crash
left behind was one of E210's release conditions, so it would have let E210 through on a
failure; removed with the fix.
no verdict written against it
The pre-registration, as written
E215's fit would have trained on nothing, and a dry run caught it (2026-09-21 14:2x).train_ece.py splits train/valid/test by composition, to stop the same alloy appearing
on both sides. The first Ising label set was equiatomic Mo₂₇Ta₂₇ only, so all 2001 rows fell
in one bucket: 1 distinct compositions -> train 0/0 frames, valid 1/2001, test 0/0. The
queued chain would have run overnight and produced a model fitted on zero training
frames — and E215 is the experiment that decides whether the sampler or the model carries
rung 1's factor of three, so a garbage model there would have been worse than no answer.
Fixed by labelling nine compositions (16–38 B atoms on 54 sites) instead of one, which
costs nothing — the Ising energy is exact at any decoration — conditions the expansion
better, and makes the split work: train 7/1162, valid 1/667, test 1/166. The sweep stays
canonical at 50/50. Caught by dry-running a queued command, not by a test; the test exists
now.
While pinning it, a second assertion of mine was wrong in the interesting direction: I had
asserted a random solution's energy is ~0. It is not. Drawing 27 A and 27 B without
replacement makes unlike pairs slightly more likely than like ones,
P_unlike = 2·27·27/(54·53) = 0.5094, so the exact expectation is 4J(1−2P_unlike) =
−1.44 meV/atom; measured −1.40 over 2000 draws, agreeing to 0.4 standard errors. The
test now asserts that value rather than asserting the bias away.
E215's fit crashed on a latent import, and the queued v7 chain would have died the same
way (2026-09-21 15:2x).ModuleNotFoundError: No module named 'forager' — run as a
script, sys.path[0] is scripts/expansion, not the repo root, so import forager fails.
train_ece.py::ideal_frac_of grew from forager.ladder.lattice import BCC in the
2026-09-20 lattice refactor and no fit had been run since, so the break was invisible:
v5 and v6 were trained before it landed, and my own dry run returns before emit() ever
reaches that line. v7 — the first refit that closes the flywheel on the deployed model —
was queued behind E210 and would have crashed unattended. Fixed by putting the repo root
on sys.path; tests/test_scripts_run_as_scripts.py drives --help on every script a
chain invokes directly, plus a dry run that reaches the import that actually broke.
Two sequencing corrections made at the same time. (1) E210 had already started its
nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus
a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that
kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so
it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is
calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to
start while any ece mcmc or search is alive. (2) The stale e215_failed marker the crash
left behind was one of E210's release conditions, so it would have let E210 through on a
failure; removed with the fix.
Results
EXPERIMENTS.md · line 14327
E215, second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)
The redesigned E215 fitted a nearest-neighbour-only eCE (pairs ≤ 3.0 Å) to the exact Ising
labels and ran the fidelity gate before any sweep. GATE FAILED, on every probe that has
an exact answer:
probe
exact
fitted
error
random 50/50
−1.42
−1.51
−0.09
B2 with 25 % antisites
−17.35
−18.28
−0.93
perfect B2
−76.47
−59.47
+17.00
far-swap cost (total)
611.7 meV (= 32J)
310.8
half
adjacent-swap cost (total)
535.3 meV (= 28J)
370.6
and above the far swap
RMSE against the exact energy on the 24 states the exact sampler visited: 9.7 meV/atom.
The fit's own validation loss said 3.7 meV and its test loss 1.7 — both true and both
useless, because the split put the swept 50/50 composition, which holds every ordered
state in the label set, into the validation bucket: the model never trained on a single
ordered configuration and extrapolated the B2 end by 22 %. That is referee 2's point 8.3
("not stated whether 27/27 is in train, valid or test"), confirmed by the one test that could
see it. An adjacent swap priced above a far one is impossible for any pair Hamiltonian of any
range, so the fitted object was not a pair model at all, whatever its basis said.
For comparison, v5 on the identical probes in its own frame: far swap 533 meV, adjacent
458, ratio 0.86 against the pair-model value 28/32 = 0.875 — v5 prices local defects like a
pair Hamiltonian. The exact-Ising eCE trained without ordered states does not.
Fix and third attempt.train_ece.py --split random (row-wise; for a synthetic exact
Hamiltonian, composition leakage is meaningless and the composition split was actively
harmful). Two fits — nearest-neighbour basis and the default four-shell basis — each gated on
four physical probes (B2 within 0.5 meV, far-swap J₁ = 19.12 ± 0.3, adjacent cheaper than
far, RMSE < 0.5 on the exact-MC states). The nearest-neighbour fit is used if it passes, the
four-shell fit otherwise, and if neither passes E215 is abandoned as uninterpretable rather
than read — which would itself be a result about what this architecture can represent.
Running.
SOB20's frame, read off the paper (2026-09-21 17:5x) — the four Cr-bearing scorecard
systems are comparable after all. Referee 2 blocked them until the source's spin treatment
was known. Sobieraj et al. 2020, §2.1: "Since the difference between enthalpies of mixing
of anti-ferromagnetic and non-magnetic Cr-rich structures was small, the magnetism was not
considered in this study." Non-spin-polarised, VASP PAW without semicore p, PBE, 400 eV,
k-spacing 0.2 Å⁻¹, 682 structures, CV 10.2 meV/atom, canonical MC on 2000 atoms cooled
3000 → 100 K in 100 K steps — so their ODTT values carry a 100 K grid resolution, the
same caveat this record now applies to its own. Their Cr–V ordering is therefore a
non-magnetic effect in their Hamiltonian and one ours can in principle contain. The spin
field on those eight rows is nsp, the blocker is lifted on the evidence, and the field
stays: it was the right question, and it took a paragraph of the source to answer it.
Kim & Widom's ordering energy is not in the paper, and their Hamiltonian makes the bridge
undefined anyway (2026-09-21 18:0x). Referee 2 asked for K&W's own E_random − E_B2 for
Mo–Ta to test whether the nearest-neighbour mapping under-predicts on the reference
calculation itself. The number is not in the text. What the methods section does settle: their
model is a tetrahedron lookup table — every BCC tetrahedron (two "even" vertex sites,
two "odd" body-centre sites, i.e. nearest and next-nearest pairs, triangles and the
four-body term) carries its own DFT energy, plus a polarisation correction — so it is a
many-body Hamiltonian by construction, and "the T_c a NN Ising would give for its ΔE_order"
is as ill-defined for their model as it now is for v5. Their T_c is a canonical swap-move
replica-exchange result with histogram reweighting on a fixed 3.2305 Å lattice; their MAE
21 meV/atom (1 meV with the polarisation term) is what published_error_K uses. The
dE_order_meV: 110 on the scorecard row is this project's inference of what a NN mapping
would need, not their number, and is labelled so. Their table is published with their code;
reading the Mo–Ta tetrahedron energies off it is the remaining way to get E(B2) in their
model, and it stays on the open list.
EXPERIMENTS.md · line 14583
E215 RESULT — the sampler is convicted (2026-09-21 16:4x, real time)
Third design, gate passed first: the nearest-neighbour-basis eCE fitted to the exact Ising
labels (row split) reproduces the exact energy on 24 exact-sampler states with RMSE 0.024
meV/atom, perfect B2 to −0.022, and its far-swap J₁ is 19.122 meV against 19.117; an
adjacent swap is cheaper than a far one, as it must be. So the Hamiltonian handed to
ece mcmc is the exact nearest-neighbour bcc Ising with J = 19.12 meV to two hundredths of a
millielectronvolt. Its thermodynamic T_c is 1410 K; a from-scratch canonical Metropolis
on the same Hamiltonian at the same 6³ cell peaks at 1300 K (500 sweeps) and 1340 K
(1000 sweeps, referee 2).
ece mcmc, same Hamiltonian, same cell, 3000 → 100 → 3000 K, 1000–2000 sweeps per state:
leg
T_c (dE/dT)
T_c (variance)
Var/dEdT
floor
cooling
612 K
678 K
1.67
−76.5 (= exact B2)
heating
612 K
672 K
1.83
−76.5
The sampler places a known 1300–1410 K transition at 612 K: a factor of 2.1–2.3. The
same factor appears in the energies themselves: at 3000 K the sweep reports −3.1 meV/atom
where the exact leading-order high-temperature energy is −5.65 (−4J²/k_BT) and the
exact sampler measures −7.5 at 2600 K; at 1635 K it reports −5.7 against the exact sampler's
−14.7 at 1600 K. The floor is exact — it reaches perfect B2 — so the ground state and the
energy scale are right; what is wrong is the thermal population at every temperature, by
about a factor of two. That is the signature of a Metropolis step that accepts moves with
half the true energy change, or equivalently a temperature twice the one requested —
and it is the same factor of two the variance channel has carried in every run (Var/dEdT ≈ 2
in E200, E207, E214 and here). One convention error would explain all of it: an energy per
PRIM cell (two sites) used where an energy per site was meant, or the reverse. pyeCE's
source is being read for that convention now; the second sampler on v5 itself (running) is
the independent confirmation.
What this means, stated carefully. Every ordering temperature this project has reported
from ece mcmc is about a factor of two low, not three — the factor of three was the
void nearest-neighbour yardstick, and the referees were right to void it. Corrected by the
E215 factor (2.1–2.3, at 85 K resolution), v5's own Hamiltonian would place Mo–Ta at
~1000–1100 K (published 2020 ± 545), MoNbTaW at ~550–600 K (published 600 / 1110),
MoNbTaVW at ~650–700 K (published 742 / 750, different observables). Those are not
results yet — they are what a constant sampler factor would imply — and they are not to be
quoted until the factor is measured on v5 directly by the second sampler and the mechanism
is found in the code. But the direction is unmistakable: with a correct sampler, v5's
ordering temperatures land inside the published bands on all three systems, and the
model that was blamed for two days was carrying a sampler's factor of two on top of its own
honest errors.
The full record
This entry is written in 4 separate places in the log, shown here in log order.
EXPERIMENTS.md · lines 13913–13931
E215's fit would have trained on nothing, and a dry run caught it (2026-09-21 14:2x).train_ece.py splits train/valid/test by composition, to stop the same alloy appearing
on both sides. The first Ising label set was equiatomic Mo₂₇Ta₂₇ only, so all 2001 rows fell
in one bucket: 1 distinct compositions -> train 0/0 frames, valid 1/2001, test 0/0. The
queued chain would have run overnight and produced a model fitted on zero training
frames — and E215 is the experiment that decides whether the sampler or the model carries
rung 1's factor of three, so a garbage model there would have been worse than no answer.
Fixed by labelling nine compositions (16–38 B atoms on 54 sites) instead of one, which
costs nothing — the Ising energy is exact at any decoration — conditions the expansion
better, and makes the split work: train 7/1162, valid 1/667, test 1/166. The sweep stays
canonical at 50/50. Caught by dry-running a queued command, not by a test; the test exists
now.
While pinning it, a second assertion of mine was wrong in the interesting direction: I had
asserted a random solution's energy is ~0. It is not. Drawing 27 A and 27 B without
replacement makes unlike pairs slightly more likely than like ones,
P_unlike = 2·27·27/(54·53) = 0.5094, so the exact expectation is 4J(1−2P_unlike) =
−1.44 meV/atom; measured −1.40 over 2000 draws, agreeing to 0.4 standard errors. The
test now asserts that value rather than asserting the bias away.
EXPERIMENTS.md · lines 14011–14030
E215's fit crashed on a latent import, and the queued v7 chain would have died the same
way (2026-09-21 15:2x).ModuleNotFoundError: No module named 'forager' — run as a
script, sys.path[0] is scripts/expansion, not the repo root, so import forager fails.
train_ece.py::ideal_frac_of grew from forager.ladder.lattice import BCC in the
2026-09-20 lattice refactor and no fit had been run since, so the break was invisible:
v5 and v6 were trained before it landed, and my own dry run returns before emit() ever
reaches that line. v7 — the first refit that closes the flywheel on the deployed model —
was queued behind E210 and would have crashed unattended. Fixed by putting the repo root
on sys.path; tests/test_scripts_run_as_scripts.py drives --help on every script a
chain invokes directly, plus a dry run that reaches the import that actually broke.
Two sequencing corrections made at the same time. (1) E210 had already started its
nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus
a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that
kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so
it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is
calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to
start while any ece mcmc or search is alive. (2) The stale e215_failed marker the crash
left behind was one of E210's release conditions, so it would have let E210 through on a
failure; removed with the fix.
EXPERIMENTS.md · lines 14327–14389
E215, second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)
The redesigned E215 fitted a nearest-neighbour-only eCE (pairs ≤ 3.0 Å) to the exact Ising
labels and ran the fidelity gate before any sweep. GATE FAILED, on every probe that has
an exact answer:
probe
exact
fitted
error
random 50/50
−1.42
−1.51
−0.09
B2 with 25 % antisites
−17.35
−18.28
−0.93
perfect B2
−76.47
−59.47
+17.00
far-swap cost (total)
611.7 meV (= 32J)
310.8
half
adjacent-swap cost (total)
535.3 meV (= 28J)
370.6
and above the far swap
RMSE against the exact energy on the 24 states the exact sampler visited: 9.7 meV/atom.
The fit's own validation loss said 3.7 meV and its test loss 1.7 — both true and both
useless, because the split put the swept 50/50 composition, which holds every ordered
state in the label set, into the validation bucket: the model never trained on a single
ordered configuration and extrapolated the B2 end by 22 %. That is referee 2's point 8.3
("not stated whether 27/27 is in train, valid or test"), confirmed by the one test that could
see it. An adjacent swap priced above a far one is impossible for any pair Hamiltonian of any
range, so the fitted object was not a pair model at all, whatever its basis said.
For comparison, v5 on the identical probes in its own frame: far swap 533 meV, adjacent
458, ratio 0.86 against the pair-model value 28/32 = 0.875 — v5 prices local defects like a
pair Hamiltonian. The exact-Ising eCE trained without ordered states does not.
Fix and third attempt.train_ece.py --split random (row-wise; for a synthetic exact
Hamiltonian, composition leakage is meaningless and the composition split was actively
harmful). Two fits — nearest-neighbour basis and the default four-shell basis — each gated on
four physical probes (B2 within 0.5 meV, far-swap J₁ = 19.12 ± 0.3, adjacent cheaper than
far, RMSE < 0.5 on the exact-MC states). The nearest-neighbour fit is used if it passes, the
four-shell fit otherwise, and if neither passes E215 is abandoned as uninterpretable rather
than read — which would itself be a result about what this architecture can represent.
Running.
SOB20's frame, read off the paper (2026-09-21 17:5x) — the four Cr-bearing scorecard
systems are comparable after all. Referee 2 blocked them until the source's spin treatment
was known. Sobieraj et al. 2020, §2.1: "Since the difference between enthalpies of mixing
of anti-ferromagnetic and non-magnetic Cr-rich structures was small, the magnetism was not
considered in this study." Non-spin-polarised, VASP PAW without semicore p, PBE, 400 eV,
k-spacing 0.2 Å⁻¹, 682 structures, CV 10.2 meV/atom, canonical MC on 2000 atoms cooled
3000 → 100 K in 100 K steps — so their ODTT values carry a 100 K grid resolution, the
same caveat this record now applies to its own. Their Cr–V ordering is therefore a
non-magnetic effect in their Hamiltonian and one ours can in principle contain. The spin
field on those eight rows is nsp, the blocker is lifted on the evidence, and the field
stays: it was the right question, and it took a paragraph of the source to answer it.
Kim & Widom's ordering energy is not in the paper, and their Hamiltonian makes the bridge
undefined anyway (2026-09-21 18:0x). Referee 2 asked for K&W's own E_random − E_B2 for
Mo–Ta to test whether the nearest-neighbour mapping under-predicts on the reference
calculation itself. The number is not in the text. What the methods section does settle: their
model is a tetrahedron lookup table — every BCC tetrahedron (two "even" vertex sites,
two "odd" body-centre sites, i.e. nearest and next-nearest pairs, triangles and the
four-body term) carries its own DFT energy, plus a polarisation correction — so it is a
many-body Hamiltonian by construction, and "the T_c a NN Ising would give for its ΔE_order"
is as ill-defined for their model as it now is for v5. Their T_c is a canonical swap-move
replica-exchange result with histogram reweighting on a fixed 3.2305 Å lattice; their MAE
21 meV/atom (1 meV with the polarisation term) is what published_error_K uses. The
dE_order_meV: 110 on the scorecard row is this project's inference of what a NN mapping
would need, not their number, and is labelled so. Their table is published with their code;
reading the Mo–Ta tetrahedron energies off it is the remaining way to get E(B2) in their
model, and it stays on the open list.
EXPERIMENTS.md · lines 14583–14626
E215 RESULT — the sampler is convicted (2026-09-21 16:4x, real time)
Third design, gate passed first: the nearest-neighbour-basis eCE fitted to the exact Ising
labels (row split) reproduces the exact energy on 24 exact-sampler states with RMSE 0.024
meV/atom, perfect B2 to −0.022, and its far-swap J₁ is 19.122 meV against 19.117; an
adjacent swap is cheaper than a far one, as it must be. So the Hamiltonian handed to
ece mcmc is the exact nearest-neighbour bcc Ising with J = 19.12 meV to two hundredths of a
millielectronvolt. Its thermodynamic T_c is 1410 K; a from-scratch canonical Metropolis
on the same Hamiltonian at the same 6³ cell peaks at 1300 K (500 sweeps) and 1340 K
(1000 sweeps, referee 2).
ece mcmc, same Hamiltonian, same cell, 3000 → 100 → 3000 K, 1000–2000 sweeps per state:
leg
T_c (dE/dT)
T_c (variance)
Var/dEdT
floor
cooling
612 K
678 K
1.67
−76.5 (= exact B2)
heating
612 K
672 K
1.83
−76.5
The sampler places a known 1300–1410 K transition at 612 K: a factor of 2.1–2.3. The
same factor appears in the energies themselves: at 3000 K the sweep reports −3.1 meV/atom
where the exact leading-order high-temperature energy is −5.65 (−4J²/k_BT) and the
exact sampler measures −7.5 at 2600 K; at 1635 K it reports −5.7 against the exact sampler's
−14.7 at 1600 K. The floor is exact — it reaches perfect B2 — so the ground state and the
energy scale are right; what is wrong is the thermal population at every temperature, by
about a factor of two. That is the signature of a Metropolis step that accepts moves with
half the true energy change, or equivalently a temperature twice the one requested —
and it is the same factor of two the variance channel has carried in every run (Var/dEdT ≈ 2
in E200, E207, E214 and here). One convention error would explain all of it: an energy per
PRIM cell (two sites) used where an energy per site was meant, or the reverse. pyeCE's
source is being read for that convention now; the second sampler on v5 itself (running) is
the independent confirmation.
What this means, stated carefully. Every ordering temperature this project has reported
from ece mcmc is about a factor of two low, not three — the factor of three was the
void nearest-neighbour yardstick, and the referees were right to void it. Corrected by the
E215 factor (2.1–2.3, at 85 K resolution), v5's own Hamiltonian would place Mo–Ta at
~1000–1100 K (published 2020 ± 545), MoNbTaW at ~550–600 K (published 600 / 1110),
MoNbTaVW at ~650–700 K (published 742 / 750, different observables). Those are not
results yet — they are what a constant sampler factor would imply — and they are not to be
quoted until the factor is measured on v5 directly by the second sampler and the mechanism
is found in the code. But the direction is unmistakable: with a correct sampler, v5's
ordering temperatures land inside the published bands on all three systems, and the
model that was blamed for two days was carrying a sampler's factor of two on top of its own
honest errors.
Related entries
E210 — the reversible protocol on every scorecard system (2026-09-20 23:39; queued behind E207)
E214 — pre-registered before it runs (2026-09-21 07:4x)
E200 — is v5's Mo–Ta T_c finite-size? (2026-09-19 00:01; queued)
E207 — does the ordering sweep equilibrate? Heating vs cooling at 10–30× the sweeps (2026-09-20…