Experiments · E215

Does the ordering sampler get the right temperature on a model with a known answer?

No. It placed a known 1300-1410 K transition at 612 K, a factor of 2.1-2.3 too low.

In the log: second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)

confirmedDate 2026-09-21 17:4x, as written in the logrung 4 · DFT2 predictions · 2 result paragraphsEXPERIMENTS.md lines 13913–13931, lines 14011–14030, lines 14327–14389, lines 14583–14626
exp E215 diagram
What E215 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E215.svg).

Pre-registration

  1. (1)
    E210 had already started its nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to start while any ece mcmc or search is alive.
    no verdict written against it
  2. (2)
    The stale e215_failed marker the crash left behind was one of E210's release conditions, so it would have let E210 through on a failure; removed with the fix.
    no verdict written against it
The pre-registration, as written

E215's fit would have trained on nothing, and a dry run caught it (2026-09-21 14:2x). train_ece.py splits train/valid/test by composition, to stop the same alloy appearing on both sides. The first Ising label set was equiatomic Mo₂₇Ta₂₇ only, so all 2001 rows fell in one bucket: 1 distinct compositions -> train 0/0 frames, valid 1/2001, test 0/0. The queued chain would have run overnight and produced a model fitted on zero training frames — and E215 is the experiment that decides whether the sampler or the model carries rung 1's factor of three, so a garbage model there would have been worse than no answer. Fixed by labelling nine compositions (16–38 B atoms on 54 sites) instead of one, which costs nothing — the Ising energy is exact at any decoration — conditions the expansion better, and makes the split work: train 7/1162, valid 1/667, test 1/166. The sweep stays canonical at 50/50. Caught by dry-running a queued command, not by a test; the test exists now.

While pinning it, a second assertion of mine was wrong in the interesting direction: I had asserted a random solution's energy is ~0. It is not. Drawing 27 A and 27 B without replacement makes unlike pairs slightly more likely than like ones, P_unlike = 2·27·27/(54·53) = 0.5094, so the exact expectation is 4J(1−2P_unlike) = −1.44 meV/atom; measured −1.40 over 2000 draws, agreeing to 0.4 standard errors. The test now asserts that value rather than asserting the bias away.

E215's fit crashed on a latent import, and the queued v7 chain would have died the same way (2026-09-21 15:2x). ModuleNotFoundError: No module named 'forager' — run as a script, sys.path[0] is scripts/expansion, not the repo root, so import forager fails. train_ece.py::ideal_frac_of grew from forager.ladder.lattice import BCC in the 2026-09-20 lattice refactor and no fit had been run since, so the break was invisible: v5 and v6 were trained before it landed, and my own dry run returns before emit() ever reaches that line. v7 — the first refit that closes the flywheel on the deployed model — was queued behind E210 and would have crashed unattended. Fixed by putting the repo root on sys.path; tests/test_scripts_run_as_scripts.py drives --help on every script a chain invokes directly, plus a dry run that reaches the import that actually broke.

Two sequencing corrections made at the same time. (1) E210 had already started its nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to start while any ece mcmc or search is alive. (2) The stale e215_failed marker the crash left behind was one of E210's release conditions, so it would have let E210 through on a failure; removed with the fix.

Results

EXPERIMENTS.md · line 14327

E215, second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)

The redesigned E215 fitted a nearest-neighbour-only eCE (pairs ≤ 3.0 Å) to the exact Ising labels and ran the fidelity gate before any sweep. GATE FAILED, on every probe that has an exact answer:

probe exact fitted error
random 50/50 −1.42 −1.51 −0.09
B2 with 25 % antisites −17.35 −18.28 −0.93
perfect B2 −76.47 −59.47 +17.00
far-swap cost (total) 611.7 meV (= 32J) 310.8 half
adjacent-swap cost (total) 535.3 meV (= 28J) 370.6 and above the far swap

RMSE against the exact energy on the 24 states the exact sampler visited: 9.7 meV/atom. The fit's own validation loss said 3.7 meV and its test loss 1.7 — both true and both useless, because the split put the swept 50/50 composition, which holds every ordered state in the label set, into the validation bucket: the model never trained on a single ordered configuration and extrapolated the B2 end by 22 %. That is referee 2's point 8.3 ("not stated whether 27/27 is in train, valid or test"), confirmed by the one test that could see it. An adjacent swap priced above a far one is impossible for any pair Hamiltonian of any range, so the fitted object was not a pair model at all, whatever its basis said.

For comparison, v5 on the identical probes in its own frame: far swap 533 meV, adjacent 458, ratio 0.86 against the pair-model value 28/32 = 0.875 — v5 prices local defects like a pair Hamiltonian. The exact-Ising eCE trained without ordered states does not.

Fix and third attempt. train_ece.py --split random (row-wise; for a synthetic exact Hamiltonian, composition leakage is meaningless and the composition split was actively harmful). Two fits — nearest-neighbour basis and the default four-shell basis — each gated on four physical probes (B2 within 0.5 meV, far-swap J₁ = 19.12 ± 0.3, adjacent cheaper than far, RMSE < 0.5 on the exact-MC states). The nearest-neighbour fit is used if it passes, the four-shell fit otherwise, and if neither passes E215 is abandoned as uninterpretable rather than read — which would itself be a result about what this architecture can represent. Running.

SOB20's frame, read off the paper (2026-09-21 17:5x) — the four Cr-bearing scorecard systems are comparable after all. Referee 2 blocked them until the source's spin treatment was known. Sobieraj et al. 2020, §2.1: "Since the difference between enthalpies of mixing of anti-ferromagnetic and non-magnetic Cr-rich structures was small, the magnetism was not considered in this study." Non-spin-polarised, VASP PAW without semicore p, PBE, 400 eV, k-spacing 0.2 Å⁻¹, 682 structures, CV 10.2 meV/atom, canonical MC on 2000 atoms cooled 3000 → 100 K in 100 K steps — so their ODTT values carry a 100 K grid resolution, the same caveat this record now applies to its own. Their Cr–V ordering is therefore a non-magnetic effect in their Hamiltonian and one ours can in principle contain. The spin field on those eight rows is nsp, the blocker is lifted on the evidence, and the field stays: it was the right question, and it took a paragraph of the source to answer it.

Kim & Widom's ordering energy is not in the paper, and their Hamiltonian makes the bridge undefined anyway (2026-09-21 18:0x). Referee 2 asked for K&W's own E_random − E_B2 for Mo–Ta to test whether the nearest-neighbour mapping under-predicts on the reference calculation itself. The number is not in the text. What the methods section does settle: their model is a tetrahedron lookup table — every BCC tetrahedron (two "even" vertex sites, two "odd" body-centre sites, i.e. nearest and next-nearest pairs, triangles and the four-body term) carries its own DFT energy, plus a polarisation correction — so it is a many-body Hamiltonian by construction, and "the T_c a NN Ising would give for its ΔE_order" is as ill-defined for their model as it now is for v5. Their T_c is a canonical swap-move replica-exchange result with histogram reweighting on a fixed 3.2305 Å lattice; their MAE 21 meV/atom (1 meV with the polarisation term) is what published_error_K uses. The dE_order_meV: 110 on the scorecard row is this project's inference of what a NN mapping would need, not their number, and is labelled so. Their table is published with their code; reading the Mo–Ta tetrahedron energies off it is the remaining way to get E(B2) in their model, and it stays on the open list.

EXPERIMENTS.md · line 14583

E215 RESULT — the sampler is convicted (2026-09-21 16:4x, real time)

Third design, gate passed first: the nearest-neighbour-basis eCE fitted to the exact Ising labels (row split) reproduces the exact energy on 24 exact-sampler states with RMSE 0.024 meV/atom, perfect B2 to −0.022, and its far-swap J₁ is 19.122 meV against 19.117; an adjacent swap is cheaper than a far one, as it must be. So the Hamiltonian handed to ece mcmc is the exact nearest-neighbour bcc Ising with J = 19.12 meV to two hundredths of a millielectronvolt. Its thermodynamic T_c is 1410 K; a from-scratch canonical Metropolis on the same Hamiltonian at the same 6³ cell peaks at 1300 K (500 sweeps) and 1340 K (1000 sweeps, referee 2).

ece mcmc, same Hamiltonian, same cell, 3000 → 100 → 3000 K, 1000–2000 sweeps per state:

leg T_c (dE/dT) T_c (variance) Var/dEdT floor
cooling 612 K 678 K 1.67 −76.5 (= exact B2)
heating 612 K 672 K 1.83 −76.5

The sampler places a known 1300–1410 K transition at 612 K: a factor of 2.1–2.3. The same factor appears in the energies themselves: at 3000 K the sweep reports −3.1 meV/atom where the exact leading-order high-temperature energy is −5.65 (−4J²/k_BT) and the exact sampler measures −7.5 at 2600 K; at 1635 K it reports −5.7 against the exact sampler's −14.7 at 1600 K. The floor is exact — it reaches perfect B2 — so the ground state and the energy scale are right; what is wrong is the thermal population at every temperature, by about a factor of two. That is the signature of a Metropolis step that accepts moves with half the true energy change, or equivalently a temperature twice the one requested — and it is the same factor of two the variance channel has carried in every run (Var/dEdT ≈ 2 in E200, E207, E214 and here). One convention error would explain all of it: an energy per PRIM cell (two sites) used where an energy per site was meant, or the reverse. pyeCE's source is being read for that convention now; the second sampler on v5 itself (running) is the independent confirmation.

What this means, stated carefully. Every ordering temperature this project has reported from ece mcmc is about a factor of two low, not three — the factor of three was the void nearest-neighbour yardstick, and the referees were right to void it. Corrected by the E215 factor (2.1–2.3, at 85 K resolution), v5's own Hamiltonian would place Mo–Ta at ~1000–1100 K (published 2020 ± 545), MoNbTaW at ~550–600 K (published 600 / 1110), MoNbTaVW at ~650–700 K (published 742 / 750, different observables). Those are not results yet — they are what a constant sampler factor would imply — and they are not to be quoted until the factor is measured on v5 directly by the second sampler and the mechanism is found in the code. But the direction is unmistakable: with a correct sampler, v5's ordering temperatures land inside the published bands on all three systems, and the model that was blamed for two days was carrying a sampler's factor of two on top of its own honest errors.

The full record

This entry is written in 4 separate places in the log, shown here in log order.

EXPERIMENTS.md · lines 13913–13931

E215's fit would have trained on nothing, and a dry run caught it (2026-09-21 14:2x). train_ece.py splits train/valid/test by composition, to stop the same alloy appearing on both sides. The first Ising label set was equiatomic Mo₂₇Ta₂₇ only, so all 2001 rows fell in one bucket: 1 distinct compositions -> train 0/0 frames, valid 1/2001, test 0/0. The queued chain would have run overnight and produced a model fitted on zero training frames — and E215 is the experiment that decides whether the sampler or the model carries rung 1's factor of three, so a garbage model there would have been worse than no answer. Fixed by labelling nine compositions (16–38 B atoms on 54 sites) instead of one, which costs nothing — the Ising energy is exact at any decoration — conditions the expansion better, and makes the split work: train 7/1162, valid 1/667, test 1/166. The sweep stays canonical at 50/50. Caught by dry-running a queued command, not by a test; the test exists now.

While pinning it, a second assertion of mine was wrong in the interesting direction: I had asserted a random solution's energy is ~0. It is not. Drawing 27 A and 27 B without replacement makes unlike pairs slightly more likely than like ones, P_unlike = 2·27·27/(54·53) = 0.5094, so the exact expectation is 4J(1−2P_unlike) = −1.44 meV/atom; measured −1.40 over 2000 draws, agreeing to 0.4 standard errors. The test now asserts that value rather than asserting the bias away.

EXPERIMENTS.md · lines 14011–14030

E215's fit crashed on a latent import, and the queued v7 chain would have died the same way (2026-09-21 15:2x). ModuleNotFoundError: No module named 'forager' — run as a script, sys.path[0] is scripts/expansion, not the repo root, so import forager fails. train_ece.py::ideal_frac_of grew from forager.ladder.lattice import BCC in the 2026-09-20 lattice refactor and no fit had been run since, so the break was invisible: v5 and v6 were trained before it landed, and my own dry run returns before emit() ever reaches that line. v7 — the first refit that closes the flywheel on the deployed model — was queued behind E210 and would have crashed unattended. Fixed by putting the repo root on sys.path; tests/test_scripts_run_as_scripts.py drives --help on every script a chain invokes directly, plus a dry run that reaches the import that actually broke.

Two sequencing corrections made at the same time. (1) E210 had already started its nine-system scan the moment E214 finished, which would have put two Monte Carlo jobs plus a 6-rank DFT on a machine whose budget is one search plus pw.x — the configuration that kernel-panicked it on 2026-09-18. E210 is stopped (its first system had no done marker, so it loses nothing) and re-gated behind E215, because a nine-system scan before the sampler is calibrated produces nine numbers in unknown units. Its per-system gate now also refuses to start while any ece mcmc or search is alive. (2) The stale e215_failed marker the crash left behind was one of E210's release conditions, so it would have let E210 through on a failure; removed with the fix.

EXPERIMENTS.md · lines 14327–14389

E215, second attempt: the gate failed, and the failure is the finding (2026-09-21 17:4x)

The redesigned E215 fitted a nearest-neighbour-only eCE (pairs ≤ 3.0 Å) to the exact Ising labels and ran the fidelity gate before any sweep. GATE FAILED, on every probe that has an exact answer:

probe exact fitted error
random 50/50 −1.42 −1.51 −0.09
B2 with 25 % antisites −17.35 −18.28 −0.93
perfect B2 −76.47 −59.47 +17.00
far-swap cost (total) 611.7 meV (= 32J) 310.8 half
adjacent-swap cost (total) 535.3 meV (= 28J) 370.6 and above the far swap

RMSE against the exact energy on the 24 states the exact sampler visited: 9.7 meV/atom. The fit's own validation loss said 3.7 meV and its test loss 1.7 — both true and both useless, because the split put the swept 50/50 composition, which holds every ordered state in the label set, into the validation bucket: the model never trained on a single ordered configuration and extrapolated the B2 end by 22 %. That is referee 2's point 8.3 ("not stated whether 27/27 is in train, valid or test"), confirmed by the one test that could see it. An adjacent swap priced above a far one is impossible for any pair Hamiltonian of any range, so the fitted object was not a pair model at all, whatever its basis said.

For comparison, v5 on the identical probes in its own frame: far swap 533 meV, adjacent 458, ratio 0.86 against the pair-model value 28/32 = 0.875 — v5 prices local defects like a pair Hamiltonian. The exact-Ising eCE trained without ordered states does not.

Fix and third attempt. train_ece.py --split random (row-wise; for a synthetic exact Hamiltonian, composition leakage is meaningless and the composition split was actively harmful). Two fits — nearest-neighbour basis and the default four-shell basis — each gated on four physical probes (B2 within 0.5 meV, far-swap J₁ = 19.12 ± 0.3, adjacent cheaper than far, RMSE < 0.5 on the exact-MC states). The nearest-neighbour fit is used if it passes, the four-shell fit otherwise, and if neither passes E215 is abandoned as uninterpretable rather than read — which would itself be a result about what this architecture can represent. Running.

SOB20's frame, read off the paper (2026-09-21 17:5x) — the four Cr-bearing scorecard systems are comparable after all. Referee 2 blocked them until the source's spin treatment was known. Sobieraj et al. 2020, §2.1: "Since the difference between enthalpies of mixing of anti-ferromagnetic and non-magnetic Cr-rich structures was small, the magnetism was not considered in this study." Non-spin-polarised, VASP PAW without semicore p, PBE, 400 eV, k-spacing 0.2 Å⁻¹, 682 structures, CV 10.2 meV/atom, canonical MC on 2000 atoms cooled 3000 → 100 K in 100 K steps — so their ODTT values carry a 100 K grid resolution, the same caveat this record now applies to its own. Their Cr–V ordering is therefore a non-magnetic effect in their Hamiltonian and one ours can in principle contain. The spin field on those eight rows is nsp, the blocker is lifted on the evidence, and the field stays: it was the right question, and it took a paragraph of the source to answer it.

Kim & Widom's ordering energy is not in the paper, and their Hamiltonian makes the bridge undefined anyway (2026-09-21 18:0x). Referee 2 asked for K&W's own E_random − E_B2 for Mo–Ta to test whether the nearest-neighbour mapping under-predicts on the reference calculation itself. The number is not in the text. What the methods section does settle: their model is a tetrahedron lookup table — every BCC tetrahedron (two "even" vertex sites, two "odd" body-centre sites, i.e. nearest and next-nearest pairs, triangles and the four-body term) carries its own DFT energy, plus a polarisation correction — so it is a many-body Hamiltonian by construction, and "the T_c a NN Ising would give for its ΔE_order" is as ill-defined for their model as it now is for v5. Their T_c is a canonical swap-move replica-exchange result with histogram reweighting on a fixed 3.2305 Å lattice; their MAE 21 meV/atom (1 meV with the polarisation term) is what published_error_K uses. The dE_order_meV: 110 on the scorecard row is this project's inference of what a NN mapping would need, not their number, and is labelled so. Their table is published with their code; reading the Mo–Ta tetrahedron energies off it is the remaining way to get E(B2) in their model, and it stays on the open list.

EXPERIMENTS.md · lines 14583–14626

E215 RESULT — the sampler is convicted (2026-09-21 16:4x, real time)

Third design, gate passed first: the nearest-neighbour-basis eCE fitted to the exact Ising labels (row split) reproduces the exact energy on 24 exact-sampler states with RMSE 0.024 meV/atom, perfect B2 to −0.022, and its far-swap J₁ is 19.122 meV against 19.117; an adjacent swap is cheaper than a far one, as it must be. So the Hamiltonian handed to ece mcmc is the exact nearest-neighbour bcc Ising with J = 19.12 meV to two hundredths of a millielectronvolt. Its thermodynamic T_c is 1410 K; a from-scratch canonical Metropolis on the same Hamiltonian at the same 6³ cell peaks at 1300 K (500 sweeps) and 1340 K (1000 sweeps, referee 2).

ece mcmc, same Hamiltonian, same cell, 3000 → 100 → 3000 K, 1000–2000 sweeps per state:

leg T_c (dE/dT) T_c (variance) Var/dEdT floor
cooling 612 K 678 K 1.67 −76.5 (= exact B2)
heating 612 K 672 K 1.83 −76.5

The sampler places a known 1300–1410 K transition at 612 K: a factor of 2.1–2.3. The same factor appears in the energies themselves: at 3000 K the sweep reports −3.1 meV/atom where the exact leading-order high-temperature energy is −5.65 (−4J²/k_BT) and the exact sampler measures −7.5 at 2600 K; at 1635 K it reports −5.7 against the exact sampler's −14.7 at 1600 K. The floor is exact — it reaches perfect B2 — so the ground state and the energy scale are right; what is wrong is the thermal population at every temperature, by about a factor of two. That is the signature of a Metropolis step that accepts moves with half the true energy change, or equivalently a temperature twice the one requested — and it is the same factor of two the variance channel has carried in every run (Var/dEdT ≈ 2 in E200, E207, E214 and here). One convention error would explain all of it: an energy per PRIM cell (two sites) used where an energy per site was meant, or the reverse. pyeCE's source is being read for that convention now; the second sampler on v5 itself (running) is the independent confirmation.

What this means, stated carefully. Every ordering temperature this project has reported from ece mcmc is about a factor of two low, not three — the factor of three was the void nearest-neighbour yardstick, and the referees were right to void it. Corrected by the E215 factor (2.1–2.3, at 85 K resolution), v5's own Hamiltonian would place Mo–Ta at ~1000–1100 K (published 2020 ± 545), MoNbTaW at ~550–600 K (published 600 / 1110), MoNbTaVW at ~650–700 K (published 742 / 750, different observables). Those are not results yet — they are what a constant sampler factor would imply — and they are not to be quoted until the factor is measured on v5 directly by the second sampler and the mechanism is found in the code. But the direction is unmistakable: with a correct sampler, v5's ordering temperatures land inside the published bands on all three systems, and the model that was blamed for two days was carrying a sampler's factor of two on top of its own honest errors.

Related entries

Built with PRISMWebsite and visualizations made using Claude