Is the energy model badly wrong outside the compositions it was trained on?
Withdrawn. The comparison mixed conventions; done like-for-like, the model's error is 3 meV/atom, better than its stated 5.67.
In the log: WITHDRAWN: the expansion is accurate; the comparison that condemned it was not
withdrawnDate not stated in the log; it was written between the commit of 2026-09-13 08:16 and the first commit that contains it, 2026-09-16 02:04rung 2 · hull, MACE0 predictions · 1 result paragraphEXPERIMENTS.md lines 3547–3599
What E65 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E65.svg).
Results
EXPERIMENTS.md · line 3547
E65 — WITHDRAWN: the expansion is accurate; the comparison that condemned it was not
This entry claimed the cluster expansion degrades badly outside its fitting box - four to
six times its stated error at the compositions the generator reaches, and up to 563
meV/atom with the wrong sign at binaries - and that the leverage guard failed to detect it.
All of that was an artefact of how the comparison was set up, and it is withdrawn.
Two errors compounded:
The expansion predicts the energy of a structure on a fixed ideal lattice at
3.2935 A. It was compared against relaxed_bcc_energy, which relaxes the cell to its
own equilibrium. The difference between those is the relaxation energy - a real
quantity, measured in E59 at 16 to 112 meV/atom - not a model error.
The expansion's output had been moved onto equilibrium references by
correct_references (E55), and was compared against MACE values computed against
shared-lattice references. Those differ by up to 505 meV/atom for vanadium alone.
Repeated with the same lattice, the same references and no relaxation on either side, so
that the only difference is the model:
pair
expansion
MACE
error
Hf-Zr
+443
+444
-1
Nb-Ta
+2
+5
-3
Mo-Ta
-55
-54
-1
Mo-W
+189
+200
-10
V-Zr
+241
+241
-1
Mean absolute error 3 meV/atom, against a stated cross-validation error of 5.67. The
expansion is doing better than it claims, at exactly the binary compositions this entry
accused it of failing at.
The number that started it - Hf-Zr at +443 meV/atom, read as "two elements that mix freely
predicted to separate" - is not a chemical statement at all. It is the elastic cost of
holding hafnium and zirconium, which want lattice constants of 3.565 and 3.600 A, at
3.2935. That is the same strain E53 found charged to every mixing energy in the project,
and it is still present in any quantity quoted against shared-lattice references.
What was actually wrong was the reading, and the lesson is narrow and repeatable: a
number from this expansion is only comparable to another number computed on the same
lattice, with the same references, and with the same treatment of relaxation. Three
conventions, and getting any one of them wrong produces a difference of hundreds of meV
that looks exactly like a discovery.
The pair-interaction matrix in data/pair_interactions.npz is therefore valid as what it
is - ideal-lattice mixing energies against equilibrium references - but it is dominated by
elastic strain rather than chemistry, so it is still not the ordering descriptor it was
built to be. A useful version would evaluate each pair at its own equilibrium lattice.
Nothing else in the record depends on this entry. The off-lattice driving forces, the
rankings, the fly's results and the candidates were all computed consistently and are
untouched.
The full record
EXPERIMENTS.md · lines 3547–3599
E65 — WITHDRAWN: the expansion is accurate; the comparison that condemned it was not
This entry claimed the cluster expansion degrades badly outside its fitting box - four to
six times its stated error at the compositions the generator reaches, and up to 563
meV/atom with the wrong sign at binaries - and that the leverage guard failed to detect it.
All of that was an artefact of how the comparison was set up, and it is withdrawn.
Two errors compounded:
The expansion predicts the energy of a structure on a fixed ideal lattice at
3.2935 A. It was compared against relaxed_bcc_energy, which relaxes the cell to its
own equilibrium. The difference between those is the relaxation energy - a real
quantity, measured in E59 at 16 to 112 meV/atom - not a model error.
The expansion's output had been moved onto equilibrium references by
correct_references (E55), and was compared against MACE values computed against
shared-lattice references. Those differ by up to 505 meV/atom for vanadium alone.
Repeated with the same lattice, the same references and no relaxation on either side, so
that the only difference is the model:
pair
expansion
MACE
error
Hf-Zr
+443
+444
-1
Nb-Ta
+2
+5
-3
Mo-Ta
-55
-54
-1
Mo-W
+189
+200
-10
V-Zr
+241
+241
-1
Mean absolute error 3 meV/atom, against a stated cross-validation error of 5.67. The
expansion is doing better than it claims, at exactly the binary compositions this entry
accused it of failing at.
The number that started it - Hf-Zr at +443 meV/atom, read as "two elements that mix freely
predicted to separate" - is not a chemical statement at all. It is the elastic cost of
holding hafnium and zirconium, which want lattice constants of 3.565 and 3.600 A, at
3.2935. That is the same strain E53 found charged to every mixing energy in the project,
and it is still present in any quantity quoted against shared-lattice references.
What was actually wrong was the reading, and the lesson is narrow and repeatable: a
number from this expansion is only comparable to another number computed on the same
lattice, with the same references, and with the same treatment of relaxation. Three
conventions, and getting any one of them wrong produces a difference of hundreds of meV
that looks exactly like a discovery.
The pair-interaction matrix in data/pair_interactions.npz is therefore valid as what it
is - ideal-lattice mixing energies against equilibrium references - but it is dominated by
elastic strain rather than chemistry, so it is still not the ordering descriptor it was
built to be. A useful version would evaluate each pair at its own equilibrium lattice.
Nothing else in the record depends on this entry. The off-lattice driving forces, the
rankings, the fly's results and the candidates were all computed consistently and are
untouched.
Related entries
E59 — The question that decides costs four milliseconds, and twenty thousand compositions say…
E55 — The reference fix reached the source and never reached the data
E53 — The 2.2x was three things, none of them the physics; and every mixing energy was half…