Experiments · E130

Is the cheap energy model much worse against quantum calculations than its advertised error?

No. With lattice size and atom positions matched, its error is 8.1 meV/atom, below the advertised 9.09; the earlier 44.6 was a mismatch.

In the log: How to measure rung 0 against DFT without repeating E129's mistake

mixedDate not stated in the log; it was written between the commit of 2026-09-16 09:57 and the first commit that contains it, 2026-09-16 10:56rung 4 · DFT0 predictions · 2 result paragraphsEXPERIMENTS.md lines 7798–7831, lines 7833–7865, lines 7867–7910
exp E130 diagram
What E130 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E130.svg).

Results

EXPERIMENTS.md · line 7833

E130 interim: prediction 1 partly falsified, and the reason reshapes the measurement.

Undisplaced frames do exist - 15 of them, 4 in bcc_alloys and 11 in bcc_alloys_ordered, at a mean displacement of 0.0000 to 0.0097 A. So the database is not uniformly rattled and option (a) is not dead on the grounds predicted.

It is dead on other grounds. Those 15 are almost all binaries, and they sit at lattice constants of 2.84 to 3.29 A. The expansion is fitted at a fixed 3.2935 A with no volume degree of freedom, so comparing against a frame at 2.84 A is the lattice leg of the same trap that E129 was withdrawn for. Exactly one of the fifteen is near our lattice constant. Fifteen frames, fourteen unusable, one data point.

But the volume constraint turns out not to bind. Across 6449 cubic bcc-commensurate frames, 613 sit within 0.01 A of 3.2935 - of which 467 have at least 54 atoms (clear of the 6.0 A pair cutoff that made E128's cells self-interact) and 375 have three or more elements rather than being binaries. That is a usable set at matched volume, which is what E128 never had.

So the measurement is: match the lattice by selection, match the relaxation by correction. scripts/validate/rhea_matched.py takes only frames within 0.01 A and at least 54 atoms, and for each one evaluates MACE twice on the same cell and occupations - once displaced as RHEA has it, once on ideal sites - so that MACE(displaced) - MACE(ideal) is a relaxation energy that can be subtracted from the DFT energy. E78 measured MACE against DFT at 1 to 4 meV/atom on differences of this kind.

The circularity is real and is bounded rather than denied. MACE is being used to correct a measurement of a model fitted to MACE. That is only acceptable because the correction is a difference over one structure rather than an absolute energy, and because the frames are pre-selected so the correction is small. If the corrected error lands near MACE's own error against DFT, the measurement cannot separate the two and the entry must say so rather than report a number.

Checkpointed per structure, resumable — the defect the audit found in the E128 script.

EXPERIMENTS.md · line 7867

E130 result: prediction 2 confirmed, prediction 3 FALSIFIED on its stated condition. The expansion is as accurate against first principles as against the surrogate it was fitted to, and the case for replacing rung 0 collapses.

467 RHEA structures, all 54 atoms, all within 0.01 A of 3.2935, nine elements, element- referenced both sides, displacement corrected by MACE on the same cell and occupations. All inputs verified finite; the element design matrix is full rank 9 of 9.

CONVENTION-MATCHED      MAE  8.1   RMSE 11.1   meV/atom
displacement ignored    MAE 13.5   RMSE 18.0
MACE relaxation energy  mean +71.1, sd 23.2, range +5.9 to +135.3

advertised cross-validation against MACE:  9.09
E128, withdrawn, conventions unmatched:   44.6

Prediction 2 confirmed. The relaxation energy averages +71.1 meV/atom, squarely inside the 16-112 band this project measured in E59. That is the term E128 was mostly reporting.

Prediction 3 falsified on exactly the condition written in advance: "Falsified if the corrected error comes out at or below 9.09, which would mean the expansion is as good against first principles as against the surrogate it was fitted to, and the whole case for replacing rung 0 collapses." 8.1 is below 9.09.

So rung 0 is not broken. Its cross-validation against MACE was an honest estimate of its error against DFT all along, on the eight-plus-Cr bcc space at its own lattice constant. The 44.6 was convention error, and it decomposes: on these matched-lattice frames, ignoring displacement alone costs 13.5 against 8.1, so the larger part of E128's 44.6 was the lattice mismatch — frames from 2.84 to 3.55 A scored against an expansion with no volume degree of freedom — not the displacement it named.

What this does to the plan. Tier 1, the pyeCE-on-RHEA rebuild, was justified by a 44.6 meV/atom bottom rung. That justification is gone. pyeCE may still be worth having for the reason the 2026-09-14 note gives — one model across the composition space, no refit when an element is added — but it is no longer an accuracy repair, and OVERNIGHT task 0 must stop describing it as one.

The caveat I said I would state rather than work around. MACE performs the displacement correction, and the corrected error, 8.1, is close to MACE's own error against DFT on differences of this kind — E78 measured 1 to 4 meV/atom on gaps. The measurement therefore cannot resolve the expansion's error below roughly that floor, and 8.1 should be read as an upper bound with a few meV of MACE inside it, not as a sharp value. What it does establish, robustly, is that the error is of order ten and not of order forty-five.

231 passing.

The full record

This entry is written in 3 separate places in the log, shown here in log order.

EXPERIMENTS.md · lines 7798–7831

E130 — How to measure rung 0 against DFT without repeating E129's mistake

E129 withdrew E128: the expansion was evaluated on ideal sites and differenced against DFT energies of displaced structures, so the 44.6 meV/atom was expansion error plus relaxation energy, inseparably. The question E128 was meant to answer is still open and still the most load-bearing unmeasured quantity in the project.

Three ways to close it, in order of how much they assume:

  • (a) Find undisplaced structures. If any RHEA bcc group sits on ideal sites, the comparison is direct and assumes nothing.
  • (b) Correct with MACE. MACE(displaced) - MACE(ideal) on the same cell is a relaxation energy; subtracting it from the DFT energy gives an estimated DFT(ideal). This imports MACE's error, but only on a difference over one structure, where it is far better behaved than on an absolute energy.
  • (c) Relax with DFT ourselves. Correct and unaffordable.

Predicted:

  1. No RHEA bcc group is undisplaced - the database exists to train interatomic potentials, so rattling is the point. Option (a) fails, and the mean displacement will be non-zero in every bcc config_type.
  2. The relaxation energy MACE reports is of order the 16-112 meV/atom this project already measured (E59) - large enough to have been the dominant term in E128's 44.6, which is the quantitative statement of why E129 was a withdrawal and not a footnote.
  3. After correction by (b), the expansion's error against DFT is smaller than 44.6 but still well above the 9.09 it advertises - I expect 15 to 35 meV/atom.

Falsified if the corrected error comes out at or below 9.09, which would mean the expansion is as good against first principles as against the surrogate it was fitted to, and the whole case for replacing rung 0 collapses. Also falsified, differently, if the MACE relaxation energies scatter so widely that the correction is not usable - in which case option (b) is out and the honest answer is that this project cannot measure rung 0 against DFT with the data it has, which must then be stated rather than worked around.

EXPERIMENTS.md · lines 7833–7865

E130 interim: prediction 1 partly falsified, and the reason reshapes the measurement.

Undisplaced frames do exist - 15 of them, 4 in bcc_alloys and 11 in bcc_alloys_ordered, at a mean displacement of 0.0000 to 0.0097 A. So the database is not uniformly rattled and option (a) is not dead on the grounds predicted.

It is dead on other grounds. Those 15 are almost all binaries, and they sit at lattice constants of 2.84 to 3.29 A. The expansion is fitted at a fixed 3.2935 A with no volume degree of freedom, so comparing against a frame at 2.84 A is the lattice leg of the same trap that E129 was withdrawn for. Exactly one of the fifteen is near our lattice constant. Fifteen frames, fourteen unusable, one data point.

But the volume constraint turns out not to bind. Across 6449 cubic bcc-commensurate frames, 613 sit within 0.01 A of 3.2935 - of which 467 have at least 54 atoms (clear of the 6.0 A pair cutoff that made E128's cells self-interact) and 375 have three or more elements rather than being binaries. That is a usable set at matched volume, which is what E128 never had.

So the measurement is: match the lattice by selection, match the relaxation by correction. scripts/validate/rhea_matched.py takes only frames within 0.01 A and at least 54 atoms, and for each one evaluates MACE twice on the same cell and occupations - once displaced as RHEA has it, once on ideal sites - so that MACE(displaced) - MACE(ideal) is a relaxation energy that can be subtracted from the DFT energy. E78 measured MACE against DFT at 1 to 4 meV/atom on differences of this kind.

The circularity is real and is bounded rather than denied. MACE is being used to correct a measurement of a model fitted to MACE. That is only acceptable because the correction is a difference over one structure rather than an absolute energy, and because the frames are pre-selected so the correction is small. If the corrected error lands near MACE's own error against DFT, the measurement cannot separate the two and the entry must say so rather than report a number.

Checkpointed per structure, resumable — the defect the audit found in the E128 script.

EXPERIMENTS.md · lines 7867–7910

E130 result: prediction 2 confirmed, prediction 3 FALSIFIED on its stated condition. The expansion is as accurate against first principles as against the surrogate it was fitted to, and the case for replacing rung 0 collapses.

467 RHEA structures, all 54 atoms, all within 0.01 A of 3.2935, nine elements, element- referenced both sides, displacement corrected by MACE on the same cell and occupations. All inputs verified finite; the element design matrix is full rank 9 of 9.

CONVENTION-MATCHED      MAE  8.1   RMSE 11.1   meV/atom
displacement ignored    MAE 13.5   RMSE 18.0
MACE relaxation energy  mean +71.1, sd 23.2, range +5.9 to +135.3

advertised cross-validation against MACE:  9.09
E128, withdrawn, conventions unmatched:   44.6

Prediction 2 confirmed. The relaxation energy averages +71.1 meV/atom, squarely inside the 16-112 band this project measured in E59. That is the term E128 was mostly reporting.

Prediction 3 falsified on exactly the condition written in advance: "Falsified if the corrected error comes out at or below 9.09, which would mean the expansion is as good against first principles as against the surrogate it was fitted to, and the whole case for replacing rung 0 collapses." 8.1 is below 9.09.

So rung 0 is not broken. Its cross-validation against MACE was an honest estimate of its error against DFT all along, on the eight-plus-Cr bcc space at its own lattice constant. The 44.6 was convention error, and it decomposes: on these matched-lattice frames, ignoring displacement alone costs 13.5 against 8.1, so the larger part of E128's 44.6 was the lattice mismatch — frames from 2.84 to 3.55 A scored against an expansion with no volume degree of freedom — not the displacement it named.

What this does to the plan. Tier 1, the pyeCE-on-RHEA rebuild, was justified by a 44.6 meV/atom bottom rung. That justification is gone. pyeCE may still be worth having for the reason the 2026-09-14 note gives — one model across the composition space, no refit when an element is added — but it is no longer an accuracy repair, and OVERNIGHT task 0 must stop describing it as one.

The caveat I said I would state rather than work around. MACE performs the displacement correction, and the corrected error, 8.1, is close to MACE's own error against DFT on differences of this kind — E78 measured 1 to 4 meV/atom on gaps. The measurement therefore cannot resolve the expansion's error below roughly that floor, and 8.1 should be read as an upper bound with a few meV of MACE inside it, not as a sharp value. What it does establish, robustly, is that the error is of order ten and not of order forty-five.

231 passing.

Related entries

Built with PRISMWebsite and visualizations made using Claude