Is the energy model far less accurate against quantum calculations than its fit suggests?

|a - 3.2935|      n      MAE     RMSE   meV/atom
0.00 - 0.02     281     44.6     57.3
0.02 - 0.05     603     41.1     52.4
0.05 - 0.10     744     76.3     90.7
0.10 - 0.20     849     74.6    105.4
0.20 - 1.00     210    145.7    188.5
pooled         2687     70.0     97.6

for comparison, the cross-validation against MACE:   9.09

EXPERIMENTS.md · lines 7651–7689

E128 — Rung 0 against real DFT, not against MACE

The cluster expansion has never been compared to first principles. Its cross-validation of 9.09 meV/atom (E121) is measured against MACE-MPA-0, which is the thing it was fitted to. MACE has its own error against DFT, so the number quoted for rung 0's error everywhere in this project is a fit residual, not an accuracy.

The data to fix that has been sitting in data/rhea unused. The Byggmästar RHEA database (Zenodo 10.5281/zenodo.18863415, VASP-PBE) covers Cr-Hf-Mo-Nb-Ta-Ti-V-W-Zr - nine of our twelve elements and contains a group built for exactly this question:

bcc_alloys_ordered    4750 structures    ~12 atoms average    ordered bcc configurations

Ordered configurations on a bcc lattice in 16-atom cubic cells - the same cell size the expansion is fitted on. E18 used this database once, from scripts in /tmp that were never brought into the repository, and nothing in the codebase reads it now.

The comparison has a convention problem and it must not be hidden. The expansion is fitted at a fixed lattice constant of 3.2935 A and has no volume degree of freedom. RHEA's ordered cells sit near 3.23 A and vary structure to structure. Energies are therefore stratified by |a - 3.2935| rather than pooled, so that expansion error and volume mismatch are separable. Both sides are element-referenced by least squares first, which is the standard treatment and what E18 did.

Predicted:

1. The expansion's error against DFT is substantially larger than 9.09 meV/atom - at least 15 and plausibly 40 - because 9.09 measures agreement with MACE and this measures agreement with the truth MACE approximates.
2. The error grows with |a - 3.2935|, monotonically in the stratified bins, because the expansion cannot represent a volume change at all.
3. The bin closest to 3.2935 A gives the honest estimate of rung 0's error, and that number should replace the cross-validation figure wherever rung 0's accuracy is quoted - including in spec.py, which currently carries the fit residual.

Falsified if the error is at or below 9.09 meV/atom, which would mean MACE-MPA-0 is essentially exact for ordered bcc refractory alloys and the expansion inherits that - a far stronger result than anything claimed so far, and one that should be checked twice before being believed.

EXPERIMENTS.md · lines 7691–7738

E128 result: prediction 1 confirmed and then some. Rung 0's error against DFT is 44.6 meV/atom, not 9.09.

2687 ordered bcc structures from RHEA, nine elements (Cr Hf Mo Nb Ta Ti V W Zr), VASP-PBE, element-referenced both sides, stratified by distance from the expansion's fixed lattice:

|a - 3.2935|      n      MAE     RMSE   meV/atom
0.00 - 0.02     281     44.6     57.3
0.02 - 0.05     603     41.1     52.4
0.05 - 0.10     744     76.3     90.7
0.10 - 0.20     849     74.6    105.4
0.20 - 1.00     210    145.7    188.5
pooled         2687     70.0     97.6

for comparison, the cross-validation against MACE:   9.09

Prediction 1 confirmed, above the predicted range: 44.6 at matched lattice against a predicted 15 to 40. Prediction 2 partly confirmed - the error rises with volume mismatch overall, but not monotonically: the 0.02-0.05 bin (41.1) sits slightly below the 0.00-0.02 bin (44.6), so the two closest bins are within noise of each other and the trend only becomes clear past 0.05. Prediction 3 stands: 44.6 meV/atom is the honest figure for rung 0's accuracy, and 9.09 is a fit residual against a surrogate.

A caveat that must be settled before this number is quoted. icet warned ClusterExpansionCalculator self-interacts on the smaller cells. The expansion's pair cutoff is 6.0 A; a 16-atom cell at a = 3.23 A is 6.46 A across, barely above it, and bcc_alloys_ ordered averages twelve atoms, so many frames are smaller still. Self-interaction inflates the predicted energy and therefore the error. The number above is an upper bound until re-run on cells large enough to avoid it.

This confirms a warning already in the project's own notes and not recalled. The prior-art survey of 2026-09-14 records: "Do NOT run MC directly on a foundation model: MACE/CHGNet/ M3GNet reproduce mixing enthalpies poorly (Casillas-Trujillo, PRM 8, 113803, 2024) - exactly the quantity ordering depends on." Every ordering temperature in this project comes from a cluster expansion fitted to MACE mixing energies. The failure mode was documented two days before it was measured.

The route the notes also record. pyeCE (arXiv 2609.10190, epfl-mades/pyece) is an embedded cluster expansion - a learnable chemical embedding plus a neural network on pair clusters to 8 A - covering one model for the whole bcc composition space of Cr-Hf-Mo-Nb-Ta-Ti-V-W-Zr, which is precisely RHEA's nine elements. It reports 3.2 / 9.4 / 10.9 meV/atom train/val/test on 1703 DFT configurations. RHEA holds 4750 ordered bcc structures plus 2150 random solid solutions plus 1608 binaries, all VASP-PBE, on those same nine elements - substantially more than pyeCE was trained on.

So the next move is not another fit to MACE. It is to train the pyeCE architecture on RHEA DFT directly, which removes the surrogate from the bottom of the ladder entirely and replaces a 44.6 meV/atom error with something in the neighbourhood of 10.