Was the new energy model being trained on structures that can show atoms ordering?

EXPERIMENTS.md · lines 8995–9063

E149 — The eCE was being trained on the half of RHEA that cannot see ordering

E152's control and the eCE build were both launched before this was checked. It should have been checked first.

build_ece_rhea.py selected bcc_alloys + bcc_binary_alloys and filtered to exactly 54 atoms. A histogram of the whole database shows what that leaves out:

bcc_alloys_ordered   4750     <- ordered superstructures, cells of 2/3/4/5/16 atoms
bcc_alloys           2150     <- random solid solutions, all 54 atoms
bcc_binary_alloys    1608     <- random, all 54 atoms
intermetallics        570     <- off-lattice, 3-36 atoms

The 54-atom filter keeps 6 of the 4750 ordered structures. Combined with the group selection it kept none. The model now training in runs/ece_rhea_v1 has seen 3,758 random solid solutions and zero ordered configurations.

Two results from the CE literature say this is the specific training set that produces a low validation error and a wrong ordering temperature:

• Random configurations at a given composition have nearly identical cluster vectors, so a training set built from them spans a narrow region of correlation space; enumerated and ordered structures are what widen it (Ekborg-Tanner et al., PRX Energy 3, 042001, 2024).
• The accuracy of a predicted order-disorder transition temperature tracks the accuracy of the ground-state energies specifically, not the mean error — convex-hull preservation is the stated prerequisite for predicting a phase boundary (Xie, Zhou, Jin & Jiang, J. Chem. Theory Comput. 20, 2024; also Huang et al., npj Comput. Mater. 3, 30, 2017).

The 54-atom filter came from E128's self-interaction caveat. That caveat is about evaluating a fitted expansion on a cell shorter than its pair cutoff. It is not an argument about what to fit on: small ordered cells are the standard CE training set, and every CE cited above uses them.

Two runs are now training, differing in one variable.

Predicted:

1. v1's own held-out MAE will look good — under 15 meV/atom — and will not reveal the defect, because its test split is drawn from the same random solid solutions as its training split. A low number here is not evidence the model is sound.
2. Evaluated on the 4,750 ordered structures it never saw, v1's error will be several times its own held-out MAE, because the ordering terms were never constrained. I expect above 40 meV/atom.
3. v2 will be worse than v1 on v1's random-only test set and far better on the ordered set, since it spends capacity on a wider correlation space.

Falsified if v1 predicts the ordered structures to within its own held-out MAE, which would mean random solid solutions do constrain the ordering terms in this system and the literature result above does not transfer to a 9-element bcc alloy. That is the outcome worth knowing either way, and it is the measurement, not the two internal MAEs, that settles it.

A second defect fixed in passing. The PRIM lattice constant was estimated as a cell edge divided by a guessed cubic supercell repeat, which is only valid for cubic supercells. On the small ordered cells it returned values from -4.03 to 6.50 Å. Replaced with a = (2V/n)^(1/3), correct for any cell shape; the same 8,508 structures then give a median of 3.2300 Å over a range of 2.665 to 3.818. v1 is unaffected — its cells are all cubic 54-atom — and it continues as the control.

Open, and not fixed here: the paper's own tuning uses pair clusters to 10 Å and triplets to 4 Å; both runs use 6.0 Å and 4.5 Å. That is a second variable and is deliberately held fixed so this comparison has one.