Experiments · E14
No. Its training data holds not one metal alloy of four or more of these elements; a public set of 2,150 exists.
In the log: Why the old model failed: a census, not a hypothesis
book/assets/diagrams/exp/E14.svg).Date 2026-09-12 · Question Is chapter 7's distributional explanation correct? · Provenance literature and database review; Materials Project OPTIMADE census
Method. Exhaustive enumeration of the Materials Project OPTIMADE endpoint (154,387 entries; MPtrj covers 145,923 materials, so this bounds it tightly), counting structures by how many of {Mo, Nb, Ta, W, V, Ti, Zr, Hf} they contain. Plus a measured census of 137,361 real OMat24 structures decoded from two validation subsets generated by different routes.
Result — Materials Project / MPtrj, the entire MACE-MP-0 training set:
| containing | count |
|---|---|
| >= 1 of the eight | 39,421 |
| >= 2 | 3,783 |
| >= 3 | 90 |
| >= 4 | 7 — all borides or a carbide, no metals |
| refractory-only, 4+ elements | 0 |
The 65 refractory-only ternaries are 4-atom ordered prototypes (MoNbV2, Nb2TaW,
NbTaW, NbTiZr, …), not solid-solution supercells.
OMat24, extrapolated from the measured 137,361-structure sample: ~155,000 frames containing 3+ of the eight (0.153%), ~4,400 refractory-only quaternary (0.004%), and zero quinary observed (95% upper bound <2,200). Those that exist are rattled decorations of Alexandria prototypes in triclinic and monoclinic settings — not random near-equiatomic bcc supercells.
Interpretation. Chapter 7's explanation is confirmed with a number: MACE-MP-0's training set contains not one quaternary or quinary metallic refractory alloy, and not one random bcc solid-solution supercell in this chemistry. The model had never seen anything resembling the cells it was being asked about.
The published name for the resulting symptom is systematic softening — Deng et al., npj Comput. Mater. 11, 9 (2025), arXiv:2405.07105 — which documents energy and force under-prediction in exactly these regimes, attributes it to "biased sampling of near-equilibrium atomic arrangements in uMLIP pre-training datasets", and reports it is "effectively rectified by fine-tuning with a single additional data point".
And the missing distribution already exists, computed. The Byggmästar RHEA database (Zenodo 10.5281/zenodo.18863415, 1.9 GB, CC-BY-4.0, VASP-PBE): 2,150 bcc alloy structures of 54 atoms, 3 to 9 components, elements Ti-Zr-Hf-V-Nb-Ta-Cr-Mo-W — a superset of ours — randomly ordered, randomly displaced, three volumes per composition, with isolated-atom references included. Plus binaries, liquids, defects, surfaces and intermetallics.
Cross-code compatibility is second-order: after element-wise referencing the VASP-QE residual is ~5 meV/atom RMS for these transition metals, against the 70-114 meV/atom error being fixed. If sources are ever combined, switching QE from PSLibrary to SSSP-precision closes most of the remaining gap (W 6.28 -> 1.43, Ta 3.27 -> 1.02 meV/atom).
Licence note. MACE-OMAT-0, the MACE checkpoint trained on OMat24, is ASL — academic non-commercial only, as are all GRACE models. MACE-MPA-0 is MIT. GNoME's weights are not downloadable at all.
Result — Materials Project / MPtrj, the entire MACE-MP-0 training set:
Date 2026-09-12 · Question Is chapter 7's distributional explanation correct? · Provenance literature and database review; Materials Project OPTIMADE census
Method. Exhaustive enumeration of the Materials Project OPTIMADE endpoint (154,387 entries; MPtrj covers 145,923 materials, so this bounds it tightly), counting structures by how many of {Mo, Nb, Ta, W, V, Ti, Zr, Hf} they contain. Plus a measured census of 137,361 real OMat24 structures decoded from two validation subsets generated by different routes.
Result — Materials Project / MPtrj, the entire MACE-MP-0 training set:
| containing | count |
|---|---|
| >= 1 of the eight | 39,421 |
| >= 2 | 3,783 |
| >= 3 | 90 |
| >= 4 | 7 — all borides or a carbide, no metals |
| refractory-only, 4+ elements | 0 |
The 65 refractory-only ternaries are 4-atom ordered prototypes (MoNbV2, Nb2TaW,
NbTaW, NbTiZr, …), not solid-solution supercells.
OMat24, extrapolated from the measured 137,361-structure sample: ~155,000 frames containing 3+ of the eight (0.153%), ~4,400 refractory-only quaternary (0.004%), and zero quinary observed (95% upper bound <2,200). Those that exist are rattled decorations of Alexandria prototypes in triclinic and monoclinic settings — not random near-equiatomic bcc supercells.
Interpretation. Chapter 7's explanation is confirmed with a number: MACE-MP-0's training set contains not one quaternary or quinary metallic refractory alloy, and not one random bcc solid-solution supercell in this chemistry. The model had never seen anything resembling the cells it was being asked about.
The published name for the resulting symptom is systematic softening — Deng et al., npj Comput. Mater. 11, 9 (2025), arXiv:2405.07105 — which documents energy and force under-prediction in exactly these regimes, attributes it to "biased sampling of near-equilibrium atomic arrangements in uMLIP pre-training datasets", and reports it is "effectively rectified by fine-tuning with a single additional data point".
And the missing distribution already exists, computed. The Byggmästar RHEA database (Zenodo 10.5281/zenodo.18863415, 1.9 GB, CC-BY-4.0, VASP-PBE): 2,150 bcc alloy structures of 54 atoms, 3 to 9 components, elements Ti-Zr-Hf-V-Nb-Ta-Cr-Mo-W — a superset of ours — randomly ordered, randomly displaced, three volumes per composition, with isolated-atom references included. Plus binaries, liquids, defects, surfaces and intermetallics.
Cross-code compatibility is second-order: after element-wise referencing the VASP-QE residual is ~5 meV/atom RMS for these transition metals, against the 70-114 meV/atom error being fixed. If sources are ever combined, switching QE from PSLibrary to SSSP-precision closes most of the remaining gap (W 6.28 -> 1.43, Ta 3.27 -> 1.02 meV/atom).
Licence note. MACE-OMAT-0, the MACE checkpoint trained on OMat24, is ASL — academic non-commercial only, as are all GRACE models. MACE-MPA-0 is MIT. GNoME's weights are not downloadable at all.
Built with PRISMWebsite and visualizations made using Claude