Experiments · E116

On twelve elements, does the search still collapse onto one corner of the composition space?

Yes. It collapses at least as hard, onto vanadium and tungsten; the statistical flaw first blamed was then measured and ruled out.

In the log: Does the collapse survive the wider space?

mixedDate not stated in the log; it was written between the commit of 2026-09-13 08:16 and the first commit that contains it, 2026-09-16 02:04search baselines0 predictions · 2 result paragraphsEXPERIMENTS.md lines 6783–6809, lines 6811–6859, lines 6861–6896
exp E116 diagram
What E116 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E116.svg).

Results

EXPERIMENTS.md · line 6811

E116 result: prediction 2 falsified. The search does not go to tantalum and tungsten on twelve elements - it goes nowhere good at all.

Same protocol, twelve-element default:

element   early    late          early -> late
V         0.210   0.582     entropy    2.367 -> 1.300  (uniform ln 12 = 2.485)
W         0.117   0.224     supports      72 -> 19
Ti        0.086   0.103
Ta        0.069   0.010     <- ninth, not first

Predictions 1 and 4 confirmed: it still collapses, and it collapses at least as hard. Prediction 2 falsified: V and W take the mass, Ta falls to ninth. Prediction 3 half confirmed: Cr, Ni, Cu and Co are all driven under 2 per cent, so they are being rejected rather than being absent, which was the point - but they are not the four smallest, Hf and Ta are down there with them. The falsification condition did not trigger - none of the fcc four holds over 5 per cent - so E114's diagnosis stands and is not withdrawn.

But the V-rich basin is not a find, and chasing that is what produced the real result. Both expansions were asked the same question and they agree to a few meV/atom, so it is not a fit artefact:

composition              8-element    12-element
V.58 W.22 Ti.10 Nb.10        +73.9         +72.9     <- rejected by the screen
V0.91 Mo0.09                 +15.2         +14.9
MoNbTaW                      -87.7         -71.7
Mo0.55 Ta0.45               -113.7         -82.4     <- the actual optimum

CrossEntropy maximises the negative of this, so it should have gone to Mo-Ta. It did not. Over 240 proposals on twelve elements its best find was +36.5 meV/atom - nothing stable at all.

Cause: the elite count does not scale with the dimension, so the covariance is singular. np.cov of elite points in n dimensions has rank at most elite - 1, and elite was hardcoded at 12 against twelve elements. The fitted Gaussian sampled inside a degenerate subspace. On eight elements 12 elites for an 8x8 covariance is merely marginal, which is why this survived unnoticed. elite now defaults to max(12, 4 * n_elements).

The fix is not yet justified by measurement and must not be treated as settled. At 240 proposals it makes things worse - best +93.7 against +36.5 - because 48 of those proposals go to random seeding and the remaining 192 cannot converge. That says the budget is too small for twelve dimensions either way, not that singular covariance is preferable. A sweep of elite in {12, 24, 48, 96} at 1200 proposals, two seeds, is running to settle it.

Consequence for E115, which is running now: it was launched before this change, so its CrossEntropy arm uses elite = 12 on twelve elements - the rank-deficient setting. E115 therefore compares MAP-Elites against a weakened baseline and cannot be read as a fair verdict on quality-diversity. It must be re-run against whichever elite count the sweep selects before any conclusion is drawn from it.

EXPERIMENTS.md · line 6861

E116 correction: the causal claim above is WITHDRAWN.

The entry blamed CrossEntropy's failure on the twelve-element space on a rank-deficient covariance, and changed the elite count on that basis. The sweep it called for falsifies it. 1200 proposals, twelve elements, two seeds, best driving force in meV/atom:

elite            12 (deficient)   24 (full)   48 (full)   96 (full)
best                      -78.7       -76.8       -62.7       -35.3
best composition   Ta.55 Mo.42   Mo.41 Ta.34  Ta.42 Mo.30  W.44 Ta.33

Monotonically worse as the elite count grows, and the rank-deficient setting is the best of the four. 12 and 24 differ by 1.9 meV/atom on two seeds, which is noise; 48 and 96 are clearly worse. The reason is the one already visible at 240 proposals: the arm samples uniformly until it holds elite observations, so a larger elite starts later and has less budget left to converge.

So the singularity is real as arithmetic and is not the cause of anything measured. The floor regularisation handles the deficient direction. What actually broke CrossEntropy at 240 proposals was the budget: given 1200 it finds Ta0.55 Mo0.42 at -78.7 whatever the elite count. elite is restored to 12, now documented with the sweep that justifies it rather than with a rank argument that does not. 227 passing.

Two consequences.

First, E115 is NOT compromised and the warning written against it is withdrawn. It is running with elite = 12, which this sweep identifies as the best of the four settings, so its CrossEntropy arm is the strongest baseline available rather than a crippled one. E115 can be read as a fair test when it lands.

Second, and worth recording as a pattern rather than an incident: this is the second time in this session that a repair was shipped with a stated cause that later measurement falsified - the first was the claim that the centre-of-mass estimator scattered four times more than the parabolic peak, which turned out to be a drift-mask defect in the repair itself (E113). Both times the fix was written before the measurement that was supposed to justify it. The rule this suggests: when a defect is identified by reasoning rather than by measurement, run the measurement before changing the code, not after.

The full record

This entry is written in 3 separate places in the log, shown here in log order.

EXPERIMENTS.md · lines 6783–6809

E116 — Does the collapse survive the wider space?

E114 measured CrossEntropy concentrating onto W and Ta, but it ran on the eight-element expansion, before the default was widened. That measurement describes a search we no longer run, so it cannot be left standing as the diagnosis without being repeated. Identical protocol

  • 30 rounds, 8 proposals, drive_conservative, same seed - on the twelve-element default.

The four added elements are the four worst on a bcc lattice. Their shared-lattice elemental anchors are Cu 505.3, Cr 975.1, Ni 994.8 and Co 1100.1 meV/atom, against Nb 4.4 and Ta 6.1. A search that rewards driving force should discard them almost immediately.

Predicted:

  1. CrossEntropy still collapses. Element-mass entropy falls substantially from its starting value, which is now against a uniform ln(12) = 2.485 rather than ln(8) = 2.079.
  2. W and Ta still take the largest share, because the landscape's optimum is still Mo-Ta; adding elements that cannot compete does not move a peak.
  3. The fcc four - Cr, Ni, Cu, Co - are driven below 2 per cent atomic mass each, and are the four smallest at the end. They are being rejected, which is the correct behaviour and is different from being absent, which is what E114 found.
  4. Late entropy lands near or below the 1.443 measured on eight elements, because the extra four are eliminated rather than explored, so the effective space is not larger.

Falsified if any of Cr, Ni, Cu or Co holds more than 5 per cent of the atomic mass at the end, in which case the twelve-element landscape is genuinely different, E114's diagnosis was measured on the wrong space and must be withdrawn rather than carried forward, and the E115 comparison now running needs re-reading in that light.

EXPERIMENTS.md · lines 6811–6859

E116 result: prediction 2 falsified. The search does not go to tantalum and tungsten on twelve elements - it goes nowhere good at all.

Same protocol, twelve-element default:

element   early    late          early -> late
V         0.210   0.582     entropy    2.367 -> 1.300  (uniform ln 12 = 2.485)
W         0.117   0.224     supports      72 -> 19
Ti        0.086   0.103
Ta        0.069   0.010     <- ninth, not first

Predictions 1 and 4 confirmed: it still collapses, and it collapses at least as hard. Prediction 2 falsified: V and W take the mass, Ta falls to ninth. Prediction 3 half confirmed: Cr, Ni, Cu and Co are all driven under 2 per cent, so they are being rejected rather than being absent, which was the point - but they are not the four smallest, Hf and Ta are down there with them. The falsification condition did not trigger - none of the fcc four holds over 5 per cent - so E114's diagnosis stands and is not withdrawn.

But the V-rich basin is not a find, and chasing that is what produced the real result. Both expansions were asked the same question and they agree to a few meV/atom, so it is not a fit artefact:

composition              8-element    12-element
V.58 W.22 Ti.10 Nb.10        +73.9         +72.9     <- rejected by the screen
V0.91 Mo0.09                 +15.2         +14.9
MoNbTaW                      -87.7         -71.7
Mo0.55 Ta0.45               -113.7         -82.4     <- the actual optimum

CrossEntropy maximises the negative of this, so it should have gone to Mo-Ta. It did not. Over 240 proposals on twelve elements its best find was +36.5 meV/atom - nothing stable at all.

Cause: the elite count does not scale with the dimension, so the covariance is singular. np.cov of elite points in n dimensions has rank at most elite - 1, and elite was hardcoded at 12 against twelve elements. The fitted Gaussian sampled inside a degenerate subspace. On eight elements 12 elites for an 8x8 covariance is merely marginal, which is why this survived unnoticed. elite now defaults to max(12, 4 * n_elements).

The fix is not yet justified by measurement and must not be treated as settled. At 240 proposals it makes things worse - best +93.7 against +36.5 - because 48 of those proposals go to random seeding and the remaining 192 cannot converge. That says the budget is too small for twelve dimensions either way, not that singular covariance is preferable. A sweep of elite in {12, 24, 48, 96} at 1200 proposals, two seeds, is running to settle it.

Consequence for E115, which is running now: it was launched before this change, so its CrossEntropy arm uses elite = 12 on twelve elements - the rank-deficient setting. E115 therefore compares MAP-Elites against a weakened baseline and cannot be read as a fair verdict on quality-diversity. It must be re-run against whichever elite count the sweep selects before any conclusion is drawn from it.

EXPERIMENTS.md · lines 6861–6896

E116 correction: the causal claim above is WITHDRAWN.

The entry blamed CrossEntropy's failure on the twelve-element space on a rank-deficient covariance, and changed the elite count on that basis. The sweep it called for falsifies it. 1200 proposals, twelve elements, two seeds, best driving force in meV/atom:

elite            12 (deficient)   24 (full)   48 (full)   96 (full)
best                      -78.7       -76.8       -62.7       -35.3
best composition   Ta.55 Mo.42   Mo.41 Ta.34  Ta.42 Mo.30  W.44 Ta.33

Monotonically worse as the elite count grows, and the rank-deficient setting is the best of the four. 12 and 24 differ by 1.9 meV/atom on two seeds, which is noise; 48 and 96 are clearly worse. The reason is the one already visible at 240 proposals: the arm samples uniformly until it holds elite observations, so a larger elite starts later and has less budget left to converge.

So the singularity is real as arithmetic and is not the cause of anything measured. The floor regularisation handles the deficient direction. What actually broke CrossEntropy at 240 proposals was the budget: given 1200 it finds Ta0.55 Mo0.42 at -78.7 whatever the elite count. elite is restored to 12, now documented with the sweep that justifies it rather than with a rank argument that does not. 227 passing.

Two consequences.

First, E115 is NOT compromised and the warning written against it is withdrawn. It is running with elite = 12, which this sweep identifies as the best of the four settings, so its CrossEntropy arm is the strongest baseline available rather than a crippled one. E115 can be read as a fair test when it lands.

Second, and worth recording as a pattern rather than an incident: this is the second time in this session that a repair was shipped with a stated cause that later measurement falsified - the first was the claim that the centre-of-mass estimator scattered four times more than the parabolic peak, which turned out to be a drift-mask defect in the repair itself (E113). Both times the fix was written before the measurement that was supposed to justify it. The rule this suggests: when a defect is identified by reasoning rather than by measurement, run the measurement before changing the code, not after.

Related entries

Built with PRISMWebsite and visualizations made using Claude