Does reporting the best find from each niche, instead of the top five, reveal more?
Yes. It surfaced 18 compositions instead of 5 at no cost to the best find, but only one of the new ones qualified.
In the log: Promotion by niche instead of top-k
confirmedDate not stated in the log; it was written between the commit of 2026-09-16 04:53 and the first commit that contains it, 2026-09-16 05:56unclassified0 predictions · 1 result paragraphEXPERIMENTS.md lines 7320–7346, lines 7348–7377
What E123 did and how it came out, drawn from this record and the files it names (book/assets/diagrams/exp/E123.svg).
Results
EXPERIMENTS.md · line 7348
E123 result: all three predictions confirmed — and the fix reveals diversity, it does not
create it.
One converged CrossEntropy run, 600 proposals, both policies reading the identical trace:
Prediction 1 confirmed: the top five span two niches, and four of the five are the same
Mo-Ta-W-Nb composition at slightly different mixing ratios. Prediction 2 confirmed: 18
promoted against the predicted 8, and 16 of them are absent from the top five entirely.
Prediction 3 confirmed: the best find is identical under both, to the digit - the niche
policy costs nothing at the top, because the global best is always the elite of its own cell.
The caveat the data forces, and it matters more than the result. Of the 16 compositions
the niche policy surfaces, only one is below the -40 meV/atom qualifying threshold
(Mo0.71 Nb0.24 at -25.0 is the next best, and does not qualify). The rest run from +16 to
+181. The niche policy surfaces what the run found; a CrossEntropy run did not find much.
That is exactly E115's finding seen from the other end. The scalar arm's trace contains two
useful niches because the scalar arm only ever visits two; MAP-Elites at 20,000 evaluations
covers 86.9 per cent of the space and finds a reproducible Mo-Ti-Ta basin at -47 meV/atom that
CrossEntropy never reaches. The reporting fix and the generator fix are complementary and
neither is sufficient alone: top-k reporting would have hidden the Mo-Ti-Ta basin even if
the archive had found it, and niche reporting over a mode-seeking trace surfaces mostly
mediocre compositions.
So the promotion change is correct and it is not the win. The win requires the archive arm to
be the one generating, which is a separate change to stage_b's default and has not been made.
The full record
This entry is written in 2 separate places in the log, shown here in log order.
EXPERIMENTS.md · lines 7320–7346
E123 — Promotion by niche instead of top-k
stage_b.py reports sorted(trace, key=-p)[:5] - the five highest-scoring compositions of a
run. E114 and E116 measured what the generator does to a trace: element-mass entropy falls
from 2.37 to 1.30 and distinct element supports from 72 to 19. A converged optimiser's top
five are therefore five near-copies of one composition, and E115's reproducible Mo-Ti-Ta basin
at -47 meV/atom - which the scalar arm never reaches - would not appear in a report of this
shape even if the archive had found it.
This is a reporting change and nothing else. Both policies read the same trace, so no
new sampling is involved and the comparison is exact rather than statistical. The niche is the
MAP-Elites cell already defined in forager.search.generate.MapElites: valence electron
concentration against atomic size mismatch.
Predicted, on one stage_b run, comparing the two policies over the identical trace:
The top-5 policy returns compositions from at most 2 distinct niches, because the
generator concentrates and the five best are near-copies.
One-elite-per-niche returns at least 8 distinct compositions, and at least one of them
is absent from the top 5 entirely - that is the whole claim: the report is currently
hiding finds the run already made.
The single best composition is identical under both policies. A niche policy must not
cost the best find; if it does, it is not a reporting change but a worse ranking.
Falsified if the top-5 already spans many niches, which would mean the trace is more
diverse than E114/E116 measured and the reporting was never the bottleneck - in which case
the promotion problem is upstream, in what the generator proposes, not in what gets reported.
EXPERIMENTS.md · lines 7348–7377
E123 result: all three predictions confirmed — and the fix reveals diversity, it does not
create it.
One converged CrossEntropy run, 600 proposals, both policies reading the identical trace:
Prediction 1 confirmed: the top five span two niches, and four of the five are the same
Mo-Ta-W-Nb composition at slightly different mixing ratios. Prediction 2 confirmed: 18
promoted against the predicted 8, and 16 of them are absent from the top five entirely.
Prediction 3 confirmed: the best find is identical under both, to the digit - the niche
policy costs nothing at the top, because the global best is always the elite of its own cell.
The caveat the data forces, and it matters more than the result. Of the 16 compositions
the niche policy surfaces, only one is below the -40 meV/atom qualifying threshold
(Mo0.71 Nb0.24 at -25.0 is the next best, and does not qualify). The rest run from +16 to
+181. The niche policy surfaces what the run found; a CrossEntropy run did not find much.
That is exactly E115's finding seen from the other end. The scalar arm's trace contains two
useful niches because the scalar arm only ever visits two; MAP-Elites at 20,000 evaluations
covers 86.9 per cent of the space and finds a reproducible Mo-Ti-Ta basin at -47 meV/atom that
CrossEntropy never reaches. The reporting fix and the generator fix are complementary and
neither is sufficient alone: top-k reporting would have hidden the Mo-Ti-Ta basin even if
the archive had found it, and niche reporting over a mode-seeking trace surfaces mostly
mediocre compositions.
So the promotion change is correct and it is not the win. The win requires the archive arm to
be the one generating, which is a separate change to stage_b's default and has not been made.
Related entries
E114 — The search space was eight elements, and the search collapses inside it