Can the fly generate new alloys that beat ranking a fixed list, on two goals?

PREREGISTER_generative.md · lines 1–59

Pre-registration: generating crystals against several objectives

Fixed before the code runs. Second pre-registration; the first (multi-objective selection) is recorded as a loss in E35 and is not superseded by this.

What is being asked, and why it differs from E35

E35 compared a Gaussian process and the circuit as selectors from a shared 4,000-point pool. That removed the only structural difference between them: a Gaussian process cannot generate. It ranks a list it is given and can never propose a composition outside it. The circuit walks composition space and proposes as it goes.

The question that matters is therefore not which ranks a list better, but whether a learned generator can produce crystals meeting several objectives at once - including compositions no list contained. That is also the question standing behind the generative-flow-network work: if a learned generator is competitive here, that approach has a basis; if nothing learned can beat drawing a large pool and ranking it, it does not.

Corrections to the E35 design, all made before seeing any result here

1. Objectives normalised to [0, 1] over the reference sweep before any hypervolume is computed. In E35 free energy spanned 250% of its mean and the solid-solution fraction 7%, so the metric was dominated by one and the test was close to single-objective.
2. Improvement over the shared start is the score, not absolute hypervolume. In E35 the common 20-point start supplied 94.4% of every method's number.
3. Matched acquisition. Both the process and the circuit select greedily on their own predicted scalarisation, with the same ParEGO weights drawn from the same stream. E35 gave the process an uncertainty-aware rule and the circuit pure greed, so part of what it measured was the acquisition. A separate arm gives the circuit the cluster expansion's measured uncertainty (E24) as an exploration term; both arms are reported.
4. Training scales with heads, each head owning its share of the plastic synapses.

Design

• Objectives: free energy (minimise) and solid-solution fraction (maximise), from the 128-site environment.
• Budget: 60 evaluations, 20 a shared Latin hypercube start.
• Seeds: 10, indices 800-809, fixed now.
• Score: normalised hypervolume gained over the shared start, reference point the per-objective minimum of the 4,000-draw sweep, fixed before any run.
• Arms:
  • random generation (draw and evaluate);
  • Gaussian process + ParEGO, selecting from a 4,000-point pool - its native mode;
  • the fly, generating by run-and-tumble, two compartment heads, greedy;
  • the fly, generating, with the expansion's uncertainty as an exploration term.
• Also reported, and decided now: how often each arm finds a point that dominates everything in the 4,000-draw pool. Only a generator can do this; the process cannot by construction, and its score there is zero by definition rather than by failure.

Verdicts

• Win: a fly arm exceeds the process by at least two standard errors on hypervolume gained.
• Draw: within two standard errors.
• Loss: the process exceeds both fly arms by at least two standard errors.

Reported whichever occurs. No seed selection, no metric changes, no tuning of either method on this benchmark. Sparsity stays 0.12 and its outstanding confirmation on an unseen pool is still outstanding and still reported as such.