Papers

Working papers built on the HumpDay benchmark suite. Sources, run logs, and every generated program are in the papers/ directory of the repository.

HumpDay: derivative-free optimizers in pure Python and JavaScript, with a contamination-resistant real-world benchmark

The software paper: 23 optimizers behind one contract, in two languages held together by parity tests; a recommender driven by rankings on 81 real-application objectives; and a disguise mechanism that makes the benchmark meaningful even when the candidates are written by language models.

Alloy: a machine-designed derivative-free optimizer, and a full account of the search that found it

Alloy was generated by a language model from a prompt requiring an equal blend of Nelder–Mead, Differential Evolution, CMA-ES, pattern search and simulated annealing, part by part. On twenty-nine problems never used in any selection step it has the best mean rank at every budget from 60 to 480 evaluations. The paper also reports what failed: the continuous search over blends never beat evaluating the obvious point, and a tuned generic template matched every selection score yet collapsed out of sample. Alloy ships in the package.

Thinking outside the simplex: signed compositions of optimization algorithms

Recipes with negative weights: −100% algorithm A, +200% algorithm B means build B, shadow A's proposal logic at zero cost, and steer away from where A would go. The best signed recipe beat every unsigned draw at selection; an ablation then reversed the story, since disabling the repulsion improved the same program. A record of the construction, the trap, and the protocol that avoids it: no signed recipe counts until it beats its own ablated twin.

Do synthetic benchmarks rank optimizers the way real problems do?

Rank correlation between optimizer leaderboards on synthetic test functions and on disguised real-world objectives, across evaluation budgets. Where the two disagree, recommendations tuned on synthetic suites mislead.

An evolved optimizer for the disguised benchmark

A genetic search over a 14-gene parametric template of DE/ES mechanisms, including a surrogate trust-region gene whose ablation more than halves regret on the disguised suite.