Papers

Working papers built on the HumpDay benchmark suite. Sources, run logs, and every generated program are in the papers/ directory of the repository.

HumpDay: derivative-free optimizers in pure Python and JavaScript, with a contamination-resistant real-world benchmark

The software paper: 23 optimizers behind one contract, in two languages held together by parity tests; a recommender driven by rankings on 81 real-application objectives; and a disguise mechanism that makes the benchmark meaningful even when the candidates are written by language models.

The inspiration simplex: using derivative-free optimization in concept space to create new derivative-free optimizers

Established algorithms at the vertices of a simplex; a language model turns each mixing recipe into a working program. First product: Alloy, best mean rank at every budget on twenty-nine held-out problems, shipped in the package. Extending recipes to negative weights, one of six semantics survives out of sample, with a rule: negative weights repair weak hosts and spoil strong ones. Failures reported throughout; further worked examples planned.

Do synthetic benchmarks rank optimizers the way real problems do?

Rank correlation between optimizer leaderboards on synthetic test functions and on disguised real-world objectives, across evaluation budgets. Where the two disagree, recommendations tuned on synthetic suites mislead.

An evolved optimizer for the disguised benchmark

A genetic search over a 14-gene parametric template of DE/ES mechanisms, including a surrogate trust-region gene whose ablation more than halves regret on the disguised suite.