Powell's Method - Implementation Details

← Back to Optimizer Competition

Powell's Method (Powell, 1964) is a derivative-free optimizer that minimises along a set of search directions, then updates the direction set to capture local curvature. Each outer iteration: line-search along every direction d_i, then add the cumulative displacement as a new direction (replacing the one that gave the largest decrease). HumpDay's Powell is a faithful pure-Python port of scipy's method='Powell' with bounded golden-section / Brent-style line search and the standard direction-set update.

HumpDay's Powell at a glance

Initial direction set: identity matrix — coordinate directions.
Line search: bounded Brent search per direction, clipped to the unit cube.
Direction update: after a full sweep, add x_final − x_start as a new direction, replacing the direction that gave the largest decrease (standard Powell update).
Convergence: ftol = 1e-12 on relative function change. Tightened from scipy's default 1e-4 so the algorithm uses its budget on smooth basins.

Interactive 3D Visualization

See Powell in action on 3D optimization surfaces:

Loading 3D visualization...

Requires WebGL support

Instructions: Choose a test function and algorithm, then click Start to watch the step-by-step optimization process.

Auto-selection metadata. Consulted by humpday.minimize() when picking a default algorithm.

Overhead tier 1: light (~100µs/iter). Eligible when measured eval_time ≥ 10 µs.
Dimensional cap: n_dim ≤ 30 — direction-set line searches stall.
Minimum trials: 10 — baseline minimum.

Implementation Details

Component	Details	Links
Original Algorithm	Michael J.D. Powell Conjugate direction method for unconstrained optimization Derivative-free approach using line searches Published: 1964	Wikipedia
Reference Implementation	SciPy optimize.minimize Method: 'Powell' scipy.optimize.minimize(method='powell') Package: scipy	SciPy Docs Source
HumpDay Python	HumpDay `Powell` Pure-Python port of scipy's Powell with direction-set update and bounded Brent line search. No required dependencies. File: `humpday/optimizers/scipy_algorithms.py`	Source
HumpDay JavaScript	Browser `Powell` Mirrors the Python port. Class: `Powell`	JS Port

Performance Characteristics

Best for: Smooth, low-dimensional objectives where conjugate directions capture curvature without needing gradients.
Worst for: High dimensions — the direction-set update degrades and the per-iteration cost grows.
Per outer iteration: n line searches (each is a 1-D Brent search inside [0, 1]), plus the direction-set update.
Relative to scipy.optimize Powell at n_trials=200, n_dim=2: ties on sphere and Ackley; a small (2×) gap on Rosenbrock from the interpreter tax on the line searches. See SOTA status for the live numbers.

Educational Resources

Algorithm Theory