Particle Swarm Optimization - HumpDay Implementation

Abstract

Particle Swarm Optimization (PSO) is a population-based stochastic optimization technique introduced by Kennedy and Eberhart (1995). A swarm of particles flies through the search space; each particle's velocity is steered toward its personal best position and the swarm's global best position. HumpDay's ParticleSwarm uses an annealed inertia / cognitive / social schedule, velocity clamping that shrinks over the run, and finishes with an L-BFGS-B polish from the swarm best.

Auto-selection metadata. Consulted by humpday.minimize() when picking a default algorithm.

Overhead tier 1: light (~100µs/iter). Eligible when measured eval_time ≥ 10 µs.
Dimensional cap: no cap — linear-in-dim or better.
Minimum trials: 10 — baseline minimum.

Algorithm Description

The Particle Swarm Optimization algorithm maintains a swarm of particles, where each particle $i$ has a position $\mathbf{x}_i$ and velocity $\mathbf{v}_i$ in the $D$-dimensional search space. The algorithm updates these parameters according to the following equations:

Velocity Update:

$$\mathbf{v}_i^{t+1} = w\mathbf{v}_i^t + c_1r_1(\mathbf{p}_i - \mathbf{x}_i^t) + c_2r_2(\mathbf{g} - \mathbf{x}_i^t)$$

Position Update:

$$\mathbf{x}_i^{t+1} = \mathbf{x}_i^t + \mathbf{v}_i^{t+1}$$

where:

$w$ is the inertia weight
$c_1, c_2$ are acceleration coefficients (cognitive and social parameters)
$r_1, r_2$ are random numbers in [0,1]
$\mathbf{p}_i$ is the personal best position of particle $i$
$\mathbf{g}$ is the global best position found by the swarm

Implementation Details

Component	Description	Reference
Original Algorithm	Kennedy, J. & Eberhart, R. (1995). Particle swarm optimization technique introduced as population-based stochastic optimization inspired by social behavior of bird flocking.	Kennedy & Eberhart (1995)
Python Implementation	HumpDay modular implementation in evolutionary_algorithms.py	Source Code
JavaScript Implementation	Browser-compatible implementation in modular structure	JavaScript Source
Reference Package	Cross-validated against PySwarms implementation	PySwarms Documentation

HumpDay's PSO at a glance. Annealed coefficients (w: 0.9 → 0.4, c₁: 2.5 → 1.5, c₂: 1.5 → 2.5) over the run shift the balance from exploration to exploitation. Velocity is clamped to v_max = 0.2·(1 − 0.5·t/T). After the swarm exhausts its share of the budget, an L-BFGS-B polish from the swarm best (min(20·n_dim, n_trials/2) evals) refines to high precision — this is the change that pushes HumpDay's PSO past mealpy PSO on the SOTA snapshot.

Stagnation reseed (SPSO-2011-style). A canonical PSO has no restart mechanism — once the global best stops improving, particles collapse onto it and the swarm has no way to escape a local optimum. HumpDay's PSO tracks the global best across iterations and, when it has stalled for K = max(10, max_iterations/5) consecutive iterations, reseeds the worst half of the swarm with fresh uniform positions and small random velocities. The personal-best memory of the better half is preserved, so prior progress isn't wasted and the next reseed cycle starts from the global best already found. This follows the diversification mechanism used in the SPSO-2011 reference standard (Clerc et al., 2012, HAL preprint).

Algorithmic Parameters

Parameter	Value/Range	Description
Swarm Size	min(40, max(15, 3×D))	Number of particles, scaled by problem dimension
Inertia Weight ($w$)	0.9 → 0.4	Decreasing linearly with iteration progress
Cognitive Coefficient ($c_1$)	2.5 → 1.5	Decreasing emphasis on personal best
Social Coefficient ($c_2$)	1.5 → 2.5	Increasing emphasis on global best
Velocity Clamping	±20% of search range	Decreasing with iteration progress

References

Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. Proceedings of ICNN'95 - International Conference on Neural Networks, 4, 1942-1948. IEEE.
Shi, Y., & Eberhart, R. (1998). A modified particle swarm optimizer. 1998 IEEE international conference on evolutionary computation proceedings, 69-73. IEEE.
Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE transactions on Evolutionary Computation, 6(1), 58-73.
Poli, R., Kennedy, J., & Blackwell, T. (2007). Particle swarm optimization: an overview. Swarm intelligence, 1(1), 33-57.