Abstract
<title>Abstract</title> <p>This article introduces ESSIM (Ensemble Swarm Statistical Inference Method), an innovative probabilistic swarm optimization framework grounded in Bayesian statistical inference. In ESSIM, each particle is represented by a Gaussian probability distribution characterized by a mean vector and a covariance matrix. The covariance provides an explicit measure of uncertainty, allowing the search process to naturally balance exploration and exploitation. At each iteration, candidate solutions are sampled from these Gaussian beliefs, evaluated using the objective function, and the best-performing particles (elite) are used to construct a collective probabilistic model. Each particle subsequently updates its belief through a Bayesian fusion mechanism inspired by the Kalman filter, while a controlled process noise preserves exploration and prevents premature convergence. Unlike PSO, whose global-optimum-based topology rapidly reduces swarm diversity by drawing all particles towards the global optimum (gbest), ESSIM exploits Bayesian uncertainty (individual covariance) to maintain active diversity within the swarm. This continuous exploration enables it to avoid local minima and map a wide solution space, whereas PSO becomes prematurely stuck. However, this increased robustness comes at the cost of longer computation times, as convergence requires a greater number of iterations to explore and merge information from the elite. Experimental results obtained on the multimodal benchmark functions of Rastrigin and Schwefel, as well as on the optimisation of the hyperparameters of a Gaussian Process Regression (GPR) model using three different datasets, show that ESSIM outperforms classical PSO on both of these benchmarks. For GPR hyperparameter optimisation problems, ESSIM proves to be highly competitive across all datasets and even outperforms PSO in the case of the synthetic dataset.</p>