Abstract
<title>Abstract</title> <p>UAV path planning in complex 3D environments is an NP-hard problem. Standard metaheuristics (MH) suffer from premature convergence, while pure reinforcement learning (RL) exhibits slow initial convergence, motivating hybrid RL--MH frameworks. This systematic review characterizes the predominant RL--MH algorithmic combinations, integration architectures, application contexts, and validation approaches reported for single-UAV and independently-planned multi-UAV path planning. The review was conducted following the PRISMA 2020 guidelines. Peer-reviewed journal articles and indexed conference proceedings published in English between 2016 and 2025 proposing explicit algorithmic hybridization between MH and RL for UAV path planning or navigation were included, provided each vehicle plans its trajectory independently. Cooperative swarms, formation control, multi-agent coordination, and non-hybridized algorithms were excluded. Systematic queries were executed across Web of Science and Scopus using predefined RL and metaheuristic keyword blocks. Title, abstract, and full-text screenings were performed independently by two reviewers to minimize selection bias. Methodological quality was appraised using a five-domain rubric adapted to simulation-based algorithmic studies. Out of 34 full-text reports assessed, 32 studies (2022--2025) comprised the final corpus. Four mutually exclusive hybridization architectures were identified: (A) meta-control, in which RL adaptively tunes MH parameters and operators---the predominant pattern (17 studies, 53.1\%), most frequently implemented with tabular Q-learning (9/17) and hosted by Particle Swarm Optimization variants (7/17), with the Grey Wolf Optimizer a distant second (3/17); (B) metaheuristic-assisted deep RL via warm-start and replay-buffer initialization (3 studies, 9.4\%); (C) RL-driven portfolio selection and orchestration (9 studies, 28.1\%); and (D) multi-stage task decomposition through decoupled pipelines (3 studies, 9.4\%). Across the whole corpus, PSO and its variants are the dominant metaheuristic host (13 studies, 40.6\%), ahead of genetic and evolutionary algorithms (9, 28.1\%) and the Grey Wolf Optimizer family (4, 12.5\%). The corpus comprises 19 journal articles and 13 indexed conference papers; four of the five studies that model onboard perception in the planning loop are conference papers, which is why proceedings were retained. Applications were dominated by 3D routing (military, logistics, agricultural, and IoT networks) using weighted scalar cost functions that blend path length, safety, smoothness, and energy constraints. Validation relied exclusively on numerical simulation (32/32, 100\%); no study reported physical UAV hardware flights, and standard robotics middleware (ROS/ROS2, Gazebo, Webots) was absent from the entire corpus. Rigorous statistical testing (Wilcoxon/Friedman) was uneven, and no study released code or data, capping reproducibility. While hybrid RL--MH frameworks consistently improve adaptability and convergence speed over non-hybrid baselines, their technological readiness for real-world deployment remains unproven. Future research must increase statistical rigor, standardize disaggregated evaluation metrics, expand physical hardware validation, and transition toward dynamic and cooperative multi-agent conditions.</p>