Abstract
<title>Abstract</title> <p>This paper presents a spatiotemporal fusion approach that co-registers laterally resolved multispectral intensity maps (RGB multichannel) from a CMOS active-pixel sensor with achromatic direct timeof-flight (dToF) photon-arrival histograms for object detection that retains material-aware structure beyond appearance-based cues. A dual-module pipeline supplies localisation and category labels from spatial single-shot or transformer-style modules, while a compact convolutional material detection module classifies material from cropped time-resolved transients. The architecture addresses crossfeature sourcing: spatial features are taken from RGB intensity maps, while material composition and range are derived from time-resolved transients, distinct from conventional RGB-plus-depth or depth–AI fusion. The case study employs a SPADbased direct time-of-flight sensor (4 × 4 zones), a CMOS active-pixel sensor supplying three-channel RGB intensities, eight household produce and vessel categories, and systematic pose and illumination sweeps, reporting combined spatial–material scores near 94.5%–98.5% depending on the spatial backbone (YOLOv3, YOLOv8, DINOv3), with a consolidated material detection module near 99% validation accuracy at millisecond-class inference on CPU hardware. Natural versus synthetic discrimination, including LED screen and print spoofing resistance, is demonstrated. The paper closes with a custom spatiotemporal benchmark and open problems including multi-object detection, camouflage sensing, and scale estimation from transient signals.</p>