Back to Search View Original Cite This Article

Abstract

<title>Abstract</title> <p>Reliable monitoring of Hadoop clusters requires models that can capture both the structural dynamics of distributed node behavior and the stochastic variability of workloads, failures, and recovery processes. Existing approaches based solely on rule-based monitoring, statistical modeling, or machine learning often face complementary limitations: threshold-based methods lack adaptability, formal stochastic models provide limited predictive flexibility, and deep learning approaches frequently sacrifice interpretability. To address these limitations, this paper proposes a hybrid Convolutional Neural Network--Stochastic Petri Net (CNN--SPN) framework for classifying Hadoop nodes as stable or unstable. The proposed approach combines a multi-density Stochastic Petri Net, using exponential, normal, log-normal, Poisson, and Weibull firing-time distributions, with a convolutional neural network that learns discriminative patterns from node-level monitoring features. The framework further incorporates a hybrid information fusion mechanism and a dynamic firing strategy to couple stochastic state evolution with learned feature representations. The model is evaluated on a reproducible synthetic dataset of \((10,000)\) node observations generated through SPN-based simulation, and its components are examined through ablation analysis. The results show that integrating the SPN state and hybrid fusion improves performance over a CNN-only baseline, while the complete framework achieves the best overall scores, with \((68.10%)\) accuracy, \((68.91%)\) precision, \((68.78%)\) recall, \((68.85%)\) F1-score, and a ROC-AUC of \((0.7434)\). Although real-cluster validation remains future work, the proposed framework provides an interpretable and structurally grounded basis for stochastic monitoring and node-instability detection in distributed Hadoop environments.</p>

Show More

Keywords

stochastic monitoring framework hadoop hybrid

Related Articles

PORE

About

Connect