ETL Application with R Shiny Visualization

<jats:p>We present a modular ETL framework tailored to the healthcare domain, designed for flexibility across heterogeneous data sources and robust correctness through automated validation and concurrency control. Coupled with an interactive R/Shiny application, our solution provides real-time monitoring, error exploration, and performance visualization for complex parallel data pipelines. Benchmarking against legacy linear workflows demonstrates a reduction in processing times by a factor of 1.59 and error detection times of factor 2.8 with the help of the R/Shiny monitoring. This paper details the system’s architecture, logging schema, and dashboard features, highlighting its impact on operational transparency and data quality assurance in clinical research.</jats:p>