Abstract
<title>Abstract</title> <p>Foundation models (FMs) are large-scale models pretrained on extensive datasets to learn general representations that can be adapted to multiple tasks through fine-tuning. FMs are gaining traction in the electrocardiogram (ECG) analysis field, also thanks to their ability to effectively address downstream tasks for which less data is available. However, the choice of the adaptation strategy is often underestimated, usually relying only on full fine-tuning, ignoring the risk of overfitting due to the high model’s complexity and the limited dataset size. In this study, we propose parameter-efficient fine-tuning (PEFT) as an alternative to full fine-tuning to increase model generalizability and reduce overfitting. Specifically, we examine partial fine-tuning, BitFit, LayerNorm tuning, and linear probing. We evaluate the generalizability of the fine-tuned models on unseen indomain and out-of-domain data, and with different downstream tasks and dataset characteristics. We demonstrate that PEFT outperforms full fine-tuning not only in resource efficiency, but also in enhancing model generalization, especially on out-of-domain data.</p>