Abstract
<jats:p> <jats:bold>Motivation</jats:bold> Protein language models (PLMs) have emerged as powerful tools for sequence-based prediction of protein function, yet systematic benchmarks comparing frozen embeddings, fine-tuning strategies like Low-Rank Adaptation (LoRA) and classical machine learning (ML) remain limited. We benchmarked four ML strategies: ML using amino acid descriptors (SL-AAFeat), ML using frozen embeddings from 20 PLMs across various pooling strategies (SL-Embed), full model fine-tuning (FT-Full) and LoRA-based fine-tuning (FT-LoRA). Performance was evaluated on the in-house VHH phage display dataset (VHH) for binding affinity prediction and the TAPE fluorescence dataset (FLS and FLS10) for mutational effect prediction. <jats:bold>Results</jats:bold> Model performance depended strongly on the dataset and adaptation strategy. Max pooling consistently improved embedding-based models, while amino acid descriptors remained competitive under specific datasets and resource constraints. Fine-tuning generally provided the highest predictive performance, but the advantage is not universal. Hyperparameter optimization significantly enhanced FT-LoRA, enabling it to outperform FT-Full on the VHH dataset with less than 10% model parameter adaptation. In contrast, FT-Full achieved the best performance on FLS and FLS10. Several medium-sized PLMs performed comparably to larger models, highlighting favorable performance-efficiency trade-offs. Overall, this paper presents a thorough review of PLM utilization strategies and practical recommendations for selecting suitable strategies based on dataset characteristics and available computational resources. <jats:bold>Availability</jats:bold> The source code used in this manuscript is available in a Zenodo repository at https://doi.org/10.5281/zenodo.21466255. </jats:p>