Parameter-efficient Fine-tuning (or “PEFT”)

Parameter-efficient Fine-tuning (PEFT) is a technique that focuses on optimizing and adapting a pre-trained model’s parameters to new tasks with minimal additional training data. It tries to improve model performance while minimizing the need for lengthy retraining and lowering the risk of overfitting.