Main Concept
Fine-tuning is the process of taking a pre-trained foundation model and improve it by training on a specific dataset to specialize it for a particular task, domain, or behavior. The main idea is that you’re NOT training from scratch -you’re adjusting and exiting model.
Context
Fine-tuning is important because it allows companies or organizations to make a trained model that is already good behave in a certain specific way, understand specific terminology, or learn proprietary information, saving the money and time to train a foundation model by themselves.
Key Points
- Different methods of fine-tuning exist; we can classify them into these two main categories:
- Supervised Fine-Tuning (uses labeled data)
- Reinforcement Fine-Tuning
- Fine-tuning is more expensive than other customization methods (prompt engineering, few-shot learning, and RAG)
- Sometimes people present Distillation as a fine-tuning method; however, strictly speaking, it is more considered a model compression method.
- There is also a technique called Continued pre-training; however, this is NOT a fine-tuning method.
🔗 Related Concepts
- Supervised Fine-Tuning
- Reinforcement Fine-Tuning
- Transfer Learning: Broader concept (fine-tuning is one type)
- Prompt Engineering: Simpler alternative
- Retrieval Augmented Generation (RAG): Complementary technique
- Model Domain Adaptation: not a method, but an objective