19. Instruct Fine-Tuning
Learning objectives
- Introduce Fine Tuning
- Motivate Instruct Fine Tuning
Sources
Following the Gen AI Handbook, we looked at
Fine Tuning
Module
fine tuning
- “… to add a small set of parameters to a pretrained LLM. Only the newly added parameters are finetuned while all the parameters of the pretrained LLM remain frozen.” — Sebastian Raschka (author, article)
Where were we?
Wei, et al., 2022
Instruct Fine Tuning
Definition Seeking
getting definitions
Instruction Guidance
providing instructions
Alignment Tuning = Instruction Tuning
- open-ended tasks
- creative generation
- human feedback
Instruction Data
Mixing few-shot settings
Task diversity
Data Augmentation
- e.g. turning a question-answering task into a question-generation task
Mixing weights
Recent Developments
Natural Instructions: Q
question generation, Mishra et al., 2022
Natural Instructions: A
answer generation, Mishra et al., 2022
Unnatural Instructions
Honovich, et al., 2023
Exemplars
Flan 2022
Input Inversion
improve task diversity with input inversion
19. Instruct Fine-Tuning