Input text corpus
Remember SFT
Full name: Supervised Fine-Tuning
Memory aid: Supervised examples Fine-Tune behavior.
Fine-tuning does not automatically keep knowledge current.
Study area: AI & machine learning · Batch 5
Relevant study for: Azure AI Fundamentals AI-901 · Azure AI Apps and Agents Developer AI-103 · Azure AI Fundamentals (retired) AI-900 · Azure AI Engineer (retired) AI-102
Practice with exam flashcards →AI Models & Assistants / Knowledge library
SFT (Supervised Fine-Tuning)
Supervised fine tuning on examples of desired responses
Why learn SFT (Supervised Fine-Tuning)?
Teaches a model a task format, response style, or demonstrated behavior. Dataset quality and held-out evaluation are essential because imitating examples does not guarantee factual correctness or broad generalization.
The problem it solves
Adapts pretrained capabilities to a demonstrated task or response pattern using explicit target examples.
Where it came from
Supervised fine-tuning adapts a pretrained model using examples of desired outputs. In instruction tuning, those examples often pair prompts or conversations with target assistant responses. It can be a standalone adaptation step or precede preference optimization, and may update all weights or a smaller set of adapter parameters.
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Write five prompt-response examples that demonstrate a consistent output format.
- Separate training examples from held-out prompts before any fine-tuning.
- Inspect a chat template and identify which tokens contribute to the training loss.
- Compare full-parameter fine-tuning with an adapter-based setup conceptually.
- Define an evaluation for format adherence and factual accuracy on unseen prompts.
Documentation and sources
- TRL supervised fine-tuning trainer (huggingface.co)
- InstructGPT: demonstrations and subsequent preference training (arxiv.org)
Connect the ideas
Explore next
Supervised LearningRLHF (Reinforcement Learning from Human Feedback)DPO (Direct Preference Optimization)LoRA AdaptersIn-Context Learning (ICL)LLM (Large Language Model)PEFT (Parameter-Efficient Fine-Tuning)Transfer Learning
Real World Example
Legal Document Summarization Fine-Tuning
Legal tech firms providing automated contract analysis for corporate law departments.
The business problem
Generic base models frequently hallucinate legal terminology or fail to adhere to the specific, dense formatting requirements of corporate contract summaries, leading to costly manual review cycles.
How this topic solves it
Supervised Fine-Tuning (SFT) allows the firm to train a base model on a curated dataset of high-quality, human-verified contract summaries, forcing the model to learn the precise structure and professional tone required for legal documentation.
- Prepare a conversational dataset containing pairs of raw legal contracts as user inputs and expert-verified summaries as assistant outputs.
- Define a chat template to ensure the model correctly distinguishes between the contract text and the required summary structure.
- Configure the SFTTrainer with assistant_only_loss=True to ensure the model focuses its learning exclusively on the quality of the generated summary rather than the input contract text.
- Execute the training process using a PEFT configuration to efficiently adapt the model while minimizing hardware resource consumption.
- Validate the model on a held-out set of contracts to ensure adherence to the desired summary format.
The flow shows how raw legal data is transformed into a specialized model.
Applying chat templates
Supervised learning engine
Specialized summarizer
- Raw Contracts → preprocess → Dataset Formatting
- Dataset Formatting → train → SFTTrainer
- SFTTrainer → output → Fine-Tuned Model
Implementation example
This code initializes the SFTTrainer with a configuration that ensures the model learns specifically from the assistant's summaries, ignoring the input contract text during loss calculation.
from trl import SFTTrainer, SFTConfig
# Configure trainer to focus loss only on assistant responses
training_args = SFTConfig(assistant_only_loss=True)
trainer = SFTTrainer(
model="Qwen/Qwen3-0.6B",
train_dataset=legal_dataset,
args=training_args
)
trainer.train()Expected business outcome
The model consistently produces summaries that mirror the structure and professional precision of human-written legal documents, significantly reducing the time required for attorney oversight.
Limitations and checks
Over-fitting to the training examples can lead to a loss of general reasoning capabilities; this is checked by monitoring the validation loss and evaluating the model on diverse, unseen contract types.