Data is only useful if it improves a model you can measure, and only usable if you are allowed to use it. This lesson closes the loop: train, evaluate, and stay on the right side of the rules.
Watch
A ten-minute overview of how LLM fine-tuning works, which is where your dataset gets used.
Train small, train often
Use a small open base model so you can run many experiments cheaply.
- Supervised fine-tuning (SFT) on instruction-response data. Hugging Face TRL provides trainers for SFT and for preference methods such as DPO.
- Parameter-efficient methods such as LoRA and QLoRA let you tune on modest hardware by training small adapters, with QLoRA also quantizing the base model.
- Preference tuning with DPO on chosen/rejected pairs, after SFT.
- Keep hyperparameters fixed while comparing datasets, so differences come from the data.
Treat data like code: ablations
The core method of data-centric work is the controlled comparison:
| Run | Training data | Question |
|---|---|---|
| A | Base model, no tuning | Starting point |
| B | Small human-written set | Is synthetic better than a small real set? |
| C | Synthetic, unfiltered | Value of generation alone |
| D | Synthetic, filtered | Value of your filters |
| E | Synthetic + real mix | Does mixing help? |
Change one variable at a time, run each more than once if you can, and record the dataset version with every run.
Evaluate on the skill, on real data
- General regression checks with a harness such as EleutherAI's lm-evaluation-harness, to catch forgetting and degraded abilities.
- Skill-specific evaluation on a held-out set of real examples written or verified by people, never generated by your pipeline.
- LLM-judge comparisons between model outputs, with the validation steps from lesson 3.
- Read outputs. Look at 50 side-by-side responses. Numbers hide style regressions.
- Safety and bias checks relevant to your use.
Generating at scale
Large jobs need fast inference. Serving engines such as vLLM provide high-throughput generation, and pipeline frameworks such as Distilabel help orchestrate prompts, models, filters and outputs. Cache results, log seeds and settings, and make runs resumable.
Licensing and terms
This is not legal advice. It is a checklist of questions to answer before you publish or ship a model trained on synthetic data.
- The generating model's terms. Providers set conditions on how their outputs may be used. Some restrict using outputs to train competing models. Open-weight models come with licenses that differ, so read the one for the model you used.
- Source data. If generations were seeded with or grounded in documents, you also inherit those documents' licenses and any privacy obligations.
- Personal data. Check that generated records do not reproduce real personal information.
- Your own release. Choose a license for the dataset and model, and be clear about what is open. The Open Source Initiative's Open Source AI Definition describes what it expects an open AI system to provide.
- Document provenance. Record the generator, version, prompts, filters and date for every dataset.
Write a dataset card
A short document published with the data:
Name / version
Purpose and intended use
How it was generated: models, prompts, seeds, parameters
Filters applied, with counts at each stage
Known limitations and biases
Evaluation results of models trained on it
License, and licenses of upstream sources and generators
Contact / how to report issues
Label synthetic data as synthetic, say which model produced it, and report what you removed. People building on your work need to know.
Resources
Trainers for SFT, DPO and related methods.
Fine-tuning large models on modest hardware.
A framework for running standard evaluations.
High-throughput inference for large generation jobs.
Loading, processing, versioning and sharing datasets.
A reference point for what "open" means for AI systems.
A free course on transformers and fine-tuning if you want to fill in background.
Check yourself
- Fine-tune a small model on 1,000 synthetic examples with LoRA. Evaluate before and after on a real held-out set.
- Run the A to E comparison table above on a small scale. Which variable moved the result most?
- Write the dataset card for your data, including license questions you could not resolve.