Vibe Coder · Unit 19: Fine-Tuning a Model · Project
Milestone 19 of 23
A model tuned for one job
A small experiment, measured against your best prompt.
One product, built one milestone at a time
Vibe Coder has one project. You pick an idea in Unit 1 and every unit adds a piece to it, until the last unit launches it. Every milestone ends with a push to your own GitHub repository. Fine-tuning is the last tool to reach for. This milestone runs one small, honest experiment: build a dataset, tune a small open model, and find out whether it beats the prompt you already have. It is required for engineers and optional for everyone else.
- 1Your builder’s kitA one-line product idea, your role, and five working accounts.
- 2A fact-checked product briefOne page on your idea, tested against how the models really behave.
- 3Your product project in ClaudeA project with instructions and files, and your first real piece of work from it.
- 4A brand kit and research fileA name, images and market research for your product, with a second opinion.
- 5Your prompt libraryThe prompts you will reuse to build the product, each one tested.
- 6Your product specThe problem, the people, the first version and how you will measure it.
- 7A clickable prototypeA version people can click through, shown to three of them.
- 8A working first draftYour product generated from the brief, with sign-in and data, in your own GitHub repository.
- 9The first working versionYour app running on your computer, built with Claude Code.
- 10Your Claude Code setupProject instructions, a custom command and a skill made for your app.
- 11One AI featureA feature in your app that calls a model, with its key kept safe.
- 12A connected assistantClaude or ChatGPT connected to a tool you already use, with the right permissions.
- 13Sign-in and saved dataAccounts and a database behind your app.
- 14Your app, liveOnline at its own address, updating every time you save.
- 15Your product’s back officeThe automatic jobs behind your product, as tested workflows in GitHub.
- 16An agent that works for your productOne agent with narrow tools, limits, approval and a test set.
- 17A helper that answers from your documentsAnswers with sources, scored on your own questions.
- 18A product that can chargeEmail from your own domain and a tested checkout, in test mode.
- 19A model tuned for one jobA small experiment, measured against your best prompt.
- 20A brand image kitOn-brand images from one saved workflow, with your own LoRA.
- 21A launch videoThirty to sixty seconds, with voice, music and sound.
- 22A tested, locked-down appEvery flow checked, and each person able to see only their own data.
- 23Your launched productThe finished product, in front of people, with a page that shows your work.
You start with
A product that can charge
Email from your own domain and a tested checkout, in test mode.
Milestone 18Your product gains
A measured fine-tuning experiment
That makes 19 of 23 pieces in place.
Your role
The steps change with your role. Choose yours to see your version of this milestone.
Start with this prompt
Paste it into Claude or ChatGPT, or into Claude Code once you are building. Fill in the parts in square brackets with your own product.
I am considering fine-tuning a model for one task in my product. The task is: [describe it]. Here are five real examples of input and the output I want: [paste] First tell me what I should try before fine-tuning, and how I would know it was not enough. Then help me design the experiment: what a training example looks like, how many I need to start, how to split a hold-out set, and a yes-or-no rubric for judging outputs. Work with me one step at a time. Ask me one question, wait for my answer, then go on. Do not write the finished thing for me: show me a draft, tell me what is weak about it, and let me decide.
Build it step by step
4 steps, one from each lesson of the unit. Do each step after its lesson. A box you tick here is ticked in the lesson too.
Step 1 of 4 · from the lesson When to fine-tune
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA model tuned for one job Decide whether your product needs fine-tuning
Pick the one task in your product that a model does most often. Write down what you have already tried on the ladder and how well it works, using real examples. Decide: no need, or worth an experiment. Either way, continue the unit to learn the method on a small task.
Done when
The ten examples are the start of your dataset.
Step 2 of 4 · from the lesson Training data
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA model tuned for one job Build a small dataset
For the task you chose, build at least fifty examples in chat format: real inputs, with outputs you have checked line by line. Remove private details. Split them into a training file and a hold-out file.
Done when
Keep both files. They go into your repository if they contain nothing private.
Step 3 of 4 · from the lesson Fine-tuning with Unsloth
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA model tuned for one job Train your first LoRA on a text model
Open an Unsloth notebook for a small open model. Run your hold-out inputs through the base model and save the outputs. Then train on your train.jsonl, export to GGUF, and load the result in LM Studio.
Done when
Keep the notebook with your settings. It is how you, or anyone, can reproduce the run.
Step 4 of 4 · from the lesson Is it better?
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA model tuned for one job Compare, decide and publish
Run your hold-out inputs through the base model, your best prompt and your tuned model. Score them blind against a rubric, add the costs, and write your decision. Publish the experiment to GitHub.
Done when
Model files are large. Leave them out of the repository and say in the README where they are kept.
Publish it to GitHub
Your product repository
Add the finetune/ folder: the notebook, the rubric, the results and a README. Leave model files and private data out. Commit and push.
A milestone is not finished until it is pushed. A public repository with a history is how other people, including employers, can see what you built. Never commit a key, a token or anyone’s private data.
When it is done
Every box above is ticked, the work is pushed to GitHub, and you can show the result to someone who has not seen it before. Then move on: milestone 20 is a brand image kit.