Vibe Coder · Unit 17: RAG and Your Data · Project
Milestone 17 of 23
A helper that answers from your documents
Answers with sources, scored on your own questions.
One product, built one milestone at a time
Vibe Coder has one project. You pick an idea in Unit 1 and every unit adds a piece to it, until the last unit launches it. Every milestone ends with a push to your own GitHub repository. A model does not know your product. This milestone gives yours a helper that answers from your own documents and says where each answer came from, and it measures how often the helper is right.
- 1Your builder’s kitA one-line product idea, your role, and five working accounts.
- 2A fact-checked product briefOne page on your idea, tested against how the models really behave.
- 3Your product project in ClaudeA project with instructions and files, and your first real piece of work from it.
- 4A brand kit and research fileA name, images and market research for your product, with a second opinion.
- 5Your prompt libraryThe prompts you will reuse to build the product, each one tested.
- 6Your product specThe problem, the people, the first version and how you will measure it.
- 7A clickable prototypeA version people can click through, shown to three of them.
- 8A working first draftYour product generated from the brief, with sign-in and data, in your own GitHub repository.
- 9The first working versionYour app running on your computer, built with Claude Code.
- 10Your Claude Code setupProject instructions, a custom command and a skill made for your app.
- 11One AI featureA feature in your app that calls a model, with its key kept safe.
- 12A connected assistantClaude or ChatGPT connected to a tool you already use, with the right permissions.
- 13Sign-in and saved dataAccounts and a database behind your app.
- 14Your app, liveOnline at its own address, updating every time you save.
- 15Your product’s back officeThe automatic jobs behind your product, as tested workflows in GitHub.
- 16An agent that works for your productOne agent with narrow tools, limits, approval and a test set.
- 17A helper that answers from your documentsAnswers with sources, scored on your own questions.
- 18A product that can chargeEmail from your own domain and a tested checkout, in test mode.
- 19A model tuned for one jobA small experiment, measured against your best prompt.
- 20A brand image kitOn-brand images from one saved workflow, with your own LoRA.
- 21A launch videoThirty to sixty seconds, with voice, music and sound.
- 22A tested, locked-down appEvery flow checked, and each person able to see only their own data.
- 23Your launched productThe finished product, in front of people, with a page that shows your work.
You start with
An agent that works for your product
One agent with narrow tools, limits, approval and a test set.
Milestone 16Your product gains
Answers from your own documents, with sources
That makes 17 of 23 pieces in place.
Your role
The steps change with your role. Choose yours to see your version of this milestone.
Start with this prompt
Paste it into Claude or ChatGPT, or into Claude Code once you are building. Fill in the parts in square brackets with your own product.
I am building a helper for my product that answers questions from my own documents. Here is what the product is: [paste]. Here are the kinds of document I have: [list them] Help me write ten real questions a user would ask, and for each one say which document should contain the answer. Tell me which answers I probably have not written down anywhere yet. Work with me one step at a time. Ask me one question, wait for my answer, then go on. Do not write the finished thing for me: show me a draft, tell me what is weak about it, and let me decide. Do not answer the questions yourself. I want to find the gaps in my documents.
Build it step by step
6 steps, one from each lesson of the unit. Do each step after its lesson. A box you tick here is ticked in the lesson too.
Step 1 of 6 · from the lesson What RAG is
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Gather your knowledge
Decide what your product’s helper should be able to answer. Collect the source material for it: help text, policies, product details, pricing rules. Put it in one folder, remove anything out of date, and write ten real questions it should answer.
Done when
The ten questions are your test set for the whole unit.
Step 2 of 6 · from the lesson Answers from your sources with NotebookLM
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Test your sources in NotebookLM
Add your source folder to a NotebookLM notebook and ask all ten test questions. Record the result of each, open the citations, and fix the documents until every question is answered correctly from the right source.
Done when
Keep the cleaned folder. It is the input to everything you build next.
Step 3 of 6 · from the lesson File search: hosted RAG
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Add a help box that answers from your documents
Create a vector store from your cleaned source folder. Add a help feature to your product that sends a question to the model with file search enabled and shows the answer with its sources. Run your ten test questions through it.
Done when
Save the results table. You will compare it with your own pipeline later.
Step 4 of 6 · from the lesson Web data with Firecrawl
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Bring your web content in
Use Firecrawl to turn your own website or help pages into clean Markdown. Check a sample by eye, then add the pages to your source material with the address and date of each.
Done when
If your product depends on someone else’s pages, write down their terms on automated access before you index anything.
Step 5 of 6 · from the lesson Vectors in Supabase
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Build your own retrieval in Supabase
Enable pgvector, create a chunks table with a user_id and a row level security policy, and build two n8n workflows: one that loads your documents, and one that answers a question with sources. Run your ten test questions and the two-account test.
Done when
Export both workflows as JSON for your repository.
Step 6 of 6 · from the lesson Checking RAG answers
Read the short notes and pictures.Do everything inside a blue frame.
ActivityBuild stepA helper that answers from your documents Score, fix and publish your helper
Score your helper on your test set: retrieval and answers separately. Make at least two improvements, re-scoring after each. Then publish the work to your GitHub repository.
Done when
Link the rag/ folder from your main README.
Publish it to GitHub
Your product repository
Add the rag/ folder: test questions, the results table, the workflows or code, and a README. Leave private documents out. Commit and push.
A milestone is not finished until it is pushed. A public repository with a history is how other people, including employers, can see what you built. Never commit a key, a token or anyone’s private data.
When it is done
Every box above is ticked, the work is pushed to GitHub, and you can show the result to someone who has not seen it before. Then move on: milestone 18 is a product that can charge.