AI & ML Engineering · Unit 16 · AI Engineering

AI Evaluation

Testing AI systems the way software is tested, so changes do not break quality.

  1. 16.1

    Building an Evaluation Dataset

    Collecting real cases, edge cases and expected answers.

  2. 16.2

    Automated Metrics

    Exact match, similarity scores and task-specific checks.

  3. 16.3

    LLM-as-a-Judge

    Using a model to grade outputs, and checking the judge.

  4. 16.4

    Human Evaluation

    Rubrics, annotation and agreement between reviewers.

  5. 16.5

    Regression Testing

    Running evaluations on every change.

  6. 16.6

    Tracing and Observability

    Recording every step so failures can be explained.

Unit project

An evaluation suite for one of your earlier projects that runs automatically and catches a regression you introduce on purpose.

Paid learners get the full brief, submit it from their dashboard and have it reviewed. It goes in your portfolio.

Open every unit

Enrol once to open every unit, its videos and a reviewed project for your portfolio.