Module 05
Measuring Trust
"The demo went great" is not a quality bar. This module is about how you actually prove an AI works, and hold a vendor to it.
By the end of this module you'll be able to
- Explain why you can't judge an AI by eye, and what to measure instead.
- Use three plain checks to catch the most common failure, a confident, made-up answer.
- Ask any vendor five questions that reveal whether they actually measure quality, or just hope for it.
Explainer · why vibes don't scale
You can't eyeball an AI
The most dangerous moment in an AI project is the demo that goes well. A handful of impressive answers feels like proof, but you've seen, by now, that the same system can answer differently next time, and that its most convincing failures are the ones that look right. Spot-checking a few replies tells you almost nothing about the thousands of real interactions to come. None of this makes demos worthless; they're genuinely useful for early exploration and for getting everyone aligned on what "good" even looks like. They just can't stand in for measurement before the system meets real customers.
What you need is the same discipline you'd expect for any other software: tests. In the AI world these are called evaluations, or "evals," a structured way to measure whether the system is doing its job across many cases, not just the cheerful ones in the demo. The rule is simple: you wouldn't ship software with no testing, so don't ship AI on vibes.
The catch is that grading AI output is harder than grading ordinary software. Traditional measures just checked whether the answer's words overlapped with a "correct" reference: useful for a translation, useless for an open-ended answer that could be phrased a hundred good ways. A reply can be worded completely differently from the reference and still be perfect, or echo it word-for-word and still be wrong. So the field needed a better way to measure meaning.
Explainer · grading at scale
Using AI to grade AI, and three checks that catch the big failure
The breakthrough was deceptively simple: use a strong AI as the grader. You hand a capable model an answer, a clear rubric, and the question, and ask it to score the result, at a scale and speed no human team could match. This is called LLM-as-a-judge, and (as the research below shows) it agrees with human reviewers often enough to be genuinely useful.
That grading approach powers the most practical quality framework for the systems most businesses actually deploy: an AI that looks things up in your documents before answering (the technical name is "retrieval-augmented generation," or RAG). For those systems, three checks catch the failure executives fear most: a confident answer that's simply made up.
- Context relevance: did it pull the right information? Before answering, the system searches your documents. If it grabs the wrong passages, everything downstream is built on sand.
- Groundedness: is the answer actually supported by what it found? The most important check. It catches hallucination, the AI stating something the source never said. Every claim should trace back to the retrieved text.
- Answer relevance: did it actually answer the question? A reply can be perfectly accurate and still miss the point. This confirms the answer addresses what was actually asked.
| The old way | The modern way | |
|---|---|---|
| What it checks | Word overlap with a "correct" answer. | Meaning, accuracy, and groundedness. |
| Open-ended answers | Breaks down: many good answers look "wrong." | Handles them: judges the substance, not the phrasing. |
| Catches made-up facts? | No. | Yes: that's what groundedness is for. |
Three refinements. First, "meaning" isn't one fixed thing: what you measure depends on the job. A document-lookup system lives or dies on faithfulness to its sources; a conversational agent is judged more on helpfulness and safety. Match the checks to the application rather than reaching for one universal score. Second, the triad is a strong automated baseline, not a complete one: a system can retrieve the right document, ground every sentence in it, and still omit a critical caveat or misread a subtle passage. Third, groundedness itself leans on the retrieval step and on cleanly separating each claim: if the right source was never pulled, or a single claim spans several documents, the check can misfire. For high-stakes answers, keep occasional human audits alongside the automated score.
Your P&L
Trust, but require the receipts
An AI system with no evaluation is an unmonitored one: you're flying blind, and you'll find out it drifted off course from an angry customer, not a dashboard. Worse, the AI may have looked fine at launch and quietly degraded when the underlying model was updated. Measurement isn't a one-time gate before go-live; it's an ongoing instrument you keep watching, because the ground moves underneath these systems. Right-size it to the stakes: not every deployment needs a full release pipeline, but even a small one benefits from a stable set of test questions you re-run after any major change.
So the practical move with any vendor or internal team is to require the receipts. Don't accept "it works great"; ask how they know, on your data, and ask to see it. The five questions in the lab below do exactly that. A serious partner will have ready answers. A vendor whose only evidence is a polished demo has just shown you the size of the risk. One fair exception: a genuinely early proof-of-concept may not have a full eval suite yet, and there the good sign isn't a finished scorecard but a vendor who offers to build and run the evaluations with you, instead of waving at "great performance."
Evidence file
Can you really grade AI with AI? The research says yes, mostly.
Two studies that made automated evaluation credible
It's fair to be skeptical of using AI to grade AI. The research is reassuring on the central question. In one widely cited study, a strong model acting as judge agreed with human reviewers more than 80% of the time, about the same rate at which two humans agree with each other. A second study showed that this kind of AI grading lined up with human judgment far better than the old word-overlap measures it replaced.
The same research is honest about the limits, and so should you be. AI judges have measurable biases: they tend to favor longer answers, and sometimes their own writing style. Some of that can be dialed down (normalizing for length, randomizing the order answers are shown in, writing the rubric carefully) but it doesn't disappear, and "more than 80%" is still not 100%. That's why mature teams treat automated evals as a powerful way to watch quality at scale, backed by regular human spot-checks on the cases that matter most. It's an instrument, not an oracle.
Sources: Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," 2023; Liu et al., "G-Eval," 2023; RAG-triad framework from the TruLens open-source project (TruEra). Open tools in this space include TruLens, Ragas, DeepEval, and Promptfoo.
Labs · run these yourself
Be the grader
The first lab lets you watch an AI catch a made-up fact. The other two are the habits you'll use to keep any AI, or any vendor, honest. As a reminder from the disclaimer up top: use made-up values only, never real customer data or anything confidential.
Catch the hallucination
~4 minutes · any chatbot
- Paste this into a fresh chat. You're asking the AI to act as a groundedness checker:
The vendor scorecard
~5 minutes · keep this one
Five questions for any AI vendor or internal team. Score each answer as specific (good) or reassuring but vague (a flag):
- How do you measure answer quality on our data, not a generic benchmark?
- What's your rate of made-up or unsupported answers, and how do you track it?
- Can we see the test set, and re-run it ourselves?
- When the underlying model updates, how do you catch a drop in quality before customers do?
- Where do humans review, and which cases always get a human's eyes?
Run the triad by hand
~2 minutes · any AI answer that used a document
Next time an AI gives you an answer based on a file, a search, or a knowledge base, run the three checks yourself before you trust it: (1) Did it pull from the right source? (2) Is every claim actually in that source, or did some appear from nowhere? (3) Did it answer what you actually asked? Do this a few times and it becomes instinct, and you'll start noticing the confident additions that aren't backed by anything. That instinct is the most valuable output of this whole module.
Beyond your four questions
Modules 01 to 04 gave you a way to assess an AI's risk: what it sees, whose instructions reach it, what it can do, and where a human approves. This module adds the other half of the job: proof that it works, and works still. Risk control keeps the bad outcomes contained; measurement keeps the good outcomes real. In practice the two aren't separate tracks: an eval that flags biased or harmful answers is also a safety signal, and a control that limits what the AI can do will shape its quality. Treat them as two lenses on one system, not two silos. Module 06 brings both together into a single, practical playbook you can run, and the layered defenses that make AI safe enough to actually use.
Plain-language glossary
The terms from this module
- Evaluation ("eval")
- A structured test that measures whether an AI does its job across many cases, the AI equivalent of software testing.
- Hallucination
- A confident answer the AI made up, stated as fact but not supported by any real source.
- RAG
- "Retrieval-augmented generation," an AI that looks things up in your documents before answering. The common business setup.
- RAG triad
- Three checks, context relevance, groundedness, and answer relevance, that find where an AI's answer went wrong.
- LLM-as-a-judge
- Using a strong AI to grade other AI output at scale. Agrees with humans often, but has biases.
- Drift / regression
- Quality silently dropping over time, often after the underlying model is updated. The reason evals must be ongoing.
Check · lock in the one thing that matters
Three quick questions
Pick an answer for each, then check the key below.
-
Why are older measures (like word-overlap scores) a poor way to judge an AI's answers?
- They run too slowly on modern hardware.
- They check whether the wording matches a reference, not whether the answer is correct or makes sense, so a good answer phrased differently looks "wrong."
- They only work in English.
-
In the "RAG triad," which check is designed to catch a made-up (hallucinated) fact?
- Context relevance.
- Groundedness.
- Answer relevance.
-
What's the honest summary of "LLM-as-a-judge," using AI to grade AI?
- It's perfect and replaces all human review.
- It's useless and should never be trusted.
- It agrees with humans often enough (about 80% or more) to be a powerful, scalable instrument, but it has biases, so it's paired with human spot-checks, not treated as an oracle.
1. Answer: B. Open-ended answers can be phrased many good ways. Word-overlap rewards matching the reference's words, not conveying the right meaning, and it can't catch a confident, made-up fact.
2. Answer: B. Groundedness checks that every claim in the answer traces back to the source the AI retrieved. If a statement isn't supported by that source, it gets flagged, which is how you catch hallucination.
3. Answer: C. The research supports using AI graders at scale and is honest about their limits. Demand measurement; don't accept claims of perfection.
The one line to remember
Don't ship AI on vibes. Require measurement, especially groundedness, to catch confident made-up answers, and keep measuring, because quality drifts.