Measuring Hallucination in Vision-Language Models: A Claim-by-Claim Method

The short answer: The most reliable way to measure hallucination in vision-language models is claim by claim: split each answer into individual factual claims, have trained raters check each one against the image and label it supported, unsupported or contradicted, then report rates by claim type, such as objects, counts, attributes, positions, text and chart values, and by image type. Whole-answer scores hide where models fail. Teams usually run this through specialist multimodal AI evaluation programs.

Why fluent answers hide hallucination

Vision-language models produce smooth, confident descriptions. A caption may correctly describe a kitchen and then mention a cat that is not there, or say there are three people when there are four. Rated as a whole, such an answer often looks good, and automated text similarity to a reference caption may score it well too. Users who rely on the answer, especially people using it for accessibility, document reading or decision support, are misled.

Measuring hallucination properly means looking at each statement the model makes and checking it against what is actually visible.

Split answers into individual claims

The first step is to break each answer into atomic claims, each making one checkable statement: there is a red car, the car is on the left, there are two people, the sign says Exit. Splitting can be done by trained raters or by a model with human review, as long as the rules are consistent. Opinions and stylistic phrases, such as a beautiful scene, are excluded or labeled separately.

Atomic claims make evaluation precise. Instead of an answer being mostly right, teams learn that it made seven claims, six of which were supported.

Label each claim against the image

Raters check each claim against the image and label it supported, if the image clearly shows it; contradicted, if the image clearly shows otherwise; or unsupported, if the image does not show enough to confirm it. Some programs add a label for claims that need outside knowledge. Clear examples for each label keep raters consistent.

Contradicted and unsupported claims are both problems, but they mean different things. Contradictions are outright errors; unsupported claims are guesses presented as facts.

Different claim types fail differently

Objects, counts, attributes such as color and size, spatial relations, actions, text in the image and values in charts each have their own failure patterns. Models often do well on prominent objects and poorly on counts, small text, spatial relations and chart values. Tagging each claim with its type lets teams see these patterns.

Charts, screenshots and documents deserve their own test sets, because reading numbers and labels accurately is a distinct skill. See document annotation for how document and table data is labeled.

Building test sets that expose hallucination

Test sets should include images where hallucination is likely: cluttered scenes, unusual objects, images where expected objects are missing, small text, ambiguous counts and charts with many values. Prompts should vary from open descriptions to specific questions, because open prompts invite more invention.

Sets should be balanced across image types, kept separate from training data and versioned, so results can be compared across model releases.

Rater training and agreement

Raters need training on the label definitions, examples of borderline cases and practice on gold images with known answers. Agreement between raters should be measured per claim type, because some types, such as spatial relations, produce more disagreement. Low agreement usually means a definition needs work.

Raters should zoom and inspect images carefully, especially for counts and text. Fast, casual review misses the very errors the evaluation is meant to find. See data labeling for gold-set and agreement methods.

Reporting hallucination rates that teams can act on

Useful reports show the share of claims supported, unsupported and contradicted, broken down by claim type, image type and prompt type, with examples of each failure. Answer-level summaries, such as the share of answers with at least one contradiction, complement claim-level rates.

Comparing two model versions on the same set, claim by claim, shows whether a change reduced hallucination or just changed its form, for example fewer invented objects but more wrong counts.

Hallucination and safety

Some hallucinations carry real risk: misreading a medication label, inventing text in a document, describing a person’s identity or emotional state without basis or misreading safety signs. Evaluation programs should flag these high-risk claim types and track them separately.

Adversarial testing, where testers try to provoke harmful or misleading outputs, is a separate program. See AI safety testing for that work.

Where automated checks help

Automated tools can speed claim-level evaluation: object detectors to flag likely missing objects, OCR to check text claims and models to split answers into claims. They should support human raters, not replace them, because detectors and OCR make their own errors. A common pattern is automated pre-checks with human review of every disagreement and a sample of agreements.

Automated graders built on vision-language models should themselves be validated against human labels before they are trusted.

How prompts change hallucination

Open prompts such as describe this image in detail invite more invented details than specific questions. Leading prompts, such as asking what the dog is doing in an image without a dog, test whether models resist false premises. Test sets should include all three prompt types, reported separately.

False-premise prompts are especially revealing, because a model that plays along with a false premise will mislead users who ask in good faith.

Running claim-level evaluation

Start with a small, hard test set, define claim types and labels, train raters on gold examples and run a baseline on your current model. Then rerun on every release and track rates by type over time.

Our multimodal AI evaluation outsourcing page explains how vetted Philippine raters evaluate vision-language, image, video and audio models, and diarization error rate explained covers evaluation of audio models in the same cluster.