An answer can be clear, polite, and wrong in exactly the place that matters. A support assistant might quote the right product description and then add a delivery promise that the source never made.<\/p>\n
I treat generation and validation as separate responsibilities. The model proposes a response. The application decides whether that response satisfies the evidence and product rules required for delivery.<\/p>\n
Not every sentence needs the same treatment. A greeting has little factual content. A price, product identifier, warranty condition, or link can directly influence a customer’s decision.<\/p>\n
I begin by identifying those concrete claims. The validator then compares them with the retrieved sources or authoritative structured data. It should preserve relationships: finding the number 49 in a source is not enough to support a price of \u20ac49 for the selected product.<\/p>\n
Currency, variant, quantity, date, and market can all change the meaning of a value. A useful check asks whether the complete claim is supported in the relevant context.<\/p>\n
A model can produce a plausible URL that does not exist. It can also produce a real URL that points to the wrong product. I prefer links selected from verified source records rather than paths assembled from generated text.<\/p>\n
The link check needs an explicit normalization policy. Comparing only domains would accept many incorrect destinations. Comparing raw strings without handling legitimate formatting differences can reject valid links.<\/p>\n
If the application fetches a generated URL to validate it, that fetch introduces its own security boundary. In many cases, matching against a controlled source catalog is simpler than allowing arbitrary network requests from the validator.<\/p>\n
Structured product identifiers and known prices are good candidates for deterministic validation. Open-ended policy paraphrases are harder: matching keywords cannot prove that a sentence preserves the original conditions.<\/p>\n
A model-based verifier can help inspect semantic claims, but it introduces another fallible component. I would calibrate it against human-reviewed examples and keep high-impact rules explicit. Asking the same generation model whether it was correct is weak evidence on its own.<\/p>\n
A possible result structure is:<\/p>\n
{\n \"decision\": \"needs_revision\",\n \"unsupported_claims\": [\"delivery_by_requested_date\"],\n \"allowed_source_ids\": [\"shipping-policy-v3\"],\n \"next_action\": \"ask_for_postcode\"\n}<\/code><\/pre>\nThis is an illustrative contract. The important part is that failure produces an actionable state instead of a generic Boolean that downstream code can ignore.<\/p>\n
Plan what happens when validation fails<\/h2>\n
The application may regenerate once using narrower evidence, remove an unsupported claim, ask for missing information, or hand the conversation to a person. That choice should depend on the reason for failure.<\/p>\n
Retries need a limit. Repeatedly generating until a response passes can increase cost while selecting for weaknesses in the validator. Every attempt should remain within the request’s time and spending budget.<\/p>\n
Streaming also changes the design. Text already shown to the customer cannot be made unseen. If the product promises validation before delivery, high-impact claims need to be buffered or generated from approved structured fields before they are released.<\/p>\n
Keep evidence validation separate from instruction trust<\/h2>\n