{ "@context": "https://schema.org", "@type": "Article", "schema_version": "1.1", "content_item_id": "claim-evidence-audit-generative-search.en", "translation_group_id": "claim-evidence-audit-generative-search", "locale": "en", "type": "guide", "section": "knowledge", "slug": "claim-evidence-audit-generative-search", "title": "GroundedGEO: how to check whether a claim is supported by a source", "description": "A clear reading of the GroundedGEO preprint and a practical editorial audit: detailed writing is not the same as supported writing. The result comes from a laboratory test, not from Google, ChatGPT, or Yandex.", "direct_answer": "The GroundedGEO preprint shows in a laboratory test that an ordering algorithm can sometimes reward detailed variants that lack support when it does not check the link between claims and evidence. The authors match each claim to supporting records and treat unsupported detail separately. This does not establish a universal factor affecting visibility or say how Google, ChatGPT, Perplexity, or Yandex rank pages: the study covers 50 commercial queries, 1,950 cases, and several frozen models.", "sections": [ { "heading": "1. What GroundedGEO actually tests", "paragraphs": [ "The paper starts with a boundary that is easy to lose: support is not a property of writing style but a relation between an individual statement and a record that supports it. A product listing can be long, confident, and full of specifications without those specifications being verified. GroundedGEO makes that relation an explicit object of evaluation during ordering.", "The practical lesson for GeoAeoAle is not to announce a new secret AI-search factor. It is an audit discipline: extract verifiable statements, show which records support them, and only then assess relevance and answer quality. The preprint tests that controlled setup, not a live search-engine result." ], "bullets": [ "Observation: what the test interface or ordering algorithm produced.", "Inference: a bounded explanation across observations.", "Hypothesis: a possible cause that needs a separate test." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "2. Why detailed text can mislead an ordering algorithm", "paragraphs": [ "The authors describe relevance inflation: a candidate adds query-aligned specifications, use cases, or credentials and therefore looks more relevant. If the system mostly reads textual compatibility, it may promote that candidate even when the added details are unsupported. This does not mean detailed pages are bad. Supported detail is valuable; the problem is that a supported and a fabricated rich variant can look equally convincing at the interface.", "That is why a simple length, term-density, or formal-style filter is not enough. It can punish an honest detailed page or miss a carefully written fabrication. GroundedGEO checks the relation between statements and supporting records rather than the appearance of prose, using atomic statements and a supplied record set." ], "bullets": [ "Detail is a relevance signal, not proof.", "A nearby link does not guarantee support for every sentence.", "The audit must preserve scope and source strength." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "3. The laboratory test", "paragraphs": [ "The laboratory sample contains 50 commercial queries and 1,950 cases. Candidates are constructed as matched groups: starting measurement, rich, supported, or neutral, while some experiments use pairs with identical text—fixed text with different supporting-record confirmations. This design lets the researchers separate text volume and format from the presence of supporting records.", "The paper also describes different sources of supporting records, including official records, buyer reviews, and registry corroboration. Human annotation covers 370 statements, and annotator identities remain private. These figures describe the experiment design. They are not a sample of the whole web and should not be read as the share of sites that receive an advantage in live generative search systems." ], "bullets": [ "Sample: 50 queries and 1,950 cases.", "Separate human-labelled set: 370 statements.", "Unit of analysis: candidate, statements, supporting record set, and ordering algorithm decision." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "4. What happened to unsupported rich variants", "paragraphs": [ "On the frozen Qwen2.5-7B ordering algorithm, the authors report a significant normalized rank gain for unsupported detailed variants: +0.065 to +0.092 across three claim profiles after Holm correction. In that controlled comparison, variants with unsupported details gained over starting measurement candidates. It would be incorrect to rewrite this as 'every AI system promotes long pages.'", "The result was model-dependent. The reported MiMo-v2.5 gains were not significant, while comparisons for GLM-5.3-Flash were near zero and also non-significant. That contrast matters: the paper demonstrates a vulnerability in particular tested ordering algorithms and settings, not a general law of generative search." ], "bullets": [ "The positive gain is relative to a starting measurement arm in a laboratory test.", "+0.092 is not traffic, a link in an answer, or a Google position percentage.", "A non-significant result for another model does not prove complete safety." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "5. What pairs with identical text demonstrate", "paragraphs": [ "Pairs with identical text are one of the most useful design elements. The candidate text stays the same while the supporting-record set changes: one version has confirmations or records and the other does not. If rank changes with that difference, the experiment shows that the decision received information about support rather than only prose form.", "The comparison shows that identical text can be evaluated differently when the available records change. It is not a finished publication-control system. In real editorial work, records may be incomplete, contradictory, stale, or poorly connected to a claim. Each record therefore needs a source, date, scope, and qualification rather than a simple present-or-absent flag." ], "bullets": [ "Fixed text controls for wording effects.", "A record set does not make a source true automatically.", "Matching a claim to an exact excerpt is separate from finding the source domain." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "6. Can an automatic check replace an editor", "paragraphs": [ "No. The paper does not support that conclusion. Against a human-labelled set of 370 statements, all tested automatic checkers failed the preregistered reliability gate. The best local checker preserved 79–100% of the reduction achieved with ideal labels. In the protected comparison groups, it showed no measured removal of candidates that should have remained, but that is still not a universal quality guarantee.", "For an editorial system, this calls for separate modes. A machine can suggest a match between a claim and an exact source excerpt, flag a missing record, and quarantine unclear source history. But a changed quantifier, a conflict between two documents, or an appropriate conclusion strength must remain visible and reviewable. An automatic label should be a transparent aid, not a way to hide uncertainty." ], "bullets": [ "An automatic suggestion is not a human-labelled set.", "Mistaken removal must be measured separately from successful matches.", "A weak source cannot become a strong statement through classification." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "7. A site-level check of claims against sources", "paragraphs": [ "A practical workflow can start with a claim register. For each factual sentence record the subject, action, object, observation or inferred status, source reference, exact excerpt, document date, and limits. Then test whether the source supports the same quantifier: 'available to some users' cannot become 'available to everyone', and 'a link was displayed' cannot become 'a visit occurred'.", "The next step is a decision trace. A page passes when its statements have suitable public records and unsupported detail is removed or marked as a hypothesis. Keep the numerator and total count for every number, and keep crawling, a system finding a page, a link in an answer, a mention, traffic, and conversion as separate events. Even without building an ordering algorithm, this improves human control and publication explainability." ], "bullets": [ "Extract statements before selecting supporting records.", "Store the exact excerpt and date, not only a URL.", "Check that the RU and EN versions match in numbers, quantifiers, and conclusion strength.", "Publish limits next to the result." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] }, { "heading": "8. Where the result stops applying", "paragraphs": [ "GroundedGEO is a laboratory study of ordering integrity. It does not measure Google Search, Google AI Overviews, ChatGPT Search, Perplexity, Yandex, or any other live platform. The paper does not show that adding records to a page will make it rank higher, appear as a linked source, attract visits, or generate sales. It also does not establish that every text-only ordering algorithm is vulnerable in the same way.", "The narrow supported conclusion is this: when an ordering algorithm reads relevance but does not verify statement support, unsupported detail can receive a reward in some tested conditions; support for each statement is therefore a sensible object for separate auditing. Transfer requires a new experiment with the target platform, a declared protocol, and direct observations—not a link in an answer to this preprint." ], "bullets": [ "Do not transfer the numbers to live AI-search systems.", "A record's presence does not prove that it supports the claim; check the exact excerpt.", "Do not promise higher positions or AI mentions.", "Measure availability, system finding a page, a link in an answer, mention, and visits separately." ], "source_ids": [ "arxiv-groundedgeo-2026-09-21" ] } ], "published_at": "2026-09-29", "modified_at": "2026-09-29", "data_through": "2026-09-21", "next_review_at": "2026-10-29", "author": "GeoAeoAle Editorial", "origin": "editorial", "topics": [ "grounded-generation", "evidence", "generative-search", "ranking-integrity", "measurement" ], "publisher": "GeoAeoAle Editorial", "license": "https://creativecommons.org/licenses/by/4.0/", "canonical_url": "https://geoaeoale.com/en/knowledge/claim-evidence-audit-generative-search/", "claims": [ { "claim_id": "benchmark", "text": "The laboratory test contains 50 commercial queries and 1,950 cases.", "status": "observed", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" }, { "claim_id": "unsupported-gain", "text": "In the frozen Qwen2.5-7B test, detailed variants without supporting records showed a normalized rank gain of +0.065 to +0.092 across three claim profiles after Holm correction.", "status": "observed", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" }, { "claim_id": "model-dependence", "text": "The result depended on the model: it was marginal for MiMo-v2.5 and did not appear in the reported GLM-5.3-Flash comparisons.", "status": "observed", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" }, { "claim_id": "oracle-defense", "text": "With ideal support labels, the share of detailed candidates without support in the first three places fell from 0.65 to 0.43 at lambda 40; no mistaken removal of supported candidates was measured in that arm.", "status": "observed", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" }, { "claim_id": "automation-gap", "text": "All tested automatic checkers failed the preregistered reliability threshold on 370 human-labelled statements. The best local checker preserved 79–100% of the reduction achieved with ideal labels and showed no measured removal of candidates that should have remained in the protected comparison groups.", "status": "observed", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" }, { "claim_id": "scope", "text": "The paper tests ordering algorithms in a laboratory; it does not establish behavior in Google, ChatGPT, Perplexity, Yandex, or a universal ordering factor.", "status": "inference", "confidence": "high", "source_ids": [ "arxiv-groundedgeo-2026-09-21" ], "publication_status": "public", "confidentiality": "public" } ], "sources": [ { "source_id": "arxiv-groundedgeo-2026-09-21", "canonical_url": "https://arxiv.org/html/2609.25189", "title": "GroundedGEO: Auditing the Evidence Gap in Generative Search Rankings", "publisher": "GroundedGEO authors / arXiv", "source_type": "primary-research", "locale": "en", "published_at": null, "checked_at": "2026-09-29", "sha256": "394012f1cdd832bc14d5e6878114797fc8f2945fe240163c0c3735459df1d151", "license": "arXiv.org perpetual non-exclusive license", "visibility": "public" } ], "related_slugs": [ "measuring-ai-visibility", "visible-links-search-record", "cross-language-ai-search-measurement" ], "limitations": [ "The preprint tests laboratory ordering algorithms, not live Google, ChatGPT, Perplexity, or Yandex.", "The sample contains 50 commercial queries and 1,950 cases, so it does not describe the whole web.", "Effects depend on the model, claim profile, record set, and frozen evaluation setup.", "The human-labelled set contains 370 claims; automatic-checker results cannot be generalized to every task.", "Oracle support labels are not an ordinary live-system signal and do not solve evidence completeness.", "The paper does not measure positions, traffic, visits, sales, or causal effects on search visibility." ], "corrections": [] }