The same prompt in five services is five different tests

When one prompt is sent to several services, teams compare the visible answers but often forget the search interface or mode. One service may have web search, another files or special databases, and another no external sources. Even within one service, mode and settings change the available context.

This does not make comparison impossible. It changes what the comparison means. You can compare the user experience as it is, or align conditions for a controlled test. Do not present one design as the other.

A controlled design fixes mode, web access, language, region, account, date, and available context. Its purpose is to make services as comparable as possible for one hypothesis. A user-experience design preserves defaults and answers a different question: what will an ordinary person see on the chosen interface or mode?

Before running, write down which design you chose. If a parameter is unknown, record “unknown.” Do not fill in assumptions about model, personalization, or search access after the fact.

A minimum passport contains service, mode, shown model or version, web access, language, region, account type, date and timezone, full prompt, full answer, and all visible sources. If the prompt ran inside an ongoing dialogue, preserve prior context or start a new dialogue.

For a repeat measurement, add page version, prompt set, and coding rules. A reviewer should understand what counted as a mention, correct fact, source link, link, and recommendation.

A ChatGPT Search answer and a ChatGPT answer without search may solve different tasks. Perplexity modes may also use different material pools. Google describes query fan-out for its AI features as branching a query into subtopics; that is not a universal property of every service and is not a reason to transfer its results to the others.

Keep separate rows for service and mode. An average over mixed rows may look precise while hiding which interface or mode produced the result and under which conditions.

One answer may contain a brand mention, a correct description, a visible link, and a useful recommendation at once–or only the first. Code these events separately. In particular, distinguish a source link from a link that leads to the person’s intended action.

If you need a metric, publish numerator, total count, period, and exclusions. “Five of twenty runs gave a link” is checkable. “The service cites better” without an interface or mode definition and counting rule is not.

A passport does not remove index changes, personalization, or generation randomness. It makes them visible. One run remains an observation; repeated runs under one protocol support a discussion of stability within the chosen interface or mode.

Before the next comparison, choose one question, one design, and one field set. Run a pilot, verify that complete answers and URLs are archived, then scale the series. Do not add services merely to make the table more impressive.