Compare AI-search products without overclaiming

Compare named experiences

“AI search” is a category, not a single instrument. When recording a response, name the product and visible mode. Include the date, language, region when known, whether web search was used, and any signed-in context that could matter. If a setting is not visible, record it as unknown instead of filling in a guess.

Keep experiments away from confidential customer or business information. A public learning prompt does not need personal data. Do not assume that using an incognito window removes every source of variation.

Keep documentation and observations in separate columns

Documentation states what a provider describes or requires. An observation records what you saw in a particular run. Neither should be silently promoted into a claim about all platforms.

Google states that its AI search features do not require a special AI markup format. That statement concerns Google’s features; it does not establish another provider’s eligibility requirements. OpenAI’s crawler documentation should likewise be used for OpenAI crawler decisions, not as a substitute for another vendor’s rules.

Define what you are counting

Suppose a response names a brand but links only to a review site. You might record one brand mention, zero links to the brand’s site, and one third-party citation. Another response might cite the official site without naming the brand in the answer body. These are distinct observations, not contradictory scores.

Before looking at results, decide whether repeated links count once per answer or once per occurrence. State whether your unit is an answer, a prompt, a cited URL, or a day of aggregated telemetry. Otherwise two honest reviewers can create incompatible totals.

Use a small, purposeful comparison

Choose a handful of questions representing learning, evaluation, and troubleshooting tasks. Keep the wording fixed for a baseline. Save enough answer context to check your interpretation, where the product permits it. Repeat observations only when doing so answers a question, not until you obtain the result you hoped for.

Write the conclusion at the right altitude

Say “in these recorded runs, this product cited these URLs.” Do not say “we own AI search” or “this platform trusts our brand.” A small comparison can reveal misleading product descriptions or missing documentation worth fixing. It cannot estimate every buyer’s exposure, identify a private ranking mechanism, or prove a stable advantage.