We Asked ChatGPT, Gemini and Google AI Mode the Same 40 Questions. They Agreed on 2 of 600 Sources.

Semrush's trend brief on AI citation fragmentation quotes a study of 3.7 million citations across 20,000 prompts: "only 2.37% of cited URLs appear in all three major LLMs for the same prompt, while 91% appear in exactly one." If that holds, "our AI visibility" is not one number. It is three, and they barely overlap.

Big studies average across every industry. We wanted to know what fragmentation looks like inside one category, on one evening, at the level of a single question — the level a marketer actually plans at. So on 9 October 2026 we asked the same 40 questions about AI software to ChatGPT, Gemini and Google AI Mode, and compared every source each one cited.

Of 600 distinct pages cited, 556 — 92.7% — were cited by only one engine. Two were cited by all three.

How we measured

• Questions: 40 about AI software, written in advance — 15 "best/which" lists, 10 comparisons, 10 questions about one product, 5 how-tos. The same panel we used to test whether ChatGPT's citations come from Bing's top 20.
• Engines: ChatGPT with web search forced on (the scraper reported gpt-6), Gemini's consumer app (reported as 3.5 Flash-Lite), and Google AI Mode. All three collected through DataForSEO, US English, between 18:32 and 18:59 UTC on 9 October 2026.
• Citations: ChatGPT's and Gemini's listed sources; AI Mode's reference links.
• Matching: per question. A page counts as shared if two engines cited the same URL — protocol, www, query strings, fragments and trailing slashes ignored — for the same question. We repeat the count at domain level.

What we found

Our panel reproduces the brief, almost exactly

A study of 20,000 prompts and our 40 land within two points of each other on single-engine citations. Our "all three" figure is lower still — in one category, on one evening, it was two pages.

Both of those pages answer the same kind of question. For "is gamma app free?" all three engines cited gamma.app/pricing. For "is suno ai free for commercial use?" all three cited the same Suno help-centre article. When the question is a narrow fact that only the vendor can state, the engines converge on the vendor's page. Everywhere else, they diverge.

Even Google does not agree with Google

The pairwise overlap is small for every pair, and smallest where you might expect it to be largest:

Gemini and AI Mode are both Google products. They overlap the most — and still share fewer than one cited page in ten for the same question. Visibility in one Google AI surface is not a proxy for the other.

Each engine has its own favourite sources

The engines did not just pick different pages. They drew from different kinds of source.

• Google AI Mode cited YouTube 22 times and Reddit 12 times. ChatGPT cited neither once. Gemini cited Reddit once and YouTube never.
• ChatGPT cited vendors' own sites heavily — canva.com (8), chatgpt.com (7), openai.com (6), elevenlabs.io (6) — and averaged 7.8 sources per answer, against Gemini's 3.4 and AI Mode's 4.9.
• Gemini leaned on blogs and explainers: eesel.ai, felloai.com and zapier.com were its most-cited sites. Three of its 40 answers cited nothing at all; so did three of AI Mode's.

Across the whole panel, the three engines cited 358 different websites. Only 12 were cited by all three engines anywhere in the 40 questions. The one that travelled best was zapier.com, cited once by ChatGPT, three times by Gemini and nine times by AI Mode.

The directory that wins ChatGPT is invisible on the other two

This is the finding that matters most to us. Our site is an AI-tools directory, and in our category one directory dominates ChatGPT's citations: when we pulled DataForSEO's LLM Mentions data for our competitors on 2 October, toolradar.com had 390 ChatGPT citations, more than any other AI-tools directory we checked. In this panel ChatGPT cited it on 5 of 40 questions — more than any other independent site ChatGPT cited.

Gemini cited it zero times. AI Mode cited it zero times.

The brief suggests measuring AI visibility three ways — Presence (share of prompts where you appear), Portability (whether the same URLs appear across engines) and Concentration (how dependent you are on one platform). Applied to that directory on our panel:

• Presence: 12.5% of questions on ChatGPT; 0% on Gemini; 0% on AI Mode.
• Portability: none of its cited pages appeared on a second engine.
• Concentration: 100% of its citations came from one platform.

A site tracking only ChatGPT would read that as a strong position. On the evidence of these 40 questions, it is a single point of failure.

We do no better, and our history shows how quickly a position on one engine moves. On 29 September ChatGPT cited one of our comparison pages as its first source for a question about maths solvers. On this panel — including a question about the same two products — none of the three engines cited us at all. A single-engine snapshot would have told us we had AI visibility. A three-engine panel says we have almost none.

The sources that travel have stronger link profiles

Why do a dozen websites reach all three engines when 305 others reach only one? We looked up every one of the 358 cited websites in DataForSEO's Backlinks database, which scores each domain's link profile from 0 to 1,000 (its own Domain Rank, not Semrush's Authority Score).

The websites that travelled had clearly stronger link profiles: blog.google (708), canva.com (693), elevenlabs.io (599), quillbot.com (575), zapier.com (546). But it is a tilt, not a gate. aicomparison.ai, with a Domain Rank of 197, was cited by all three engines — lower than the median of every group. It got there with two head-to-head comparison pages: ChatGPT and Gemini both cited its Adobe Firefly vs Canva AI page, and AI Mode cited its Midjourney vs DALL-E 3 page.

The engines also differ in how much weight they seem to give a strong profile. The median website ChatGPT cited scored 397; Gemini's scored 306; AI Mode's 348. ChatGPT leans towards established domains; Gemini reaches further down.

For comparison, toolradar.com — the directory ChatGPT cited five times and the other two never — scores 358. Our own domain scores 305. A strong profile helps a source travel, but in this panel it was neither necessary nor sufficient.

Some kinds of question fragment more than others

Single-product questions are the most consistent, because the vendor's own pages are the obvious source. "Best" lists — the open questions where independent publishers can still win — are the most fragmented of all. 95% of the pages cited for "best" questions were cited by one engine only. The questions with the most opportunity are also the ones where winning on one engine tells you least about the others.

What this means in practice

• Report AI visibility per platform. A blended score hides a 0% on Gemini behind a strong ChatGPT number. Ask for the split.
• Pick your engine by audience, then optimise for its source mix. If AI Mode matters to you, YouTube and Reddit are part of its diet; on our panel ChatGPT cited neither once.
• Own the narrow facts. The only pages all three engines agreed on were a vendor's pricing page and a vendor's help article. Clear, current pages that state one checkable fact are the closest thing to a portable citation.
• Re-measure often. The page of ours that ChatGPT cited first on 29 September was cited by none of the three engines ten days later, for a question about the same two products. Any of these numbers can move week to week.

What we could not test

• Claude and Perplexity. The brief's study covers more engines. We covered three, the three we could query in their consumer form on the same evening.
• Model and personalisation drift. Each engine was asked once, logged out, in US English. A second run would differ, and run-to-run variation is its own kind of fragmentation.
• Scale. 40 questions in one category is a case study, not a census. Its value is that it reproduces the large study's headline in a narrow vertical.
• What users click. We measured what each engine cited, not which citations people followed.

Run this on your own category

1. Write 20–40 questions your buyers ask, mixing lists, comparisons and questions about your product, before looking at any answers.
2. Ask each engine you care about on the same day, logged out, and save the cited sources — not the answer text.
3. Match per question, at page and at domain level. The domain view shows whether you are known; the page view shows whether you are cited.
4. Score yourself on presence, portability and concentration for each engine separately.
5. Find the sources each engine prefers — video, forums, vendor docs, blogs — and decide which ones you can realistically earn.
6. Repeat on a schedule. One snapshot is a photograph of something that moves.

Where Semrush fits

Steps two, four and six are what a tool should do daily, and what we did once, by hand, for one evening. The AI Visibility Toolkit in Semrush One builds the per-platform split into its Visibility Overview report. Its Knowledge Base, checked on 9 October 2026, describes an AI Visibility view to "Compare your overall visibility score against specific platforms like ChatGPT, Gemini, and AI Overview", and a Distribution by LLM panel to "see which AI models (e.g., ChatGPT, AI Overview) are referencing your brand most frequently." That is the concentration measure from this article, done for you.

The same page says the data covers "Google AI Overviews, AI Mode, Gemini, and ChatGPT" and is "refreshed daily on a rolling basis." At prompt level the report shows each engine's answer side by side — whether your brand is mentioned, whether it is cited, and how many sources that answer used. That is the gap in our table: the same question, a different engine, a different set of sources.

For the questions that matter most, Prompt Tracking inside Position Tracking — Semrush's Knowledge Base says "AI Search in Position Tracking monitors your rankings in ChatGPT Search and Google AI Mode" — lets you "Monitor your visibility for specific, high-value prompts on platforms like Google AI Mode and ChatGPT" day by day — the scheduled version of step six. Note that it lists Google AI Mode and ChatGPT; the platform split in Visibility Overview is where Gemini appears.

The finding about toolradar.com points at the other half of the job: knowing who wins where you do not. In classic search that is Keyword Gap, which Semrush's Knowledge Base says "offers a side-by-side comparison between keyword profiles of up to five competitors"; in AI answers it is Visibility Overview's Source Opportunities tab — "Sources frequently cited in AI answers that mention your competitors but not your brand." Run both against the rival that dominates one engine and you see whether its lead is portable or, like the directory above, confined to a single platform.

The link-strength table above is the question Backlink Analytics is built to answer for your own category. Its Knowledge Base lists a Network Graph report among its reports, and a Backlink Gap report that "allows you to compare backlink profiles of multiple competitors side by side." If the sources that reach every engine in your category share a link profile you lack, that is where to look first.

The only pages all three engines agreed on were a vendor's pricing page and a vendor's help article, so the pages that state your narrow facts have to be reachable by every engine's crawler. Site Audit covers that: Semrush's Site Audit page states that it "checks for AI crawler accessibility for the following bots: ChatGPT-User, OAI-SearchBot, Googlebot, Google-Extended, Perplexity-User, PerplexityBot, Claude-User, and Claude-SearchBot" — the agents behind the engines in this article and the two we could not include.

A planning note: the free trial belongs to Semrush One, not to the AI Visibility Toolkit bought on its own. Semrush One's page offers "Try free for 7 days" with "Unlimited access to all Semrush One tools", for new users only — checked on 9 October 2026. Read that as Semrush's terms, not our test; every number above was measured without a Semrush account.

What we actually learned

The brief's headline survives a narrow, single-category test almost unchanged. On 40 questions about AI software, 92.7% of cited pages were cited by one engine only, and the three engines agreed on two pages out of 600.

The detail underneath is more useful than the headline. Two Google products shared under one cited page in ten. AI Mode cited YouTube and Reddit; ChatGPT cited neither. The few websites that reached all three had stronger link profiles than the rest, though a small site with the clearest answer made it too. And the directory that leads ChatGPT's citations in our category was cited by neither of the other two engines even once — a position that looks strong on one dashboard and does not exist on the other two.

So stop treating AI visibility as one score. Measure it per engine, own the narrow facts that every engine converges on, and re-check before you trust any single snapshot — including ours.

Measure your visibility on each AI platform separately with Semrush One — start your 7-day free trial.

Sources and method

Written by Sohail Akhtar. Every figure here is a measurement on a stated date; the saved answers from all three engines and the matching script can be re-run.