Is Bing ChatGPT's Gatekeeper? We Checked 151 ChatGPT Citations Against Bing's Top 20

Semrush's trend brief on AI search opens with a number that has been repeated a lot this autumn: "~87% of ChatGPT citations match Bing top 20 results — meaning Bing indexing is now a direct requirement for AI visibility." The brief attributes it to an analysis by LLMVLab and draws the obvious conclusion: if Bing does not rank you, ChatGPT does not see you.

That is a claim we can test, and one we have a reason to care about. Our site is a Bing site. In the 24 hours to 18:55 UTC on 9 October 2026, Cloudflare logged 1,886 HTML page requests from Bingbot and 45 from Googlebot — Bing reads us roughly forty times as often as Google does. If Bing is the door to ChatGPT, we should be standing in it.

So on 9 October we asked ChatGPT 40 questions, recorded every page it cited and every search it ran to find them, then ran each of those searches on Bing and on Google and checked whether the cited page was in the top 20.

Bing explained more of ChatGPT's citations than Google did under every overall reading we tried — by a wide margin for the question as typed, by a hair for the searches ChatGPT itself ran. Under no reading did it explain anything close to 87%.

How we tested it

• Questions: 40, all from the category we cover — AI software. 15 "best/which" list questions, 10 head-to-head comparisons, 10 questions about one product ("is Leonardo AI free?"), 5 how-tos. Written before any answer was seen, and kept even when the answer was dull.
• Answers: ChatGPT's consumer interface with web search forced on, US English, collected through DataForSEO's LLM Scraper between 18:32 and 18:36 UTC on 9 October 2026. The scraper reported the model as gpt-6 on every answer. For each answer we kept the cited sources, the wider list of pages ChatGPT retrieved but did not cite, and its fan-out queries — the searches ChatGPT wrote for itself. 40 questions produced 72 distinct fan-out queries.
• Rankings: Bing and Google organic top 20, US English, desktop, for every fan-out query and for every question as typed, collected through DataForSEO between 18:36 and 19:00 UTC the same evening.
• Match: a cited page counts as "in the top 20" if the same URL appears there, ignoring protocol, www, query strings and a trailing slash. We also report the looser domain-level match.

One honest complication. Bing's API intermittently answers a different question from the one it was asked — "best free AI image generators 2026…" came back as Best Buy and dictionary definitions of best. We checked every Bing response for whether its results shared any meaningful word with the query and re-requested the ones that did not. 95 Bing responses failed that check; 20 of the 72 fan-out queries never produced a sane result in four attempts. Google had a different failure: 24 of its responses came back as server errors or partial results, and our first pass scored them as empty pages. We re-requested all 24; 21 succeeded and 3 still failed after six attempts. Every prompt with a query that never produced a valid result on either engine is excluded rather than scored as a miss. That leaves 23 prompts and 151 cited pages for the main comparison. Excluding them only removes noise; scoring them as misses would have pushed both figures down.

What we found

Bing beats Google — narrowly, until you change the question

The direction is consistent, but the size depends on which question you ask the engines. For the searches ChatGPT actually ran, Bing's lead is one page: 17 against 16. For the question as a person typed it, Bing's top 20 contained six times as many of ChatGPT's cited pages as Google's did. We cannot see why from outside; one reading is that ChatGPT rewrites a question before it searches, and Bing's results for the original wording happen to sit closer to where it ends up. If you are choosing between watching your Google rankings and your Bing rankings as a proxy for ChatGPT, Bing is the better proxy. That part of the brief holds.

But the 87% does not reproduce

The most generous reading we could construct — counting a page as matched if any page from the same website appeared in Bing's top 20 for either the question or any of ChatGPT's own searches — reaches 40.5%. The strictest, and the one most people mean by "match Bing's top 20", reaches 8.7% to 11.3%. 81.5% of cited pages were in neither engine's top 20 for the searches ChatGPT ran.

We do not know exactly how the 87% analysis matched results, and we are not claiming it is wrong for its own sample. Different questions, a different month and a different method can produce a different number. What we can say is that for questions about AI software, on 9 October 2026, a cited page usually was not a Bing top-20 result, and treating "rank in Bing's top 20" as the requirement for a ChatGPT citation would have predicted fewer than one citation in eight.

What ChatGPT cited instead

Look at the websites behind the cited pages that ranked in neither engine and a pattern appears: elevenlabs.io (7 pages), canva.com (5), and chatgpt.com, adobe.com, helpx.adobe.com and beautiful.ai (3 each). These are vendors, and the pages are deep ones — help articles, pricing pages, docs on voice cloning. ChatGPT's fan-out queries say why: 33 of its 72 searches contained the word official, usually next to a vendor's name, and it lands on the page that answers the narrow question, whether or not that page ranks for the broad one.

The split by question type shows it clearly:

For single-product questions, almost two-thirds of citations came from a website Bing ranked — usually the vendor itself — but mostly not the page Bing ranked. Bing found the right website; ChatGPT then cited a different page on it. For "best" lists, the open questions where independent sites can still win, neither engine's top 20 explained even one cited page in ten. How-to questions were the one slice where Google clearly did better.

The pages ChatGPT retrieved but did not cite matched even less: 4.3% were in Bing's top 20 for its own searches and 7.5% in Google's — on this measure Google was ahead. Whatever ChatGPT searches, it is not simply reading off either engine's first two pages.

One of our pages, through the same lens

We have one page ChatGPT has cited first. On 29 September 2026, asked "which math solver is best for algebra?", ChatGPT cited our Photomath vs Symbolab comparison as its first source, with six inline references.

Here is what Bing knows about that page, checked on 9 October through Bing Webmaster Tools: indexed, first discovered 23 June, last crawled 5 October — and no search query reported for it at all. Bing has the page and does not show it to anyone. On 9 October, asked "photomath vs symbolab which is better?", ChatGPT retrieved the page again, cited other sources instead, and our page was in neither Bing's nor Google's top 20 for that question.

A page with no Bing visibility to speak of was cited first by ChatGPT, and later passed over. Neither event followed its Bing ranking, because it did not have one.

What our server logs do show is OpenAI reading pages directly. In the same 24 hours, ChatGPT-User — the agent that fetches a page when a ChatGPT user's question needs it — requested 158 of our pages, and OAI-SearchBot, OpenAI's search crawler, requested 38. ChatGPT is not only looking at a search engine's results page. It is looking at yours.

What "gatekeeper" can still mean

None of this says Bing does not matter. It says the brief's mechanism is probably at a different layer from the one the 87% figure implies.

• Index, not ranking. A page Bing has never indexed cannot surface through anything that draws on Bing. Getting indexed is cheap, fast and the part you control. Ranking in Bing's top 20 is expensive, slow and, on our numbers, a weak predictor of being cited.
• The right website, then the right page. Bing's domain-level match on single-product questions (64%) was far higher than its page-level match (15%). Being the website Bing trusts for a topic seems to matter more than being the URL it ranks.
• Answer the narrow question somewhere. ChatGPT cited pricing pages, help articles and docs that rank for nothing broad. A page that states one checkable fact clearly is a citation candidate on its own.

The brief's practical advice survives this test: "Submitting a sitemap to Bing and fixing crawl issues can have a direct impact on AI visibility — faster than most SEO initiatives." That is the index layer, and it is worth an afternoon. Chasing Bing positions in the hope they become ChatGPT citations is the part our data does not support.

What we could not test

• ChatGPT's actual search back-end. We compared ChatGPT's searches with the public Bing and Google results for the same words, at the same time. ChatGPT may query a search partner with different settings, freshness or regional parameters. A different back-end could produce a different match rate; we can only measure what the public engines return.
• Other categories. 40 questions about AI software is one vertical. Local, medical or news queries may behave differently.
• Corrupted responses. Excluding 17 prompts with broken Bing or Google results is the right call for accuracy, but it shrinks the sample. Bing's figure would have to rise seven- to tenfold to reach 87%.
• One day. AI answers change daily. Our own page went from first citation to retrieved-only in ten days.

Run this test on your own questions

1. Write 20–40 questions your customers actually ask, before you look at any answers. Mix "best" lists, comparisons and questions about your product.
2. Record ChatGPT's cited sources and its fan-out queries, not just the answer text. The fan-out is what it really searched.
3. Pull Bing's and Google's top 20 for those searches on the same day, and validate every Bing response before you trust it.
4. Match at page level and at domain level. The gap between the two tells you whether you are losing the website or the page.
5. Check your index status on Bing Webmaster Tools for every page you want cited. This is the one step that is cheap and nearly always worth doing.
6. Track the outcome itself, not a proxy. If ChatGPT is the surface you care about, measure ChatGPT.

Where Semrush fits

Step six is the expensive one to do by hand: we spent an evening of API calls on 40 questions for a single day. Position Tracking in Semrush's SEO Toolkit is built to repeat it daily. Its Knowledge Base page, checked on 9 October 2026, says "You can track performance in Google, Bing, Baidu, ChatGPT Search, and Google AI Mode" — so the same keyword list can be tracked on Bing, where the index question lives, and in ChatGPT Search, where the citation lives, in one project.

That is the comparison our table makes, kept running instead of sampled once. One condition is worth knowing before you plan around it: the same page notes that "Prompts to track in ChatGPT Search and Google AI Mode are available only with an AI Visibility Toolkit subscription or a Semrush One subscription." Semrush One bundles the SEO Toolkit with the AI Visibility Toolkit, so it covers both halves.

Step one is where the Keyword Magic Tool earns a place. ChatGPT does not search the way people ask. Its 72 searches had a median length of ten words and only five were phrased as questions; 49 contained "2026", 33 "official" and 31 "pricing", usually stacked with product names. But the questions it starts from are the ones people type, and those are the panel you need. Semrush's Knowledge Base lists a "Questions-only filter to find keywords that ask questions" among the tool's features (checked 9 October 2026), which is a faster way to assemble a panel of the questions your buyers actually ask than writing 40 from memory, as we did.

Step two — recording which sources get cited — is what Visibility Overview in the AI Visibility Toolkit reports. Its Cited Sources tab lists, in Semrush's words, "The websites that AI systems reference when generating answers" (checked 9 October 2026). That is the list we compiled by hand above, where elevenlabs.io and canva.com sat at the top.

The index layer is where Site Audit belongs: robots rules, noindex tags, broken sitemaps and orphan pages stop every crawler, Bingbot and OAI-SearchBot included. Semrush's Site Audit page also states, checked 9 October 2026, that it "checks for AI crawler accessibility for the following bots: ChatGPT-User, OAI-SearchBot, Googlebot, Google-Extended, Perplexity-User, PerplexityBot, Claude-User, and Claude-SearchBot" — the first two being the OpenAI agents that made 196 page requests to our site in a single day. A crawler cannot tell you what Bing decided, but it can tell you whether you gave Bing, and OpenAI, the chance.

Semrush One's page offers "Try free for 7 days", with "Unlimited access to all Semrush One tools", and marks the offer "available to new users only" — checked on 9 October 2026. Read that as Semrush's terms, not our test; every measurement above was made without a Semrush account.

What we actually learned

The brief is right about the direction. Bing explained more of ChatGPT's citations than Google did under every overall reading we tried — narrowly for the searches ChatGPT ran, widely for the question as typed, with how-to questions the one slice where Google did better — and a site that ignores Bing is ignoring the better of the two proxies.

It is wrong, on our sample, about the size. For the searches ChatGPT itself ran, 11.3% of the pages it cited were in Bing's top 20, 10.6% in Google's, and 81.5% in neither. ChatGPT cited vendor help pages, pricing pages and docs that rank for nothing broad — and it cited our own page first while Bing showed that page to nobody.

So treat Bing as the index you must be in, not the ranking you must win. Get indexed, keep your crawl paths clean, and then measure ChatGPT directly instead of assuming Bing's top 20 will tell you what it cites.

Measure Bing and ChatGPT side by side with Semrush One — start your 7-day free trial — and keep Bing Webmaster Tools open beside it for the index status no outside tool can see.

Sources and method

Written by Sohail Akhtar. Every figure here is a measurement on a stated date; the raw responses, the scripts that matched them and the rejected Bing responses are saved and can be re-run.