Guide

AI Engines: How Each One Works

We collected which bot, which control and which document each engine has on one page. Where nothing is documented, we write "not documented".

AI Visibility Turkey is the Turkish-market home of Zen Media, Optimum7 and ecommerce.com.

On this page
  1. Summary table
  2. Google AI Overviews and AI Mode
  3. ChatGPT (OpenAI)
  4. Perplexity
  5. Claude (Anthropic)
  6. Microsoft Copilot and Bing
  7. Grok (xAI)
  8. DeepSeek
  9. Local note: Turkey
  10. The bot decision

Where a brand can appear depends first on access for that engine’s bot, then on content quality. Everything below rests on documents the engines publish themselves; where we could not find one, we say so.

Bot names and settings change over time. Re-check the linked document before you decide.

Summary table

What you must allow to appear, the training control, and whether source selection is documented.

Scroll the table sideways.

Engine comparison: crawlers, controls and source-selection documentation
EngineCrawler / toolTo appearTraining controlSource selection documented?
Google AI Overviews and AI ModeGooglebot; Google-Extended tokenPage indexed and snippet-eligibleGoogle-Extended (Gemini training and grounding); does not affect SearchEligibility is documented; ranking criteria not found in the documents we read
ChatGPT (OpenAI)OAI-SearchBot, GPTBot, ChatGPT-UserAllow OAI-SearchBotGPTBot (separate setting)No, only the crawlers are documented
PerplexityPerplexityBot, Perplexity-UserAllow PerplexityBotStates it is not used to crawl for foundation-model trainingNo, criteria not documented
Claude (Anthropic)ClaudeBot, Claude-SearchBot, Claude-UserAllow Claude-SearchBot and Claude-UserClaudeBotCrawlers documented; ranking criteria not found in the documents we read
Microsoft Copilot and BingBingbot; Bing Webmaster AI PerformanceBing index and Microsoft’s content guidanceNo separate document found in this researchPartly: content guidance exists, ranking criteria do not
Grok (xAI)Real-time web search toolBe findable and browsable on the webNo document found in this researchPartly: tool and citation behavior documented, not criteria
DeepSeekNot documentedNot documentedNot documentedNo, no primary documentation found

"Not documented" means we found no primary document in our research; we do not claim that none exists.

Google AI Overviews and AI Mode

Google

Google AI Overviews and AI Mode

Documented by the engine

Both features show supporting links. AI Mode runs many related searches through query fan-out, so it can reach a wider set of links than classic search. AI Overviews appear only when Google’s systems judge them useful.

  • Googlebot: indexing; the precondition for appearing in AI features
  • Google-Extended: robots.txt token for Gemini training and grounding

nosnippet, data-nosnippet, max-snippet and noindex limit display. Google-Extended does not affect inclusion in Search.

Source: Google Search Central: AI features ↗

What you need to know

  • Google says there are no additional technical requirements and that no special "AI file" or schema is needed; indexing and serving are never guaranteed.
  • Traffic from AI features is counted inside the Search Console Performance report under the "Web" search type; there is no AI-only report.
  • Google’s own advice: unique content, good page experience, structured data that matches visible text, current Merchant Center and Business Profile data.
  • In Turkey, AI Overviews and AI Mode began rolling out in Turkish gradually from 18 February 2026 (press report). See our Turkey article.
  • Our inference: on Google, GEO is largely strong SEO plus entity and data hygiene; we claim no secret Google-specific "AI trick".

Source: Google: Succeeding in AI search ↗; Google: common crawlers ↗; Webrazzi ↗

ChatGPT (OpenAI)

OpenAI

ChatGPT

Documented by the engine

To appear, allow OAI-SearchBot and OpenAI’s published IP ranges. Sites that opt out do not appear in ChatGPT search answers; they can still show as navigational links.

  • OAI-SearchBot: surfaces sites in ChatGPT search
  • GPTBot: may be used for training; independent of the search setting
  • ChatGPT-User: user-initiated fetches; robots.txt may not apply

Changes take about 24 hours to take effect.

Source: OpenAI: bots ↗

What you need to know

  • Search and training are separate settings: you can block GPTBot while allowing OAI-SearchBot.
  • ChatGPT-User is user-triggered; robots.txt may not apply to it.
  • How OpenAI selects and ranks sources is not documented.
  • The press reports ChatGPT as the most used AI assistant in Turkey; we give no exact share.

Source: Türkiye Today (press report) ↗

Perplexity

Perplexity

Perplexity

Documented by the engine

Perplexity states that PerplexityBot is not used to crawl for foundation-model training. The document does not explain how sources are selected or ranked.

  • PerplexityBot: surfaces and links sites in Perplexity search
  • Perplexity-User: fetches on user request; generally ignores robots.txt

Source: Perplexity: bots ↗

What you need to know

  • Selection criteria are unpublished; any claim in this area is a guess.
  • That Perplexity-User generally ignores robots.txt affects your blocking decisions.
  • In Ahrefs' June 2025 study (vendor research), YouTube at 16.1% and Wikipedia at 12.5% were in Perplexity’s top 10; single month, top 10 only.

Source: Ahrefs (vendor research, June 2025) ↗

Claude (Anthropic)

Anthropic

Claude

Documented by the engine

Anthropic documents that blocking the last two bots may reduce visibility in Claude search answers.

  • ClaudeBot: training data
  • Claude-User: fetches when a user question needs it
  • Claude-SearchBot: indexes for search relevance

Anthropic advises against IP blocking; crawler IPs are published at claude.com/crawling/bots.json.

Source: Anthropic: web crawling ↗

What you need to know

  • Training (ClaudeBot) and search visibility (Claude-SearchBot, Claude-User) are separate decisions.
  • We did not find source-ranking criteria in the documents we read.

Microsoft Copilot and Bing

Microsoft

Microsoft Copilot and Bing

Documented by the engine

'AI Performance' in Bing Webmaster Tools (public preview) reports citations in Microsoft Copilot, Bing AI summaries and partner integrations: total citations, average cited pages, grounding queries (sample), page-level citation activity and visibility trends.

  • Bingbot: Bing indexing

Only citation counts are reported, not placement within the answer.

Source: Microsoft Bing: AI Performance (October 2025) ↗

What you need to know

  • Microsoft Advertising (8 October 2025) recommends aligned title, H1 and meta, a question followed by a short direct answer, lists and tables, and JSON-LD; it asks that key answers not be hidden in tabs or accordions and that information not live only in PDFs or images.
  • The claim that Bing’s index feeds Copilot or ChatGPT appears only in third-party blogs; it is not confirmed in primary documentation, so we do not state it as fact.
  • AI Performance is one of the most concrete ways to see your own citations in Copilot and Bing directly.

Source: Bing Webmaster Blog ↗; Microsoft Advertising ↗

Grok (xAI)

xAI

Grok

Documented by the engine

Per xAI’s documentation, Grok’s web search tool searches the web in real time, browses pages and returns all citations; inline citations are added by the model itself.

Source: xAI: web search tool ↗

What you need to know

  • We found no separately named crawler in the documentation, so we do not invent bot names.
  • Because it relies on real-time search, current, crawlable and clear pages should stand out (our inference, not from the documentation).
  • We found no primary documentation on source selection.

Source: xAI: citations ↗

DeepSeek

Not documented

We found no primary document on how DeepSeek selects sources. So we make no concrete claim for this engine; we only say that general web-grounded principles apply.

Local note: classic search in Turkey

In classic search in Turkey, Google leads and Yandex is a strong second. StatCounter’s October 2026 data shows Google at 76.85% and Yandex at 21.63%; StatCounter is volatile for Turkey and sources conflict, so we use the direction only. We make no AI-specific claim about Yandex. Details: Turkey data.

Source: StatCounter, October 2026 ↗

The bot decision: appear in answers, or allow training?

Per the engines' documentation, search bots and training bots are separate settings. This is a business decision, not a technical one.

Scroll the table sideways.

robots.txt options, their effects and who they fit (the effect column rests on engine documents; the 'who it fits' column is our opinion)
OptionWhat happens (per the documents)Who it fits (our opinion)
Allow search bots, block training bots (e.g. allow OAI-SearchBot, block GPTBot)The site can appear in search answers; training use is switched off separately.Brands that want to appear in answers but do not want content used for training
Allow everythingBoth search and training use are possible.Brands that put wide recognition and content reach first
Block search botsThe site does not appear in that engine’s search answers (for ChatGPT it may still show as a navigational link).Publishers who deliberately give up visibility to fully protect their content
Configure nothingDefault behavior applies; a CDN or WAF may be silently blocking bots.We recommend it to no brand: audit the current state first

For robots.txt details see the AI crawlers and robots.txt guide.

Can the bots reach your site?

Send us your website and we will review robots.txt, CDN rules and indexing together. We reply by email.