# You Didn't Know GEO: AI Visibility Principles, Practices, and Trade-offs
**作者**: Tw93
**日期**: 2026-05-03T13:33:18.000Z
**来源**: [https://x.com/HiTw93/status/2050931710066565374](https://x.com/HiTw93/status/2050931710066565374)
---
## One Hour to Make AI Find Your Content
A few friends pinged me recently saying my open source tools were showing up when they asked AI questions. I hadn’t done anything deliberate, so I figured: why not spend an hour structuring things properly? After doing it, I fired off a quick tweet, but the notes were messy. People seemed genuinely interested, so I decided to write it up as a proper article for reference.
I hate gaming rankings or generating junk content. This article won’t teach you shortcuts. It’s about helping AI better understand the content you already have.
I looked into why it happened and found that AI search runs on entirely different logic. Traditional SEO is about fighting into the top 10, but 83% of AI Overview citations come from pages outside the top 10. AI rewards clear structure and reliable sourcing, not PageRank. My projects aren’t big, but the READMEs and docs are well-written enough that AI picks them up where bigger sites have thin content. That’s probably why friends were seeing Pake and MiaoYan in their AI results.
AI search is growing fast: 527% year-over-year in the first half of 2025, ChatGPT hit 900 million weekly active users by February 2026, and referral traffic converts at roughly 5x the rate of traditional search. But it still accounts for less than 1% of total referral traffic. This is a brand visibility strategy, not a traffic strategy. Worth an hour of setup, not a week, because your product is your real competitive advantage, not this.
## Use robots.txt to Sort Out Crawler Types
Most people treat robots.txt as a switch: either block AI crawlers or allow them all. But AI crawlers come in several types, and they do different things.
Training crawlers (GPTBot, ClaudeBot, Meta-ExternalAgent, CCBot) take your content to train models. Blocking them keeps your content out of training data but doesn’t affect current AI search results.
Search and retrieval crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) fetch content in real time to answer user queries. Block these and you vanish from AI search.
User-triggered fetchers (ChatGPT-User, Claude-User, Perplexity-User, Google-Agent) only fire when someone pastes your URL into a chat window. Block them and users asking “summarize this page” get nothing.
Opt-out tokens (Google-Extended, Applebot-Extended) aren’t real crawlers. They’re signals you declare in robots.txt to opt out of AI training.
Undeclared crawlers (Bytespider, xAI’s Grok bot) don’t identify themselves and don’t necessarily follow the rules.
My approach: allow search/retrieval and user-triggered crawlers, block training and undeclared ones.
```
# Search & retrieval: allow
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
# User-triggered: allow
User-agent: ChatGPT-User
Allow: /
User-agent: Claude-User
Allow: /
# Training: block
User-agent: GPTBot
Disallow: /
User-agent: CCBot
Disallow: /
# Opt-out tokens
User-agent: Google-Extended
Disallow: /
# Undeclared: block
User-agent: Bytespider
Disallow: /
```
## Write Your llms.txt and Cross-Link Your Sites
llms.txt is a new standard, similar to robots.txt but designed for AI consumption. You place a Markdown file at your site root describing what your site does, its key pages, and who’s behind it. AI systems prioritize this file when they crawl your content.
BuiltWith tracks over 840,000 websites that have deployed llms.txt, including Anthropic, Cloudflare, Stripe, and Vercel. But in SE Ranking’s survey of 300,000 domains, adoption is only 10%. It’s still early, and being early is an advantage.
The format is simple:
```
# Your Project Name
> One-line description of what this is.
## Links
- [Documentation](https://yoursite.com/docs)
- [GitHub](https://github.com/you/project)
- [Blog](https://yoursite.com/blog)
## About
Short paragraph explaining the project, its purpose,
key features, and what makes it different.
```
After creating yours, submit it to directory.llmstxt.cloud, llmstxt.site, and the llms-txt-hub repository on GitHub via PR.
I also did something interesting: I made each site’s llms.txt reference the others. I maintain tw93.fun, weekly.tw93.fun, and yobi.tw93.fun. Each site’s llms.txt links to the rest, forming a mesh. An AI crawler entering through any one site can follow the links and discover everything else.
These changes take effect after crawlers revisit your site, usually within a few days. After that, try searching for your project name in ChatGPT. The citation sources and description accuracy should improve.
## Provide Full Content and Markdown Routes
llms.txt is the summary. llms-full.txt is the complete version, typically 30-60KB, containing project descriptions, FAQs, usage scenarios, competitive comparisons, and README excerpts. Mintlify’s CDN analysis shows llms-full.txt gets 3-4x more traffic than llms.txt. AI systems that find the summary want the full version.
Markdown routes go further. Evil Martians recommend providing a .md version of every page on your site. A 15,000-token HTML page becomes a 3,000-token Markdown document, 80% less noise.
The simplest way to tell AI you have a Markdown version is adding this to your page’s <head>:
```
<link rel="alternate" type="text/markdown" href="/page.md" />
```
Claude Code and Cursor already send Accept: text/markdown headers when fetching docs. This is standard HTTP/1.1 content negotiation, around since 1997.
## Register Your Site with Search Platforms
The robots.txt and llms.txt work from the previous sections makes your content readable to AI, but AI has to find you first. ChatGPT’s search runs on Bing, Google AI Overview uses Google’s own index, and Perplexity also relies on search APIs. If your pages aren’t indexed by search engines, none of the structuring work above matters. So the first step is making sure Google and Bing have indexed your site.
The setup is straightforward: go to Google Search Console, verify your domain via DNS or HTML file upload, then submit your sitemap URL (usually yoursite.com/sitemap.xml). Check the “Pages” indexing report to see which pages are indexed and which have issues. If an important page isn’t indexed, use the URL Inspection tool to manually request indexing.
You might think Bing doesn’t matter, but Copilot, DuckDuckGo, and Yahoo all run on Bing’s index underneath. Register with Bing Webmaster Tools, submit your sitemap, and check the AI Performance panel to see how often AI cites your content. While you’re there, set up IndexNow so Bing gets notified immediately when you publish new content instead of waiting for a crawler to find it.
Setting up IndexNow means placing an API key file at your site root, then sending a POST request to api.indexnow.org/indexnow with the list of changed URLs whenever you publish. Bing picks them up within minutes. Many static site generators and CMS platforms have IndexNow plugins available.
Google Search Console doesn’t have an AI-specific panel yet, but submitting your sitemap and monitoring indexing status is still worth doing. Google’s AI Overviews pull from a wider range than traditional results, so even pages that don’t rank in the top 10 can appear in AI-generated answers.
Perplexity has more users than you’d expect. They run a publisher program at pplx.ai/publisher-program. Once approved, you get an 80/20 revenue share and access to citation analytics data.
## I Built a Dedicated Knowledge Site for AI
Instead of waiting for AI to scrape information from scattered sources, give it a single entry point with everything organized.
A knowledge site should provide three layers: an overview (llms.txt), a full version (llms-full.txt, 30-60KB), and standalone knowledge pages for each core project. Add structured JSON APIs so AI tools can fetch data programmatically. Pull data live from upstream sources like the GitHub API with caching that refreshes periodically, keeping maintenance cost near zero.
One thing that’s easy to miss: give AI a narrative structure, not just a list of projects. If you have multiple projects, write a description that connects them, how they relate, your technical direction, the overall picture. When AI answers “who is this person” or “what does this team do,” a coherent narrative works much better than a flat list.
My implementation is called Yobi (from the Japanese 呼び / よび, meaning “to call” or “to summon”). It serves an llms.txt overview, a 50KB llms-full.txt, per-project pages, and four JSON endpoints (/api/profile, /api/projects, /api/blog, /api/weekly) that pull live data from the GitHub API with ISR caching that refreshes hourly. Stack: Next.js + TypeScript on Vercel.
The JSON API returns structured project data with live GitHub stats:
## Give Each Project Its Own Page
Each project needs its own standalone page, not a row in a list, but a self-contained Markdown document with a citable summary, core features, competitive comparisons, use cases, and install commands. Ahrefs found that cited pages have titles with higher semantic similarity to user queries, and natural-language URL slugs (like /projects/pake) get cited more than opaque IDs (like /page?id=47).
URL structure matters. /projects/pake tells the model what the page is about before it reads a single line. /page?id=47 tells it nothing.
## Sync Structured Data to Your Main Domain
Subdomains carry less authority than root domains. AI crawlers that discover example.com won’t automatically find docs.example.com or api.example.com. If your llms.txt, project pages, and API data are spread across subdomains, AI may only see part of the picture.
The fix is to mirror key structured data onto your main domain so that example.com/llms.txt, example.com/projects/xxx.md, and example.com/api/projects.json all live under one roof. AI crawlers discover your main site through search indexing and find everything without leaving. Implementation options include scheduled CI sync, build-time fetching, or reverse proxying. I use a GitHub Action that syncs subdomain data to the blog repo nightly.
When launching a new site, use a checklist to avoid gaps. Core items: robots.txt (categorized crawler permissions), llms.txt (site summary with cross-references), sitemap (submitted to search engines), Bing Webmaster Tools (enable IndexNow), Google Search Console (monitor indexing). Each site’s llms.txt should reference the others, forming a discovery mesh.
The easiest trap when doing this work is getting carried away with every GEO technique you come across and trying to add them all, creating a mess and losing sight of what matters.
## These Didn’t Work
<meta name="ai-content-url"> and <meta name="llms">: no spec, no adoption by any major AI system.
/.well-known/ai.txt: multiple competing proposals, no real adoption. Wait for a winner.
HTML comments with AI hints: parsers strip comments before AI sees the content.
User-Agent sniffing to serve Markdown: returning different content to bots versus humans is cloaking. Google will penalize you.
Unofficial AI meta tags: unless a major AI provider explicitly documents support, it’s just noise.
## JSON-LD Isn’t as Useful as You’d Think
I initially thought JSON-LD would be powerful for AI visibility. Deeper research showed a more complicated picture. SearchVIU ran an experiment where they put data only in JSON-LD without showing it on the page. All five AI systems they tested failed to find it. Mark Williams-Cook’s follow-up experiment showed that LLMs treat <script type="application/ld+json"> as plain text, reading whatever words are inside without understanding the structured semantics.
The one confirmed exception is Bing/Copilot, which uses JSON-LD to enrich its search index. Keep existing JSON-LD (it helps Bing/Copilot and traditional rich results), but don’t add it expecting ChatGPT or Claude to cite you more.
## What the Research Data Says
The GEO paper from Princeton and IIT Delhi, published at KDD 2024, found that adding authoritative citations improves AI visibility by 115%, relevant statistics by 33%, and direct quotations from credible sources by 43%.
My friend @yaojingang has been doing serious research on GEO. His geo-citation-lab ran 602 prompts across three platforms and scraped tens of thousands of pages for feature analysis. His full report is worth reading. Here are the patterns most useful for content creators.
Specificity. Pages with real data, clear definitions, and side-by-side comparisons see over 50% higher impact than vague, general pages. Step-by-step structure also helps noticeably. Pure FAQ format actually hurts. Those GEO tools that tell you “add FAQ to boost your score” are giving advice the data contradicts, which also validates my decision to drop FAQ sections from my own pages.
Content depth. AI doesn’t favor short summaries. It favors long content it can slice into reusable segments. High-impact pages average nearly 2,000 words with 10+ headings. Low-impact pages average just 170 words, over a 10x gap. The sweet spot is 1,000 to 3,000 words.
Relevance. All mechanical SEO metrics (heading hierarchy, meta descriptions, keyword density) predict less than one single variable: whether your page content actually answers the question the user asked.
Platform differences. ChatGPT cites fewer sources but uses each deeply; its per-citation impact is over 5x that of Google. Perplexity casts a wider net, citing more than twice as many sources. To get cited by ChatGPT, go deep on individual pages. To get cited by Perplexity, go wide.
Content type. Official websites, news, and industry verticals account for roughly 80% of citation sources. But encyclopedia-style and explainer pages have 3x the impact of news pages. English content accounts for over 83% of global citation samples, so projects targeting an international audience need English versions.
## Retrieved Doesn’t Mean Cited
Of all the pages ChatGPT retrieves during a session, only 15% end up in the final answer. The other 85% are never cited. Getting into the retrieval pool is just the first hurdle. The model still has to decide which pages are worth citing.
Ahrefs found that cited pages have titles with noticeably higher semantic similarity to user queries, and pages with descriptive natural-language URL slugs get cited more than those with opaque IDs. This is why llms.txt and Markdown routes help: they give the model a clean, unambiguous signal about what your page covers.
Brands get cited 6.5x more often through third-party sources than through their own domains. Someone praising your project on Reddit or Hacker News carries more weight than your own marketing copy. That’s exactly why having a well-structured llms.txt matters: it gives the model a citable anchor to point to, even when the conversation that triggered the query happened somewhere else.
There are AI SEO audit tools that score your site and tell you to add FAQ sections, trust pages, or more text. Don’t let scores drive your decisions. The test is simple: does every paragraph you add contain information that isn’t already on the page? If not, don’t add it. I once added a FAQ to Yobi that just restated what the About section already said, purely to push the score up. That’s padding. I removed it.
Everything here is about helping AI understand what you have accurately, giving it a clean working environment. That lasts longer than any shortcut.
The basic configuration takes about an hour. The knowledge endpoint and per-project pages take longer, but once the data structure is in place, maintenance is easy. The daily sync runs on its own.
Give it a few days for crawlers to pick up the changes, then try searching for your name or project in ChatGPT, Perplexity, or Claude. The citations should be more accurate.
AI citation attribution is still unreliable. CJR and Tow Center tested 200 AI-generated citations and found 153 with partial or complete errors. Do the structural work because it makes your content easier to access accurately, but don’t treat an AI citation as proof that users saw your exact words. The mechanism is still improving
If you have your own products, blog, or website, give it a try. You can also hand this article to Claude Code and let it handle most of the setup.
## Further Reading
1. GEO: Generative Engine Optimization - Princeton & IIT Delhi, KDD 2024
2. Overseas GEO Research - geo-citation-lab
3. llms.txt standard specification
4. Why ChatGPT Cites One Page Over Another - Ahrefs
5. GEO Benchmark Study 2026 - ConvertMate
6. Optimizing Content for AI Discovery - Evil Martians
7. How LLMs Actually Use Schema Markup - SearchVIU
8. AI Search Has a Citation Problem - CJR / Tow Center
9. LLMs.txt: Why Brands Rely On It and Why It Doesn’t Work - SE Ranking
10. How Often Do LLMs Visit llms.txt? - Mintlify
11. IndexNow Protocol Documentation
## 相关链接
- [Tw93](https://x.com/HiTw93)
- [@HiTw93](https://x.com/HiTw93)
- [2.5K](https://x.com/HiTw93/status/2050931710066565374/analytics)
- [directory.llmstxt.cloud](https://directory.llmstxt.cloud/)
- [llmstxt.site](https://llmstxt.site/)
- [tw93.fun](https://tw93.fun/)
- [weekly.tw93.fun](https://weekly.tw93.fun/)
- [yobi.tw93.fun](https://yobi.tw93.fun/)
- [Google Search Console](https://search.google.com/search-console)
- [yoursite.com/sitemap.xml](https://yoursite.com/sitemap.xml)
- [api.indexnow.org/indexnow](https://api.indexnow.org/indexnow)
- [pplx.ai/publisher-program](https://pplx.ai/publisher-program)
- [example.com](https://example.com/)
- [docs.example.com](https://docs.example.com/)
- [api.example.com](https://api.example.com/)
- [example.com/llms.txt](https://example.com/llms.txt)
- [example.com/projects/xxx.md](https://example.com/projects/xxx.md)
- [example.com/api/projects.json](https://example.com/api/projects.json)
- [@yaojingang](https://github.com/yaojingang)
- [geo-citation-lab](https://github.com/yaojingang/geo-citation-lab)
- [full report](https://github.com/yaojingang/geo-citation-lab/blob/main/04-repet/final_report.md)
- [GEO: Generative Engine Optimization - Princeton & IIT Delhi, KDD 2024](https://arxiv.org/abs/2311.09735)
- [Overseas GEO Research - geo-citation-lab](https://github.com/yaojingang/geo-citation-lab)
- [llms.txt standard specification](https://llmstxt.org/)
- [Why ChatGPT Cites One Page Over Another - Ahrefs](https://ahrefs.com/blog/why-chatgpt-cites-pages/)
- [GEO Benchmark Study 2026 - ConvertMate](https://www.convertmate.io/research/geo-benchmark-2026)
- [Optimizing Content for AI Discovery - Evil Martians](https://evilmartians.com/chronicles/optimizing-content-for-ai-discovery)
- [How LLMs Actually Use Schema Markup - SearchVIU](https://searchviu.com/en/how-llms-actually-use-schema-markup/)
- [AI Search Has a Citation Problem - CJR / Tow Center](https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php)
- [LLMs.txt: Why Brands Rely On It and Why It Doesn’t Work - SE Ranking](https://seranking.com/blog/llms-txt/)
- [How Often Do LLMs Visit llms.txt? - Mintlify](https://www.mintlify.com/blog/how-often-do-llms-visit-llms-txt)
- [IndexNow Protocol Documentation](https://www.indexnow.org/documentation)
- [Upgrade to Premium](https://x.com/i/premium_sign_up)
- [9:33 PM · May 3, 2026](https://x.com/HiTw93/status/2050931710066565374)
- [2,598 Views](https://x.com/HiTw93/status/2050931710066565374/analytics)
---
*导出时间: 2026/5/3 22:19:54*
---
## 中文翻译
# 你所不知道的 GEO：AI 可见性的原则、实践与权衡
**作者**: Tw93
**日期**: 2026-05-03T13:33:18.000Z
**来源**: [https://x.com/HiTw93/status/2050931710066565374](https://x.com/HiTw93/status/2050931710066565374)
---
## 一小时让 AI 找到你的内容
最近有几位朋友私信我说，他们向 AI 提问时能看到我的开源工具被引用。我并没有刻意做过什么，于是想：何不花一个小时把事情规范化地梳理一下？做完之后我发了条推文，但笔记很零乱。大家似乎真的很感兴趣，所以我决定把它整理成一篇正式的文章供参考。
我讨厌刷排名或生成垃圾内容。这篇文章不会教你投机取巧，而是讲如何帮助 AI 更好地理解你已有的内容。
我研究了原因，发现 AI 搜索的运行逻辑完全不同。传统 SEO 是挤进前 10 名，但 83% 的 AI Overview 引用来自前 10 名之外的页面。AI 奖赏的是清晰的结构和可靠的来源，而不是 PageRank。我的项目规模不大，但 README 和文档写得足够好，于是在大型网站内容稀薄的地方，AI 选择了它们。这可能就是朋友们在 AI 结果中看到 Pake 和 MiaoYan 的原因。
AI 搜索增长迅速：2025 年上半年同比增长 527%，到 2026 年 2 月 ChatGPT 的周活跃用户达到 9 亿，而引荐流量的转化率约为传统搜索的 5 倍。但它占总引荐流量的比例仍不到 1%。这是一种品牌可见性策略，而非流量策略。值得花一个小时来配置，但不值得花一周，因为你的产品才是真正的竞争优势，而不是这些。
## 用 robots.txt 理清爬虫类型
大多数人把 robots.txt 当作一个开关：要么屏蔽所有 AI 爬虫，要么全部放行。但 AI 爬虫其实分为几种类型，它们的用途各不相同。
训练型爬虫（GPTBot、ClaudeBot、Meta-ExternalAgent、CCBot）会抓取你的内容用于训练模型。屏蔽它们可以让你的内容不进入训练数据，但不影响当前的 AI 搜索结果。
搜索与检索型爬虫（OAI-SearchBot、Claude-SearchBot、PerplexityBot）会实时抓取内容来回答用户的问题。屏蔽它们，你就会从 AI 搜索中消失。
用户触发的抓取器（ChatGPT-User、Claude-User、Perplexity-User、Google-Agent）只在有人把你的 URL 粘贴到聊天窗口时才会触发。屏蔽它们，用户让 AI“总结这个页面”时就得不到任何内容。
退出训练标记（Google-Extended、Applebot-Extended）并不是真正的爬虫，而是你在 robots.txt 中声明的信号，用于退出 AI 训练。
未声明的爬虫（Bytespider、xAI 的 Grok 爬虫）不会表明身份，也不一定遵守规则。
我的做法是：允许搜索/检索型和用户触发型爬虫，屏蔽训练型和未声明型爬虫。
```
# Search & retrieval: allow
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
# User-triggered: allow
User-agent: ChatGPT-User
Allow: /
User-agent: Claude-User
Allow: /
# Training: block
User-agent: GPTBot
Disallow: /
User-agent: CCBot
Disallow: /
# Opt-out tokens
User-agent: Google-Extended
Disallow: /
# Undeclared: block
User-agent: Bytespider
Disallow: /
```
## 编写你的 llms.txt 并在站点间互相链接
llms.txt 是一个新标准，类似于 robots.txt，但专为 AI 设计。你可以在网站根目录放置一个 Markdown 文件，描述你的网站是做什么的、有哪些关键页面、背后是谁在运营。AI 系统在抓取你的内容时会优先读取这个文件。
BuiltWith 追踪到超过 84 万个网站已部署 llms.txt，其中包括 Anthropic、Cloudflare、Stripe 和 Vercel。但在 SE Ranking 对 30 万个域名的调查中，采用率只有 10%。现在还处于早期阶段，而早入场本身就是优势。
格式很简单：
```
# Your Project Name
> One-line description of what this is.
## Links
- [Documentation](https://yoursite.com/docs)
- [GitHub](https://github.com/you/project)
- [Blog](https://yoursite.com/blog)
## About
Short paragraph explaining the project, its purpose,
key features, and what makes it different.
```
创建好之后，通过 PR 将它提交到 directory.llmstxt.cloud、llmstxt.site 以及 GitHub 上的 llms-txt-hub 仓库。
我还做了一件有趣的事：让每个网站的 llms.txt 相互引用。我维护着 tw93.fun、weekly.tw93.fun 和 yobi.tw93.fun，每个网站的 llms.txt 都链接到其余网站，形成一个网状结构。AI 爬虫从任何一个网站进入，都能顺着链接发现其他所有内容。
这些更改会在爬虫重新访问你的网站后生效，通常在几天之内。之后，可以试着在 ChatGPT 中搜索你的项目名称，引用来源和描述准确性应该会有所改善。
## 提供完整内容和 Markdown 路由
llms.txt 是摘要，llms-full.txt 是完整版本，通常为 30-60KB，包含项目描述、常见问题、使用场景、竞品对比和 README 摘录。Mintlify 的 CDN 分析显示，llms-full.txt 的流量是 llms.txt 的 3-4 倍。找到摘要的 AI 系统会想要完整版本。
Markdown 路由更进一步。Evil Martians 建议为网站上的每个页面提供 .md 版本。一个 15,000 token 的 HTML 页面会变成 3,000 token 的 Markdown 文档，噪音减少 80%。
告诉 AI 你有 Markdown 版本的最简单方法，是在页面的 <head> 中添加：
```
<link rel="alternate" type="text/markdown" href="/page.md" />
```
Claude Code 和 Cursor 在抓取文档时已经会发送 Accept: text/markdown 请求头。这是标准的 HTTP/1.1 内容协商机制，从 1997 年就有了。
## 将你的网站注册到搜索平台
前面章节中的 robots.txt 和 llms.txt 配置能让 AI 读懂你的内容，但 AI 首先得找到你。ChatGPT 的搜索基于 Bing，Google AI Overview 使用谷歌自己的索引，Perplexity 也依赖搜索 API。如果你的页面没有被搜索引擎收录，上面所有的结构化工作都毫无意义。所以第一步是确保谷歌和 Bing 已经收录了你的网站。
设置很简单：进入 Google Search Console，通过 DNS 或 HTML 文件上传验证你的域名，然后提交你的站点地图 URL（通常是 yoursite.com/sitemap.xml）。查看“页面”索引报告，了解哪些页面已被收录、哪些有问题。如果某个重要页面没有被收录，可以使用网址检查工具手动请求收录。
你可能觉得 Bing 不重要，但 Copilot、DuckDuckGo 和 Yahoo 底层都运行在 Bing 的索引之上。在 Bing Webmaster Tools 注册，提交你的站点地图，并查看 AI Performance 面板，了解 AI 引用你内容的频率。顺便设置 IndexNow，这样你发布新内容时 Bing 会立即收到通知，而不必等爬虫发现它。
设置 IndexNow 的方法是在网站根目录放置一个 API 密钥文件，然后在每次发布内容时，向 api.indexnow.org/indexnow 发送包含已更改 URL 列表的 POST 请求。Bing 会在几分钟内获取这些内容。许多静态网站生成器和 CMS 平台都有可用的 IndexNow 插件。
Google Search Console 目前还没有针对 AI 的专用面板，但提交站点地图并监控收录状态仍然值得去做。谷歌的 AI Overviews 检索的范围比传统搜索结果更广，所以即使是未排进前 10 名的页面也可能出现在 AI 生成的回答中。
Perplexity 的用户比你想象的要多。他们在 pplx.ai/publisher-program 运营一个发布者计划。通过审核后，你可以获得 80/20 的收入分成，以及引用分析数据的访问权限。
## 我为 AI 搭建了一个专门的知识站点
与其等 AI 从分散的来源中抓取信息，不如给它一个把所有内容都组织好的单一入口。
知识站点应该提供三个层次：一个概览（llms.txt）、一个完整版本（llms-full.txt，30-60KB），以及每个核心项目独立的知识页面。再加上结构化的 JSON API，让 AI 工具可以以编程方式获取数据。通过带定期刷新缓存的机制，从 GitHub API 等上游来源实时拉取数据，把维护成本降到几乎为零。
有一点很容易被忽略：要给 AI 一个叙述性的结构，而不只是项目列表。如果你有多个项目，写一段描述把它们串联起来——它们之间如何关联、你的技术方向、整体蓝图。当 AI 回答“这个人是谁”或“这个团队是做什么的”时，连贯的叙述比扁平的列表效果好得多。
我的实现叫 Yobi（来自日语 呼び / よび，意为“呼唤”或“召唤”）。它提供 llms.txt 概览、50KB 的 llms-full.txt、各项目的独立页面，以及四个 JSON 端点（/api/profile、/api/projects、/api/blog、/api/weekly），这些端点从 GitHub API 实时拉取数据，并配有每小时刷新的 ISR 缓存。技术栈：Vercel 上的 Next.js + TypeScript。
JSON API 返回带有实时 GitHub 统计数据的结构化项目数据：
## 给每个项目一个独立页面
每个项目都需要自己的独立页面——不是列表中的一行，而是一份自包含的 Markdown 文档，包含可引用的摘要、核心功能、竞品对比、使用场景和安装命令。Ahrefs 发现，被引用页面的标题与用户查询的语义相似度更高，而且自然语言的 URL 路径（比如 /projects/pake）比晦涩的 ID（比如 /page?id=47）更容易被引用。
URL 结构很重要。/projects/pake 能在模型读一个字之前就告诉它页面是关于什么的，而 /page?id=47 什么也说明不了。
## 把结构化数据同步到主域名
子域名的权重低于根域名。发现 example.com 的 AI 爬虫不会自动找到 docs.example.com 或 api.example.com。如果你的 llms.txt、项目页面和 API 数据分散在各个子域名上，AI 可能只能看到全貌的一部分。
解决办法是把关键的结构化数据镜像到主域名上，让 example.com/llms.txt、example.com/projects/xxx.md 和 example.com/api/projects.json 都集中在同一个域名下。AI 爬虫通过搜索索引发现你的主站后，无需离开就能找到所有内容。实现方式包括定时 CI 同步、构建时抓取或反向代理。我使用一个 GitHub Action，每晚将子域名的数据同步到博客仓库。
上线新站点时，用一个清单来避免遗漏。核心项：robots.txt（分类的爬虫权限）、llms.txt（带交叉引用的站点摘要）、sitemap（提交给搜索引擎）、Bing Webmaster Tools（启用 IndexNow）、Google Search Console（监控收录）。每个站点的 llms.txt 应该相互引用，形成一个发现网络。
做这些事最容易踩的坑，就是沉迷于遇到的每一个 GEO 技巧并试图全部加上，结果搞得一团糟，反而迷失了重点。
## 这些方法没有用
<meta name="ai-content-url"> 和 <meta name="llms">：没有规范，也没有任何主流 AI 系统采用。
/.well-known/ai.txt：多个相互竞争的提案，没有真正的采用。等一个胜出者出现再说。
带 AI 提示的 HTML 注释：解析器会在 AI 看到内容之前把注释剥掉。
通过 User-Agent 嗅探返回 Markdown：给机器人和人类返回不同的内容属于伪装（cloaking），谷歌会惩罚你。
非官方的 AI meta 标签：除非主流 AI 提供商明确文档化了支持，否则只是噪音。
## JSON-LD 没你想的那么有用
我最初以为 JSON-LD 对 AI 可见性会很有用。深入研究后发现情况更为复杂。SearchVIU 做了一个实验，他们将数据只放在 JSON-LD 中而不在页面上显示。他们测试的五个 AI 系统全部没能找到它。Mark Williams-Cook 的后续实验表明，LLM 把 `<script type="application/ld+json">` 当作纯文本，只是读取里面的文字，而无法理解其结构化语义。
唯一被证实的例外是 Bing/Copilot，它会使用 JSON-LD 来丰富其搜索索引。保留现有的 JSON-LD（它对 Bing/Copilot 和传统富媒体搜索结果有帮助），但不要指望添加它能让 ChatGPT 或 Claude 更多地引用你。
## 研究数据怎么说
普林斯顿大学和印度理工学院德里分校发表于 KDD 2024 的 GEO 论文发现，添加权威引用可提升 115% 的 AI 可见性，添加相关统计数据可提升 33%，添加来自可信来源的直接引语可提升 43%。
我的朋友 @yaojingang 一直在对 GEO 进行认真的研究。他的 geo-citation-lab 在三个平台上运行了 602 个提示词，并抓取了数万个页面进行特征分析。他的完整报告值得一读。以下是对内容创作者最有用的几个模式。
具体性。含有真实数据、清晰定义和并排对比的页面，其影响力比含糊笼统的页面高出 50% 以上。分步骤的结构也有明显帮助。纯 FAQ 格式实际上有害。那些告诉你“添加 FAQ 来提升分数”的 GEO 工具给出的建议与数据相矛盾，这也印证了我从自己的页面中移除 FAQ 部分的决定。
内容深度。AI 并不偏爱简短的摘要。它偏爱可以被切分成可复用片段的长内容。高影响力页面平均接近 2000 字，有 10 个以上的标题。低影响力页面平均只有 170 字，差距超过 10 倍。最佳区间是 1000 到 3000 字。
相关性。所有机械式的 SEO 指标（标题层级、meta 描述、关键词密度）对结果的预测力加起来还不如一个单一的变量：你的页面内容是否真正回答了用户提出的问题。
平台差异。ChatGPT 引用的来源较少但对每个来源的使用较深入；其单次引用的影响力是 Google 的 5 倍以上。Perplexity 撒网更广，引用的来源数量是两倍以上。想让 ChatGPT 引用你，就在单个页面上做深。想让 Perplexity 引用你，就做广。
内容类型。官方网站、新闻和行业垂直网站大约占引用来源的 80%。但百科式和讲解式页面的影响力是新闻页面的 3 倍。英文内容占全球引用样本的 83% 以上，因此面向国际受众的项目需要英文版本。
## 被检索到不等于被引用
ChatGPT 在一次会话中检索到的所有页面里，只有 15% 最终出现在答案中。其余 85% 从未被引用。进入检索池只是第一道关卡。模型还必须决定哪些页面值得引用。
Ahrefs 发现，被引用页面的标题与用户查询的语义相似度明显更高，而使用描述性自然语言 URL 别名的页面比使用不透明 ID 的页面被引用得更多。这就是 llms.txt 和 Markdown 路由有用的原因：它们给模型提供了一个清晰、无歧义的信号，说明你的页面涵盖什么内容。
品牌通过第三方来源被引用的频率是通过自己域名被引用的 6.5 倍。有人在 Reddit 或 Hacker News 上夸赞你的项目，比你自己写的营销文案更有分量。这正是拥有一个结构良好的 llms.txt 重要的原因：它为模型提供了一个可引用的锚点，即使触发查询的那段对话发生在别处。
市面上有一些 AI SEO 审计工具会给你的网站打分，并让你添加 FAQ 部分、信任页面或更多文字。不要让分数驱动你的决策。检验标准很简单：你添加的每一段话是否包含页面上尚不存在的信息？如果没有，就不要添加。我曾给 Yobi 添加过一个 FAQ，内容只是复述了 About 部分已有的信息，纯粹是为了拉高分数。那是凑数，我把它删掉了。
这里讲的一切都是为了帮助 AI 准确理解你拥有的内容，给它一个干净的工作环境。这比任何捷径都更持久。
基础配置大约需要一个小时。知识端点和按项目划分的页面耗时更长，但一旦数据结构就位，维护就很容易了。每日同步会自动运行。
给它几天时间让爬虫抓取这些更改，然后在 ChatGPT、Perplexity 或 Claude 中搜索你的名字或项目试试。引用应该会更准确。
AI 引用的归属仍然不可靠。CJR 和 Tow Center 测试了 200 条 AI 生成的引用，发现其中 153 条存在部分或完全错误。做这些结构性工作是因为它能让你的内容更容易被准确访问，但不要把一条 AI 引用当作用户看到了你原话的证据。这一机制仍在改进中
如果你有自己的产品、博客或网站，不妨试试。你也可以把这篇文章交给 Claude Code，让它帮你完成大部分设置。
## 延伸阅读
1. GEO: Generative Engine Optimization - 普林斯顿大学 & 印度理工学院德里分校，KDD 2024
2. 海外 GEO 研究 - geo-citation-lab
3. llms.txt 标准规范
4. ChatGPT 为什么引用这一页而不是另一页 - Ahrefs
5. GEO 基准研究 2026 - ConvertMate
6. 为 AI 发现优化内容 - Evil Martians
7. LLM 实际上如何使用 Schema 标记 - SearchVIU
8. AI 搜索的引用问题 - CJR / Tow Center
9. LLMs.txt：品牌为何依赖它以及它为何不起作用 - SE Ranking
10. LLM 多久访问一次 llms.txt？ - Mintlify
11. IndexNow 协议文档
## 相关链接
- [Tw93](https://x.com/HiTw93)
- [@HiTw93](https://x.com/HiTw93)
- [2.5K](https://x.com/HiTw93/status/2050931710066565374/analytics)
- [directory.llmstxt.cloud](https://directory.llmstxt.cloud/)
- [llmstxt.site](https://llmstxt.site/)
- [tw93.fun](https://tw93.fun/)
- [weekly.tw93.fun](https://weekly.tw93.fun/)
- [yobi.tw93.fun](https://yobi.tw93.fun/)
- [Google Search Console](https://search.google.com/search-console)
- [yoursite.com/sitemap.xml](https://yoursite.com/sitemap.xml)
- [api.indexnow.org/indexnow](https://api.indexnow.org/indexnow)
- [pplx.ai/publisher-program](https://pplx.ai/publisher-program)
- [example.com](https://example.com/)
- [docs.example.com](https://docs.example.com/)
- [api.example.com](https://api.example.com/)
- [example.com/llms.txt](https://example.com/llms.txt)
- [example.com/projects/xxx.md](https://example.com/projects/xxx.md)
- [example.com/api/projects.json](https://example.com/api/projects.json)
- [@yaojingang](https://github.com/yaojingang)
- [geo-citation-lab](https://github.com/yaojingang/geo-citation-lab)
- [完整报告](https://github.com/yaojingang/geo-citation-lab/blob/main/04-repet/final_report.md)
- [GEO：生成式引擎优化 - 普林斯顿大学与印度理工学院德里分校，KDD 2024](https://arxiv.org/abs/2311.09735)
- [海外 GEO 研究 - geo-citation-lab](https://github.com/yaojingang/geo-citation-lab)
- [llms.txt 标准规范](https://llmstxt.org/)
- [为什么 ChatGPT 引用这个页面而不是另一个 - Ahrefs](https://ahrefs.com/blog/why-chatgpt-cites-pages/)
- [2026 年 GEO 基准研究 - ConvertMate](https://www.convertmate.io/research/geo-benchmark-2026)
- [为 AI 发现优化内容 - Evil Martians](https://evilmartians.com/chronicles/optimizing-content-for-ai-discovery)
- [LLM 实际上如何使用 Schema 标记 - SearchVIU](https://searchviu.com/en/how-llms-actually-use-schema-markup/)
- [AI 搜索存在引用问题 - CJR / Tow Center](https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php)
- [LLMs.txt：为什么品牌依赖它以及为什么它不起作用 - SE Ranking](https://seranking.com/blog/llms-txt/)
- [LLM 多久访问一次 llms.txt？- Mintlify](https://www.mintlify.com/blog/how-often-do-llms-visit-llms-txt)
- [IndexNow 协议文档](https://www.indexnow.org/documentation)
- [升级到 Premium](https://x.com/i/premium_sign_up)
- [2026年5月3日 21:33](https://x.com/HiTw93/status/2050931710066565374)
- [2,598 次浏览](https://x.com/HiTw93/status/2050931710066565374/analytics)
---
*导出时间： 2026/5/3 22:19:54*
