Free tool for marketing and SEO teams
See if ChatGPT can crawl and name this site. You get a score plus copy-paste robots.txt, JSON-LD, and sitemap fixes. No signup.
What to do next
nytimes.com is blocking at least one major AI crawler. Until those bots can fetch the site, ChatGPT and similar tools cannot reliably use it as a source.
High impact · 1
Training, search, and live-answer bots each need access. If they are blocked, no amount of content work helps.
robots.txt (append)
User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: /
High impact · 2
Answer engines use structured data to attach a name, URL, and entity to the page. Paste this in the homepage <head>.
homepage <head>
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Organization",
"name": "nytimes.com",
"url": "https://nytimes.com",
"description": "nytimes.com, add a one-sentence description of what the company does."
}
</script>Medium impact · 3
If the page has no summary, models and crawlers have to guess what the company does from the rest of the HTML.
homepage <head>
<meta name="description" content="nytimes.com, add a one-sentence description of what the company does." />
robots.txt plus OpenAI, Claude, and Perplexity bots, training, search, and live answers.
16/35
robots.txt
PassA real robots.txt is live, crawlers have an explicit map.
Site is crawlable
PassUser-agent * can fetch the homepage.
OpenAI bots (training, search, live fetch)
FixGPTBot blocked · OAI-SearchBot blocked · ChatGPT-User blocked
Claude & Perplexity
FixClaudeBot blocked · PerplexityBot blocked
Title, description, H1, and Organization markup so models know who the page belongs to.
0/35
Page title
FixNo <title>. Crawlers and models get no brand name from the tab.
Meta description
FixNo meta description, add a one-line “what we do.”
H1 heading
FixNo H1 on the homepage.
Brand structured data
FixNo Organization JSON-LD. Models have no machine-readable brand name.
Sitemap, canonical URL, and optional FAQ/sameAs signals that help with clean citations.
0/11
Sitemap
FixNo sitemap.xml, crawlers often never leave the homepage.
Canonical URL
FixNo rel=canonical, duplicate URLs confuse citation.
FAQ / HowTo / Article schema
OptionalOptional. FAQ or HowTo schema helps when the page actually answers a question.
sameAs profile links
OptionalOptional. sameAs (LinkedIn, Wikipedia, Crunchbase) helps disambiguate the brand.
llms.txt and skill files help coding agents. They do not decide ChatGPT rankings.
0/10
llms.txt
OptionalOptional. A short llms.txt helps coding agents; it does not buy AI search traffic.
Agent docs
OptionalOptional. Skill files help tool-using agents, not classic GEO.
Full dossier · nytimes.com
The quick scan checks homepage basics. The full dossier runs live crawler tests, audits agent automation friction, and builds a custom 90-day execution sprint.
01 · Crawlers
Allow rules and fetch policy for GPTBot, ClaudeBot, PerplexityBot, and Google-Extended.
02 · Entity
Organization structured data and entity signals so models attach citations to the brand.
03 · Sprint
Severity-ranked P1 through P3 action items paired with copy-paste snippets and code fixes.
04 · Export
Toggle between executive summaries and developer code diffs, plus single-click PDF export.