# robots.txt for AI Daily Digest Podcast # https://homepage.marketing # Allow all web crawlers User-agent: * Allow: / # Allow major AI crawlers explicitly User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / User-agent: Googlebot Allow: / User-agent: Googlebot-Extended Allow: / User-agent: Bingbot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: anthropic-ai Allow: / User-agent: cohere-ai Allow: / User-agent: Diffbot Allow: / # Sitemap location Sitemap: https://homepage.marketing/sitemap.xml # LLMs.txt location (for AI systems) # https://homepage.marketing/llms.txt