# Robots.txt for Vinteler - Intervention apres sinistre en Belgique # https://vinteler.be # Search engines, AI search crawlers, social previews and SEO audit bots. User-agent: * User-agent: Googlebot User-agent: Googlebot-Image User-agent: Bingbot User-agent: MSNBot User-agent: DuckDuckBot User-agent: Slurp User-agent: Yandex User-agent: Baiduspider User-agent: GPTBot User-agent: ChatGPT-User User-agent: Google-Extended User-agent: ClaudeBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: anthropic-ai User-agent: Claude-Web User-agent: OAI-SearchBot User-agent: PerplexityBot User-agent: Perplexity-User User-agent: FacebookBot User-agent: meta-externalagent User-agent: cohere-ai User-agent: Applebot User-agent: Applebot-Extended User-agent: Amazonbot User-agent: Bytespider User-agent: CCBot User-agent: facebookexternalhit User-agent: Twitterbot User-agent: LinkedInBot User-agent: Pinterest User-agent: AhrefsBot User-agent: SemrushBot User-agent: MJ12bot User-agent: DotBot Allow: / # Internal source folders should stay out of crawler queues even if a deploy # misconfiguration exposes them later. Disallow: /doc/ Disallow: /docs/ Disallow: /_includes/ Disallow: /_layouts/ Disallow: /_data/ Disallow: /tools/ Disallow: /reports/ Sitemap: https://vinteler.be/sitemap.xml # AI discovery hints. Standard robots parsers ignore unknown directives; # AI agents can use them to find the canonical context files without guessing. LLMsTxt: https://vinteler.be/llms.txt LLMsTxt-Full: https://vinteler.be/llms-full.txt AI-Context: https://vinteler.be/ai-context.json