# CrawlSentry

> CrawlSentry monitors the SEO configuration of key pages every hour and emails a before → after diff when something changes: noindex (meta robots and X-Robots-Tag), robots.txt rules (including AI search crawlers like OAI-SearchBot, PerplexityBot and Claude-SearchBot), canonical tags, titles, redirects and status codes, plus daily SSL and domain expiry checks. Run by GetInput Oy, Helsinki, Finland.

Plans: Free (3 URLs), Hobby (€9/month, 25 URLs), Agency (€29/month, 250 URLs, client projects, Slack/Discord/webhook alerts, client recipients). It is not a rank tracker, site crawler or uptime pager.

## Free tools

- [SEO config check](https://crawlsentry.com/check): what Googlebot sees on a URL — noindex, robots.txt, canonical, redirects, title, SSL and domain expiry
- [robots.txt tester](https://crawlsentry.com/tools/robots-txt-tester): whether a URL is blocked for a given crawler, with the deciding rule and line
- [AI visibility checker](https://crawlsentry.com/tools/ai-visibility-checker): whether ChatGPT search, Perplexity, Claude and other AI answer engines can reach and read a page
- [AI robots.txt generator](https://crawlsentry.com/tools/ai-robots-txt-generator): robots.txt that blocks AI training crawlers while keeping AI search

## Guides

- [Is your site invisible to ChatGPT? A technical GEO checklist](https://crawlsentry.com/blog/geo-technical-checklist)
- [AI crawler user agents & robots.txt templates](https://crawlsentry.com/blog/ai-crawlers-robots-txt)
- [How to check if ChatGPT can see your website](https://crawlsentry.com/blog/can-chatgpt-see-my-website)
- [Does llms.txt matter?](https://crawlsentry.com/blog/does-llms-txt-matter)
- [Is Cloudflare blocking ChatGPT and other AI crawlers?](https://crawlsentry.com/blog/cloudflare-blocking-ai-crawlers)
- [Site migration SEO checklist](https://crawlsentry.com/blog/site-migration-seo-checklist)
- [How to monitor SEO changes: 5 approaches compared](https://crawlsentry.com/blog/seo-change-monitoring-options)
- [XML sitemap broken after a deploy? The 7 usual causes](https://crawlsentry.com/blog/xml-sitemap-broken-after-deploy)
- [hreflang mistakes that send users to the wrong country](https://crawlsentry.com/blog/hreflang-mistakes)
- [Accidental noindex: how one deploy wipes your traffic](https://crawlsentry.com/blog/accidental-noindex)
- [X-Robots-Tag: the noindex you can't see in view-source](https://crawlsentry.com/blog/x-robots-tag-noindex)
- [Canonical tags pointing to staging](https://crawlsentry.com/blog/canonical-points-to-staging)

## Contact

support@crawlsentry.com