Changelog

What's new in CrawlSentry. We ship small improvements often.

AI search (GEO) monitoring

• AI crawler alerts: get a warning when robots.txt starts blocking OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot or DuckAssistBot from a monitored page.
• AI visibility report for every monitored URL: a 0–100 score covering AI crawler rules, CDN/firewall blocks, content without JavaScript, structured data, headings, freshness and llms.txt.
• Ready-made fixes: a robots.txt group that lets AI search in (checked against your actual robots.txt), plus JSON-LD and llms.txt starters.
• Free tools: AI visibility checker and AI robots.txt generator.

More things we watch

• hreflang: alerts when language/region alternates are removed or start pointing to another host.
• XML sitemaps: a daily check that alerts when your sitemap disappears or loses more than half its URLs.
• robots.txt outages: a warning when robots.txt returns server errors for three hourly checks in a row (Google pauses crawling while that happens).
• The free SEO config check now also shows AI crawler access and hreflang.

For agencies

• Client report links: a read-only status page per project to share with clients, with current status and the last 30 days of changes. Turn it off or rotate it anytime.
• Client alert recipients: send a project's alerts to up to five extra people, with double opt-in.
• Slack, Discord and webhook alerts.

Quality of life

• Import URLs from your sitemap, export change history as CSV, and set the minimum severity that triggers an email.
• Weekly summary email (opt-in) with what needs attention right now.
• Starting a free check and then signing up now carries the URL straight into your dashboard.
• Free tools hub, guides on technical SEO and GEO, and a blog feed.

CrawlSentry is live

• Hourly checks of noindex (meta and header), robots.txt, canonical, title, redirects and status codes on your key URLs, with a plain before → after diff by email.
• Daily SSL certificate and domain expiry checks, with one warning per threshold.
• Free plan for 3 URLs, Hobby (25 URLs) and Agency (250 URLs, 50 projects).