How it identifies itself
Requests carry this user agent:
Mozilla/5.0 (compatible; RankCortexBot/1.0; +https://rankcortex.com/tools/bot)
What it does, and doesn't do
- It fetches the single page a user entered, plus that site's robots.txt. It does not crawl on its own.
- It respects robots.txt rules for the RankCortexBot token (or
*). - It sends at most 2 requests to the same site at a time and gives up after 15 seconds.
- It never bypasses logins, CAPTCHAs, Cloudflare or other bot protection. A blocked page is reported to the user, who can paste the HTML instead.
- It doesn't run JavaScript and doesn't submit forms.
Blocking RankCortexBot
Add this to your robots.txt:
User-agent: RankCortexBot Disallow: /