How the crawl works
- It starts from the URL you give and follows links to pages of the same site (www and non-www count as one site), up to 5 clicks deep.
- It respects robots.txt, sends at most two requests at a time with a pause between them, and never bypasses bot protection or logins.
- Each page's status, title, description, headings, canonical, robots directives, word count and links are recorded. Links the crawl didn't visit are then checked.
- The sitemap (from robots.txt, /sitemap.xml or /sitemap_index.xml) is compared with the crawl to find orphan pages and pages missing from it.
What the issues mean
- Errors: broken internal links, pages answering 4xx/5xx, pages without a title.
- Warnings: duplicate titles and descriptions, missing descriptions and H1s, thin pages, internal links to redirects, slow pages, orphan pages, broken external links.
- Notices: noindex pages, canonicals pointing elsewhere, deep pages, images without alt text, pages missing from the sitemap.
- The site health score is the share of crawled HTML pages with no error-level problem.
Limits of a free crawl
Up to 200 pages and 200 link checks per audit, 6 minutes at most, 4 audits per 10 minutes per network. Pages built entirely with JavaScript are seen as the server sends them. Audits run in a queue; results stay available for one hour.
Frequently asked questions
How long does an audit take?
Usually one to three minutes for 100 pages. The crawl is deliberately polite: it never hits a site with more than two requests at a time.
Will the crawl slow down my site?
Unlikely: two requests at a time with a pause is lighter than most search engine crawlers. You can block it with User-agent: RankCortexBot in robots.txt.
Can I audit more than 200 pages?
Not in the free tool. A RankCortex account monitors whole sites continuously with Search Console data.