Bulk Technical SEO Crawler & URL Auditor
How to use the technical SEO crawler
Back to the toolOne run, one request per URL — you choose how much of each page it reads back.
- 01
Enter one URL, or a bulk list
Single mode checks one address and runs on Enter. Switch to Bulk to paste a list, one URL per line, up to 30 per run. A bare domain works too; it is expanded before the request goes out. Every group you tick runs against the same list, so one paste covers every check.
- 02
Select which checks to run in Data checked
Nothing is ticked when the page opens, and Check URLs stays locked until you tick at least one: the field decides what comes back, so starting it half-full would run a check you never chose. All at the top toggles the whole set; press it again to clear everything. Each tick adds its own columns to the table.
- 03
Read the table as rows land
The first two columns are always there. Everything between them and Result comes from what you ticked, in the tool’s own order rather than the order you clicked. Two groups asking for the same column still grow one column, not two. A row lands as soon as its URL answers, so slow hosts fill in last.
- 04
Open a row, then export the table
View opens the detail panel: one section per group you ticked, which is where a check explains itself. Copy table puts the rows on your clipboard as tab-separated text; Export CSV writes them to a file, with the hop addresses and the source of every directive that the table only had room to summarise.
How to read the URL audit report
A row reads left to right: the address you gave, then what came back for it.
Columns in every run
- URL
- The address exactly as it was checked — after normalization, not always what you typed.
- HTTP status
- The second column always answers one question: what the address you gave actually returned. Under this name it is one code — what the first hop replied, with no destination address. Tick a redirect check as well and the whole chain gets a column of its own beside it.
- Result
- Whether the check itself finished. A URL that timed out or was refused says so here, as a plain badge, not a control; the button that re-runs the row sits in Actions.
- Actions
- Buttons, not answers. A row with something to expand gets View; a row that failed gets a retry that re-runs only that URL.
Columns added by each check
- Chain
- The whole journey, when the redirect check is ticked: every code the request passed through, then the address it stopped at. Past five hops the codes collapse into a count badge plus the last one. It stays empty when nothing answered at all — that case is what HTTP status reads ERR for.
- Robots
- Every indexing directive the page declares, in the wording it used. The panel says where each one was declared, because that decides which file you go and fix.
- Canonical
- The address the page names as its own preferred version. More than one is worth looking at: search engines discard the lot.
- Hreflang
- The language codes the page declares, written exactly as it wrote them. The full addresses are in the panel: a page with nine pairs would make the table unreadable.
- Open Graph
- The tag names the page declares for link previews, without the family prefix the column heading already gives.
- The same, for the card tags X reads. A page can declare one family and not the other, which is why they are two columns rather than one.
One check at a time
Every group above is also a tool of its own — same engine, one signal, and a guide that goes further than this page does.