ScanifyAuditBot

You are probably here because this string appeared in your access log. It is an automated crawler operated by Scanify (scanify.ltd). It fetches a small sample of pages from Shopify storefronts and measures them for technical SEO problems.

The exact strings

ScanifyAuditBot/0.1 (+https://api.scanify.ltd/bot)
ScanifyDiscoveryBot/0.1 (+https://api.scanify.ltd/bot)

Two components, both of them ours. The first crawls a store to audit it; the second is the smaller one that checks whether a domain is a Shopify store at all and looks for a published contact address. Neither string ever varies, and neither rotates. Requests reach you from one of two places and never a third: our own address, or — for the storefront pages and .json documents named below, and never for /robots.txt — a commercial fetch service, Firecrawl, from an address in their pool that we do not own, cannot enumerate, and which carries other customers' traffic as well as ours. Whichever carries the request, we send the same user agent above and never a browser's; nothing in the code picks a different identity, or a different class of address, after a refusal.

How to block it

Put this in your /robots.txt:

User-agent: ScanifyAuditBot
Disallow: /

User-agent: ScanifyDiscoveryBot
Disallow: /

We fetch /robots.txt from a hostname before we request anything else from it, and we do not request anything it disallows for these tokens. If the file cannot be read — a 5xx, a 429, a 403, no answer — we treat the store as disallowed rather than as unrestricted, and go away. A Crawl-delay slower than our own spacing is honoured too.

www.example.com and example.com are separate authorities and we read a /robots.txt for each, so a rule published on only one of them applies to only one of them. If the name we were given refuses us we stop there; we do not go and try the other one.

If you would rather not edit a file, mail replies@mail.scanify.ltd and say which domain. Blocking us costs you nothing and we will not ask why.

What it fetches, and how much

One audit of one store fetches, at most:

• /robots.txt, /llms.txt, /sitemap.xml and up to 20 of the sitemaps it indexes;
• one request to http:// your host, not followed, to see whether plain HTTP redirects to HTTPS;
• your homepage, up to 10 product pages and up to 5 collection pages, chosen deterministically from your sitemap;
• the .json document Shopify serves beside each of those product pages.

A small store costs 10 requests in total. A hard ceiling of 70 requests per audit is enforced by a counter, not by arithmetic: when it is reached the crawl stops where it is and reports itself as incomplete.

Those are documents, and they are the small half of the bill. Each page in that list is then loaded a second time in a browser, and a browser page load is dozens of requests rather than one. The section below is the expensive part and the numbers in it are much larger than the numbers here.

How it paces itself

• One request at a time per host. No parallelism against your server.
• At least 2 seconds between requests to the same host, and at least 5 seconds between requests to any host on the same Shopify edge — so if you are behind Shopify, we are pacing ourselves against every other Shopify store we might be reading, not just yours.
• Retry-After is honoured. On a 429 or a 503 we retry at most once, never sooner than the header asks for, and if it asks for longer than we are prepared to sit still we stop instead of retrying.
• After a 429 or 503 the host is left alone for the rest of the crawl — between 1 and 15 minutes — rather than being asked again by every remaining URL.
• One mode, used when a person has asked for a specific store to be audited, will instead wait out a Retry-After in full: up to 5 minutes per wait and 30 minutes over the whole crawl. It waits; it does not retry faster.

Requests may carry an RFC 9421 HTTP Message Signature tagged web-bot-auth. The public key is published at /.well-known/http-message-signatures-directory on this host, so a signature claiming to be us can be checked rather than believed.

A third party may fetch on our behalf

Your storefront pages — the ones an audit samples, and the few the second component reads to identify the store and look for a published contact address — and the .json documents Shopify serves beside your product pages and at /products.json, may be fetched for us by Firecrawl (firecrawl.dev) rather than directly by us. Your /robots.txt, /llms.txt and sitemaps are always fetched by us and never by them; your /robots.txt is still read before anything else and still treated as a refusal if we cannot read it. Blocking the tokens above blocks us either way — the rules are the same whoever carries the request.

Two consequences, because you would otherwise have to discover them:

• The request counts above are a floor while this is in use. Firecrawl does not tell us how many redirect hops it followed, and it may load a page in a browser, whose subresources it does not count for us. Our 70-document ceiling is still enforced on our side; what arrives at your server may be more, and we cannot tell you how much more.
• The pacing above is ours and is applied before we call them, so those intervals still hold. We cannot promise Firecrawl adds none of its own.

It runs JavaScript, and that is the noisy part

After fetching a page we load it again in headless Chromium, because Shopify themes inject titles, headings, alt text and structured data client-side and an audit that read only the served HTML would report things as missing that are not. The browser sends the same user agent above.

How much this costs you, measured rather than estimated. On 4 August 2026 we measured a single rendered page of a real store at 324 requests across 28 hosts, 223 of them to that store's own server. We had a cap set at the time and it did not stop it, because it was checked between pages rather than during one. That was our defect, the store paid for it, and the paragraph you are reading exists because the previous version of this page said only that a render is "a burst of requests" and left you to find out how big a burst.

Then we measured what one page actually needs, and raised the caps to match it. A full load of one collection page of a Shopify store — images, fonts and tracking pixels all refused — was 169 requests, 155 of them to that store's own hostname. Almost all of it is the theme's own JavaScript. So a cap of 50 was not politeness, it was an audit that stopped half-way and measured nothing; the caps below are that measurement plus headroom, and they are considerably larger than the ones this page carried before.

What is in force now:

• A single page render is capped at 250 requests, decided as the page loads — each request is allowed or refused before it leaves the browser, so the cap is a cap and not a total we notice afterwards. Every browser request the audit makes is capped at 4000 per audit on top of that. Doing the multiplication for you: we render at most 16 of your pages, so the worst case is around four thousand browser requests to render one audit of your store, and a realistic one on a theme like the one we measured is closer to two or three thousand. That is more than a search engine costs you for the same pages and less than a customer browsing them, who also loads the pictures. Both figures are counted, and your own server's share is counted separately from everybody else's.
• We refuse things we do not need. Images, video, audio and fonts are never fetched: we read <img> tags and their alt text out of the page, never the picture behind them. Nor are analytics collectors, conversion and advertising pixels, session recorders or embedded video players — which also means we are not writing a robot into your Google Analytics, your Facebook pixel or your Shopify reporting. We do still load your theme's scripts, stylesheets and the requests they make, including your apps', because that is where storefronts put the headings, structured data and alt text we came to measure.
• Those browser requests are not spaced by the intervals above and are not individually checked against robots.txt. Only the page URL is checked; what the page then loads is whatever the page loads, minus what we refuse. It may include hosts that are not yours, such as a CDN.
• If a page hits the 250-request cap we stop it there, and then we throw the result away and keep what we measured from the plain HTML instead. A page we stopped loading is not a page we are willing to say anything is missing from.
• The crawler itself only ever issues GET requests. It submits no forms, adds nothing to a cart, and logs into nothing. Scripts on your own pages run during the render and may do whatever they normally do.

Complaints and block requests

replies@mail.scanify.ltd. Name the domain. A block is permanent, and we would rather have it than an argument.