Site Qwality Monitor, our bot
Site Qwality Monitor is the bot that runs Site Qwality’s uptime checks. If it shows up in your logs, a Site Qwality customer has set up a monitor for a URL on your site, usually because the site is theirs.
What it does
Section titled “What it does”- It sends the HTTP checks our customers set up to confirm their sites are up and fast. Each check is one request to the exact URL the customer entered, with the method, headers and body they chose.
- It does not crawl. It never follows links, discovers pages or indexes content. It follows redirects only when the monitor is set to.
- It reads the response only to evaluate the check: the status code, timings, and any keyword or JSON value the customer asked for. When a check fails, the response is saved with the result so the customer can see what went wrong. Content is never used to train models or to build an index.
How often
Section titled “How often”- Each monitor runs on the interval its owner picks, at most every 30 seconds.
- A monitor can check from several locations. Each location sends its own request on every run.
- A failed request may be retried up to twice within the same check.
Where requests come from
Section titled “Where requests come from”Checks run from check locations around the globe on AWS, Google Cloud and Microsoft Azure. The current list is on the locations page. Requests leave from each provider’s shared addresses, so there is no IP list. To confirm a request came from us, verify its signature.
How to recognize it
Section titled “How to recognize it”| Header | Value |
|---|---|
User-Agent | site-qwality-check/, unless the monitor sets its own |
X-Site-Qwality-Check | 1 |
Signature-Agent | "https://bot.siteqwality.com" |
Signature-Input, Signature | A Web Bot Auth signature |
Verifying a request
Section titled “Verifying a request”Anyone can copy a User-Agent. A signature cannot be copied. Site Qwality signs check requests with Web Bot Auth, which uses HTTP Message Signatures (RFC 9421) and Ed25519 keys. Our public keys are published at:
https://bot.siteqwality.com/.well-known/http-message-signatures-directoryA request came from Site Qwality when its Signature verifies against one of those keys, its Signature-Input has tag="web-bot-auth" and covers @authority and signature-agent, and its expires time has not passed. Bot-protection services that support Web Bot Auth can run this check for you.
robots.txt
Section titled “robots.txt”Site Qwality Monitor does not read robots.txt, because it never crawls: every request goes to a URL a customer chose to monitor. If we add a feature that crawls, it will follow robots.txt, and this page will say so before it ships.
Allowing or blocking it
Section titled “Allowing or blocking it”- To let checks through a firewall or bot protection, see allowlisting our checks.
- To block it, match the
X-Site-Qwality-Checkheader or the User-Agent in your firewall. - To have checks stopped, or to report a monitor that misbehaves, email hello@siteqwality.com with the affected domain.
This page covers HTTP checks. Browser checks, which run a full Chrome browser, are not signed yet.