The Kairos research bot

You are probably here because Kairos-Research turned up in your server logs and you wanted to know who that was. Fair enough. Here is the whole thing, honestly, in about thirty seconds.

Who runs it

Kairos is a small outreach tool built and run by Taybridge Group Limited, a company registered in Scotland (No. SC885181) and based in Brechin, Angus. It is run by one person, not a data broker — so if you write to the address at the foot of this page, he is who answers.

What it does

It has three jobs, and it reads public web pages for all three.

Before one of our customers emails your company, the bot reads a small number of pages from your own public website — normally your home page and one or two of your About or Services pages — so that the email they send is about what you actually do, rather than generic filler that could have been sent to anyone.

To be sure which company it is, we may also run a few web searches about the company, and look it up on public registers — Companies House; the Financial Conduct Authority's, for a society or a firm it regulates; the Charity Commission's or the Scottish Charity Regulator's, for a charity; and those kept by professional regulators and stock exchanges — including who its directors are and which other companies they run. What the web searches return is used only to work out which company it is, and none of it goes into an email; from the registers, an email may mention the company's trade and town, and nothing else. Directors' names read on the registers are not kept and are never used in the writing. Once your site has told our bot no, we leave it out of those searches too.

When one of our customers asks Kairos to write about a topic — a product launch, a change in the rules, a local event — the bot reads a small number of public pages about that topic: an official body, the company the topic is about, the press, or another public site that covers it. It finds those pages with a web search, and keeps only sentences it can copy word for word from the page, with the address it read them at. It does not read forums or discussion threads. So if you run a government, supplier, retailer or news site, that is probably why you have seen it.

When a business becomes a Kairos customer and gives us its own website, the bot may read the whole of that site — every public page it can read as text — because the business asked us to, so that what Kairos writes for them matches what they say about themselves. We take the customer’s word that the site is theirs and do not check it, so if one of our customers has given us a site that is not theirs, block the bot or write to us as described below, and it stops. It still obeys robots.txt, still sends one request at a time with a gap, and keeps no sentence that names a person or another business.

Every time, it reads what has been published, and then it goes away.

What it does not do

• No social platforms. It never touches LinkedIn, Facebook, X, Instagram or anything like them. Those hosts are refused in the code before a request is made, not merely discouraged in a policy document. The web search it uses to find topic pages is told to leave them out too.
• No email addresses or phone numbers. It does not build a list of addresses from your site. There is no address-harvesting step in it at all, every link address — mailto: ones included — is stripped out of the text it keeps, and a sentence containing an email address or phone number is not kept.
• No personal data harvesting. It is looking for what a company does and where it does it, or what a page says about a topic — not for people. A sentence that quotes a named person is not kept from a topic page. From a company's own site, a name can stay inside a sentence it keeps, such as who founded the business.
• No bulk crawling. A handful of pages, then it stops. It does not walk your whole site — unless the site belongs to a Kairos customer who asked us to read it, as described above.
• No ignoring robots.txt. See below.
• Nothing behind a login. It does not sign in, submit forms, or follow anything that is not a plain public page.

How it behaves

• It identifies itself honestly on every single request: Kairos-Research/0.1 (+https://bot.yourkairos.tech) — and it can also sign its requests, so you can check a request really came from us (see below).
• It reads your robots.txt before it fetches a single page from your site, and again for any host it is redirected to. If your robots.txt cannot be reached, or answers with a server error, it treats that as no and fetches nothing — it fails towards leaving you alone.
• For a company: it reads at most four pages of your site — your home page, then two more chosen from the links on it, and sometimes a fourth.
• The fourth page is fetched only when those first three do not state the company’s registered details — a company number, a “trading as” name, or a registered address. It is always a page you have linked to. It is picked either because the link’s own wording points at it (a link reading “Registered office” or “Company number”, whatever address it points to), or because the link is to a legal notice, a privacy policy, terms, or a contact page. If your site offers neither, no fourth page is fetched. It never guesses an address you have not linked to.
• If one of those pages is missing or will not load, it may try one other address from your own links in its place, and no more.
• For a company we research, the most requests your server can receive from us in a single visit is ten, and on a site that answers first time it is four. Ten is the worst case with everything going wrong at once, and it is made up of: up to three tries at your home page, because we try https://, then www, then http:// if one does not answer; your robots.txt once for each of those three addresses; the two pages chosen from your home page; one replacement for a page that will not load; and the fourth page described above. Redirects (at most five per request) come on top, as they always have.
• It does not go looking for how many people a company employs. If the company's own pages say, that is noted; if they do not, nothing else is read to find out.
• We never read a cached, archived or mirrored copy of a site that has told our bot no. Once your site has told our bot no — in its robots.txt, or by asking us — the answer is no by every route: no Google cache, no Internet Archive, no text proxy, and no searching our way back in — we leave your site out of our web searches too. If your site turns away every automated visitor without saying so in its robots.txt, we may still read what a search engine shows about it, as anyone searching would, but only to work out which company it is. This crawler uses one name and announces it on every request.
• What we keep from someone else's page is a sentence quoted exactly as published, with the address it came from and, where the page states one, the date. We do not summarise it, rewrite it, or treat it as the company's own words — our record says plainly that a third party published it.
• For a topic: it asks for at most ten addresses in all, reads at most eight pages, and never reads more than three from any one site.
• Reading robots.txt, and following a redirect (at most five), come on top of those limits.
• One request at a time to your server, at least a second apart. It will never be the reason your server is busy.
• For a company we research, it honours Crawl-delay up to ten seconds. Longer than that and it gives up on the page rather than sit on your server waiting.
• For a Kairos customer’s own site, read at the customer’s request: it reads at most a thousand pages in one read, it honours a Crawl-delay of up to sixty seconds — it is in no hurry — and if a read runs long it stops and carries on later rather than keep going. It does not read PDFs, text inside images, video, or pages that only show their words in a browser.
• How often it comes back:
  • A company it is about to email: once read, not read again for about three months.
  • A company one of our customers has named as their supplier, partner, client or competitor, or a firm in the same trade near them: it may read your site again about once a month, to keep what Kairos knows current.
  • A Kairos customer’s own website, read because they asked: about once a week, and again whenever they ask for a fresh read — at most once a day for each customer who gives us the site. A page that does not load, or whose robots.txt does not answer — or a whole site that does not — is asked for again the next day, not sooner.
  • A topic page: it may read it again when that topic is looked up again — about a month later if something useful was found, a week later if not, the next day if the last try failed — or when a different topic leads to the same page.
  • While we test the bot itself, it may read the sites of a small set of businesses — some well known, some that our customers plan to write to — more than once within a few days, within the same limits on each visit. Nobody is emailed because of a test.

Checking a request is really ours

Anyone can copy a User-Agent string, so the bot can also sign its requests with Web Bot Auth, an open standard built on HTTP Message Signatures. A signed request carries Signature, Signature-Input and Signature-Agent headers, and our public key is published at https://bot.yourkairos.tech/.well-known/http-message-signatures-directory. If a request carries a Kairos-Research signature, you can check it against that key. A signature that does not check against it is not ours.

Kairos-Research is not a Cloudflare verified bot at the moment. A signature only says who is asking. It never overrides your robots.txt, and blocking the bot works exactly as described below.

How to block it

Put these two lines in your robots.txt:

User-agent: Kairos-Research
Disallow: /

It never relies on a copy of your robots.txt more than an hour old, so it stops within an hour of the change. A blanket User-agent: * rule blocks it too — we read the wildcard group as well as our own, which is why a site that already disallows everything has never heard from us.

If you would rather just tell a person than edit a file, use the address below and we will add your site to the list the bot refuses to fetch. That list is checked before every request, and a site added to it is left alone within five minutes.

Questions, or a complaint

A real person reads this and will answer you: hello@yourkairos.tech

If the bot has done something this page says it does not do, we would genuinely rather hear about it than not. Send us the log lines — the timestamp, the path and the User-Agent string are enough — and we will tell you what happened.

What we keep, why, for how long, and your rights over it — including if a page we read names you — are in our privacy notice: yourkairos.tech/privacy.