Keenable crawlers

Effective date: September 2026

Keenable.ai, Inc. operates the crawlers described on this page. We build a search index that answers queries from AI agents and applications. We publish named user agents so that site owners can identify our requests and control them separately.

Each token below is controlled on its own. Rules take effect after we next fetch your robots.txt.

KeenableBot

KeenableBot collects publicly available pages to build and maintain the Keenable search index. That index answers queries from AI agents and applications, and answers cite the source page with a link. Disallowing KeenableBot stops us from indexing your site, which removes it from those answers.

Manage it with the User-agent: KeenableBot directive in robots.txt.

Full user-agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; KeenableBot/1.0; +https://keenable.ai/bot)

Keenable-User

Keenable-User identifies requests we make on behalf of a person, or an agent acting for one, such as when someone asks for a specific page. It does not crawl the web automatically. It is published so that site owners can see user-initiated requests in their logs.

Full user-agent string

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Keenable-User/1.0; +https://keenable.ai/bot)

Controlling our crawlers

We follow the Robots Exclusion Protocol (RFC 9309). A group that names one of our tokens takes precedence over a User-agent: * group.

To block everything

User-agent: KeenableBot
User-agent: Keenable-User
Disallow: /

To limit KeenableBot to part of your site

User-agent: KeenableBot
Disallow: /private/

Indexing rules

We read the noindex, nofollow and none directives, both in a <meta name="robots"> tag and in the X-Robots-Tag response header. A directive addressed to keenablebot applies to us specifically.

Verifying our requests

A user-agent header can be copied by anyone, so it is not proof on its own. We are adding a cryptographic signature to every request, following the IETF HTTP Message Signatures drafts, so that a site can check our traffic against something harder than a string. It is not live yet, and this page will say so until it is.

Until then, write to abuse@keenable.ai with the hostname, the timestamps and the source addresses, and we will tell you whether the traffic is ours.

Contact

Write to abuse@keenable.ai to report excessive load, ask for a lower crawl rate, request removal of specific content, or report a crawler that misrepresents itself as ours.

Changes to this page

September 2026. First publication, covering KeenableBot and Keenable-User. A token we stop sending stays documented here, because rules naming it stay in your files.