BikeHQBot

If you run a bike shop and this crawler showed up in your logs, this page says what it is, what it reads, and how to reach the person running it.

What it is

BikeHQ is a Canadian bike deals site being built by one person. It records public prices from Canadian bike shops so it can show readers a price against its own history. The crawler is how the prices arrive.

How it identifies itself

Every request carries this user agent:

BikeHQBot/1.0 (+https://bikehq.ca/bot; crawler@bikehq.ca)

What it reads

• Only the public /products.json endpoint of Shopify stores. Nothing else on the site. No pages, no images, no checkout.
• It reads robots.txt first and follows it.
• It never logs in, never uses a proxy, never disguises itself.

How often

• At most one request per second to any shop.
• Each shop is read about once a day, in small hourly batches spread across the day.
• If a shop answers with a rate limit, it backs off and waits longer each time. If a shop blocks it, it stops and does not try harder.

If you would rather not be listed

Write to crawler@bikehq.ca. Your shop is removed and the crawler stops, without argument. A robots.txt rule for BikeHQBot does the same thing without the email.

What you get out of it

Listing is free and does not depend on any commercial arrangement. Every shop the crawler can read is listed. When a reader clicks through to your shop, that is a reader who already knows the price.