SiteSearch
about
A small, self-hosted search engine that fits on the Cloudflare free tier — the web tab works out of the box, the sites tab searches whatever you choose to crawl.
There are two search tabs, and they work in completely different ways:
The web
Every query is sent to several independent, open upstreams at once (currently DuckDuckGo’s HTML endpoint and the Wikipedia search API; a Brave Search API key can be added too). The worker then merges the result sets the way the SearXNG project taught everyone:
- de-duplicate by URL (preferring https, keeping the longer title/snippet),
- score each result by how many independent engines returned it and where
they ranked it:
score = engines × Σ(1/(rank+1)), - return the merged list. No ads, no promoted slots, no user profile.
Your sites
Search an index you build. Add a site in the console with a sitemap URL
(or plain seed links), and the worker crawls it on a schedule — following same-domain
links, honoring robots.txt, re-crawling stale pages — into a D1 database
with a real SQLite FTS5 full-text index. This is the piece that runs strictly on
Cloudflare, no third party involved.
where the ideas come from
SiteSearch is a learning project, and it stands on the shoulders of real open-source search software:
| Project | What SiteSearch borrows |
|---|---|
| SearXNG (AGPL-3.0) | The meta-search architecture: parallel upstreams, URL-hash de-duplication, position-weighted scoring, no user tracking. |
| YaCy (GPL-2.0+) | That a self-crawled, independent index is worth owning — the “your sites” tab is our small YaCy nod. |
| Whoogle (MIT) | A cautionary tale: Google successfully killed no-JS scraping in 2026. Proof that a search backend should not depend on a single hostile provider. |
| LibreY (AGPL-3.0) | Metasearch without the bloat — and a reminder that a clean default UI matters. |
| Mojeek / Brave Search | Independent indexes you can add as upstreams via API (Brave) so SiteSearch isn’t held hostage to any one engine. |
who runs it
SiteSearch is a personal project by a Cloudflare user, deployed to their own account. You can run an identical copy for yourself — everything is in the repository’s docs, including the one-line deploy steps.
← back to search