GitHub Project

Serpent

A free, open-source desktop SEO crawler with a built-in MCP server. Serpent crawls unlimited URLs locally, keeps everything in SQLite, and lets Claude Desktop or any MCP client start crawls and read the results directly. Optional Bright Data engine for bot-protected sites.

MIT

Serpent is a desktop site crawler for technical SEO audits. It runs entirely on your machine, stores every crawl in a local SQLite file, sends no telemetry, and has no URL cap or licence server. It is MIT licensed and the source is public.

The part that matters for agent workflows is the built-in MCP server. While the app is open it serves Model Context Protocol on http://127.0.0.1:7777/mcp, bound to localhost and never exposed to the network. Point Claude Desktop or any MCP client at it and the assistant can start a crawl, poll its progress, list past crawls, read page-level SEO data, pull issues grouped by severity, and export the whole thing as CSV. No manual export step in the middle.

Crawling runs on a local engine by default. Sites behind bot protection can be routed through an optional Bright Data Web Unlocker zone with your own key, and JavaScript pages render through Electron’s bundled Chromium, so there is no separate headless browser to install. AI analysis is bring-your-own-key across OpenAI, Anthropic, Gemini, OpenRouter and local Ollama, and keys live in the OS keychain rather than a config file.

Installers for Windows, macOS and Linux are on the releases page. The SignPath Foundation signing application is still pending, so Windows will warn on first run until it clears.

Key features

Unlimited URL crawling with no licence server and no subscription, MIT licensed
Built-in MCP server on 127.0.0.1:7777 with seven tools, so an AI client can drive crawls and read issues without a manual export
Local crawl engine by default, with an optional Bright Data Web Unlocker engine for bot-protected sites
JavaScript rendering through Electron's bundled Chromium, so no separate headless browser is needed
Semantic analysis with embeddings, which finds pages that compete on meaning rather than on shared wording
Severity-scored issues, PageRank-style internal link equity, and a treemap of the site
Google Search Console OAuth import with orphan page detection
Headless CLI mode for scripted crawls, plus CSV and JSON export on every tab
Local SQLite storage, OS-keychain secrets and zero telemetry

Built with

  • Electron 35
  • React 19
  • TypeScript 5.8
  • Vite 6
  • better-sqlite3
  • Model Context Protocol SDK
  • Playwright
  • Vitest

Want help getting cited by AI search?

These tools are free to use. If you would rather have it done for you, let us put your brand in front of ChatGPT, Perplexity and Google AI.