Watch: How to Audit Any Site With Serpent, Our Free Open-Source SEO Crawler
A hands-on guide to Serpent, our free open-source SEO crawler: a real 200-URL crawl of Smashing Magazine, the 1,502 issues it found, GEO scores for AI search, the llms.txt check, and driving crawls from Claude over MCP.
Read the full reviewTranscript
Serpent is a free, open-source SEO crawler. We built it. We pointed it at smashingmagazine.com: 200 URLs, about 1,500 issues. The critical row holds three internal links that return a 404. Two of them are the same bug, a stray quote inside an image link. The homepage has no text H1. Its only H1 wraps the logo image, so crawlers and screen readers see nothing. The GEO tab scores every page for AI search. Not one page clears 70 out of 100, and structured data scores zero, because no page ships JSON-LD. Redirects show a pattern. Every category link on the site goes through a redirect, so one template fix removes that hop from all of them. Embeddings find pages that compete for the same question. The printed book and the ebook for one title score 97 percent against each other. The crawl ran on one machine, with no account. Embeddings need your own API key, and paid features stay off until you turn them on, with a spend limit you set. Start with your own site. Keep the first crawl small, fix the 404s first, then rewrite the three pages with the lowest GEO score. Serpent is MIT-licensed, with no URL caps. The full guide is on organikpi.com.