# We Caught Our Web Host Returning 429s to GPTBot **URL:** https://organikpi.com/blog/technical-seo/gptbot-429-host-block/ **Published:** 2026-09-02 **Modified:** 2026-09-02 **Author:** Daniel Shashko > Our host answers GPTBot with 429 on every uncached page. Cached pages serve 200, so only fresh and low-traffic URLs suffer. OAI-SearchBot, Googlebot, and Bingbot all pass, so dashboards stay green. Three curl commands detect the block in five minutes. Scheduled cache warming restored 200s for us. --- > Our host answers GPTBot with 429 on every uncached page. Cached pages serve 200, so only fresh and low-traffic URLs suffer. OAI-SearchBot, Googlebot, and Bingbot all pass, so dashboards stay green. Three curl commands detect the block in five minutes. Scheduled cache warming restored 200s for us. Our robots.txt allows every AI crawler. Our web host answers GPTBot with HTTP 429 anyway, on any page that is not already sitting in cache. Search Console showed nothing, Bing Webmaster Tools showed nothing, and the block fires before WordPress runs. We measured the full pattern on 1 September 2026, and the same three curl commands will tell you in five minutes whether your host does this to your site. ## What we measured organikpi.com runs on a shared Hostinger plan behind a LiteSpeed cache. Requests with a GPTBot user agent were drawing 429 responses while every SEO dashboard stayed green. To pin down the rule, we sent requests with nine different user agents to the same blog post URL, once served from cache and once with a cache-busting query string. Two agents were blocked. Seven passed. User agentCached pageUncached pageGPTBot/1.4 (OpenAI, training)200**429**meta-externalagent/1.1 (Meta, training)200**429**OAI-SearchBot/1.4 (ChatGPT search)200200ChatGPT-User/1.0 (user-triggered fetches)200200ClaudeBot/1.0 (Anthropic)200200PerplexityBot/1.0200200Bingbot/2.0200200Googlebot/2.1200200Chrome browser (control)200200HTTP status by user agent and cache state, measured against the same live blog post URL on 1 September 2026. The pattern held on every retry. Three consecutive GPTBot requests to fresh uncached URLs drew three 429s, each answered in roughly 0.26 seconds. Those 429 responses carry no x-litespeed-cache header at all, while normal responses are stamped hit or miss. The rejection happens above the cache and above WordPress. The same URL, three outcomes. The user-agent rule only bites when the cache misses. This is documented behavior, not a misconfiguration on our end. Hostinger’s support article [on server-level rate limits](https://www.hostinger.com/support/429-errors-on-automated-integrations-and-link-previews/) states: “Hostinger has implemented server-level rate limits for traffic originating from specific network ranges, such as Meta, AWS, and Microsoft.” The same article notes the limits “cannot be disabled for individual websites or hosting plans.” ### The rule matches the user-agent string Our test requests came from a residential connection, nowhere near OpenAI’s [published GPTBot IP ranges](https://openai.com/gptbot.json). The string GPTBot in the user agent was enough to draw the 429. Hostinger’s article frames the limits around network ranges; on our server, the user agent alone triggers them. That detail is useful, because it means anyone can reproduce the block from a laptop. ### The cache decides who gets blocked A cache hit serves 200 to every crawler. A miss serves 429 to the two blocked agents. That inverts the usual risk profile: your busiest pages are almost always cached and safe, while the pages most likely to 429 are the ones you just published, the long tail nobody visits, and any URL with a query string. A new post is at its most invisible to training crawlers in the hours right after publish, which is exactly when [your sitemap is telling crawlers to come fetch it](https://organikpi.com/blog/technical-seo/xml-sitemap-priority-ai-crawler-signals/). ### The selectivity is why nothing looks broken Both blocked agents are training crawlers. OpenAI’s [crawler documentation](https://developers.openai.com/api/docs/bots) says GPTBot “is used to crawl content that may be used in training our generative AI foundation models”, while OAI-SearchBot decides whether a site appears in ChatGPT search answers. OAI-SearchBot passed every test we ran. So did Googlebot, Bingbot, ClaudeBot, and PerplexityBot. Your rankings hold, your [ChatGPT search visibility](https://organikpi.com/blog/geo-ai-search/how-to-rank-in-chatgpt-search/) holds, and no search console has a reason to warn you. The blocked crawls are precisely the ones that never appear in a dashboard. ## Why a 429 to a training crawler matters HTTP 429 means Too Many Requests. [RFC 6585](https://www.rfc-editor.org/info/rfc6585/) defines it as the server telling the client it has sent too many requests in a given amount of time, and well-behaved crawlers respond by backing off and retrying later. A page that answers 429 on every uncached fetch is a page a polite crawler learns to fetch less. Whether that costs you anything depends on a choice you have hopefully already made in your [robots.txt AI crawler policy](https://organikpi.com/blog/technical-seo/robots-txt-ai-crawlers/). If you disallow GPTBot deliberately, as many [publishers negotiating content licensing](https://organikpi.com/blog/brand-authority/content-licensing-ai-training-publishers/) do, the server rule costs you nothing. If you allow it because you want your content and brand represented in the models that answer buyer questions, the core bet behind [generative engine optimization](https://organikpi.com/blog/geo-ai-search/what-is-geo-generative-engine-optimization/), then the server is quietly overriding your stated policy. You opted in. The infrastructure opted you back out. Hostinger’s article lists the collateral beyond AI training: link previews on Facebook, Instagram, and WhatsApp can fail to render, automations in Zapier, Make, and n8n can error out, HubSpot and Salesforce syncs can fail, and uptime monitors can fire false downtime alerts. To be fair to the host, the stated purpose is keeping sites responsive for human visitors during crawler spikes, and AI crawler volume is a real cost on shared infrastructure. The problem is the silence, since none of this is visible to the site owner without testing for it. ## The five-minute test for your own site Three curl commands. Run them from any machine. # 1. A popular page, GPTBot user agent. Expect 200. curl -s -o /dev/null -w "%{http_code}\n" \ -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \ "https://yourdomain.com/" # 2. The same page with a cache-busting query. A 429 here is the tell. curl -s -o /dev/null -w "%{http_code}\n" \ -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \ "https://yourdomain.com/?t=$RANDOM" # 3. Control: another cache-busted URL, normal browser user agent. Expect 200. curl -s -o /dev/null -w "%{http_code}\n" \ -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" \ "https://yourdomain.com/?t=$RANDOM" Read the results as a pair. If the bot user agent gets 429 and the browser user agent gets 200 on the same uncached URL, something above your CMS is filtering by user agent. Add a header dump (curl -sI) to see your cache layer’s hit or miss stamp; on LiteSpeed that is x-litespeed-cache, on Cloudflare it is cf-cache-status. Then confirm the scale in your [server access logs](https://organikpi.com/blog/technical-seo/ai-crawler-log-file-analysis-citation-optimization/) by counting 429s per user agent. If you are behind Cloudflare, also check the [AI Crawl Control panel](https://organikpi.com/blog/technical-seo/cloudflare-ai-crawl-control-guide/), since Cloudflare adds its own bot-management layer with its own defaults on top of whatever your origin does. ## What actually helps, in order **1. Test first, then open a ticket with the evidence.** Hostinger’s own article says the limits cannot be disabled per site, but a ticket with measured output tells you whether your specific server applies the rule and whether anything has changed. The article also says the company is “currently refining these server-level protections”, so the answer may improve over time. Hosts respond better to a reproducible curl transcript than to a vague complaint about AI visibility. **2. Cache warming, the workaround that worked for us.** Since cached pages serve 200 to everyone, the fix is to keep everything cached. A small script walks the sitemap and fetches every URL on a schedule; ours warms 288 URLs every three hours from a cron job, plus a re-warm pass after every publish. After deploying it, warmed pages answered 200 to GPTBot consistently. Gaps remain: URLs with query strings and uncached images still draw 429s, because the warmer never touches them. **3. Move the decision into infrastructure you control.** On a VPS or dedicated server there is no host-level user-agent rule sitting above your config. A proxy or CDN layer you configure yourself can also serve crawlers from its own cache before requests ever reach the origin. Both options cost money and attention, so weigh them against what AI training inclusion is actually worth to your business. For most small sites, the warmer covers the bulk of the exposure. Whichever route you take, watch the outcome rather than the plumbing. Track whether AI engines mention and cite your brand with a [DIY visibility audit](https://organikpi.com/blog/seo-strategy/diy-ai-brand-visibility-audit/), a set of [AI visibility metrics](https://organikpi.com/blog/seo-strategy/ai-brand-visibility-tracking-metrics/) you check monthly, or one of the [dedicated tracking tools](https://organikpi.com/blog/geo-ai-search/best-ai-visibility-tools/). ## The limits of this test This is one host, one shared server, one measurement day. The matrix above describes Hostinger shared hosting behind LiteSpeed as measured on 1 September 2026, and the vendor says the protections are being refined, so the specific statuses may change. The transferable finding is the shape of the problem: a robots.txt Allow does not bind any layer above your CMS, hosts and CDNs make crawler decisions at the infrastructure level, and no SEO tool currently surfaces them. The only way to know what your server tells an AI crawler is to ask it as one. ## Frequently Asked Questions ### Why is GPTBot getting 429 errors on my site? Some hosts apply server-level rate limits to AI crawler traffic. On the Hostinger shared server we tested, any request with a GPTBot user agent receives a 429 whenever the page is not in the LiteSpeed cache. The rule sits above WordPress, so nothing in your CMS or SEO tooling reports it. ### Does a 429 to GPTBot hurt ChatGPT search visibility? Not directly on the server we tested. GPTBot collects training data, while OAI-SearchBot decides whether a site appears in ChatGPT search answers, and OAI-SearchBot passed every test. The block affects training inclusion rather than search visibility. ### How do I check if my host blocks AI crawlers? Request the same uncached URL twice with curl, once with a GPTBot user agent and once with a normal browser user agent. If the bot gets a 429 and the browser gets a 200, a layer above your CMS is filtering by user agent. Confirm the scale by counting 429s per user agent in your server access logs. ### Can Hostinger disable the AI crawler rate limit for one site? Hostinger's support documentation says the limits are applied at the server infrastructure level and cannot be disabled for individual websites or hosting plans. Cache warming is the practical workaround, since cached pages serve 200 to every crawler. ### Which AI crawlers were blocked in the test? Two of nine user agents: GPTBot and meta-externalagent, both training crawlers, and only on cache misses. OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Bingbot, and Googlebot returned 200 in every cache state.