TL;DR
- Our host answers GPTBot with 429 on every uncached page.
- Cached pages serve 200, so only fresh and low-traffic URLs suffer.
- OAI-SearchBot, Googlebot, and Bingbot all pass, so dashboards stay green.
- Three curl commands detect the block in five minutes.
- Scheduled cache warming restored 200s for us.
Our robots.txt allows every AI crawler. Our web host answers GPTBot with HTTP 429 anyway, on any page that is not already sitting in cache. Search Console showed nothing, Bing Webmaster Tools showed nothing, and the block fires before WordPress runs. We measured the full pattern on 1 September 2026, and the same three curl commands will tell you in five minutes whether your host does this to your site.
What we measured
organikpi.com runs on a shared Hostinger plan behind a LiteSpeed cache. Requests with a GPTBot user agent were drawing 429 responses while every SEO dashboard stayed green. To pin down the rule, we sent requests with nine different user agents to the same blog post URL, once served from cache and once with a cache-busting query string. Two agents were blocked. Seven passed.
| User agent | Cached page | Uncached page |
|---|---|---|
| GPTBot/1.4 (OpenAI, training) | 200 | 429 |
| meta-externalagent/1.1 (Meta, training) | 200 | 429 |
| OAI-SearchBot/1.4 (ChatGPT search) | 200 | 200 |
| ChatGPT-User/1.0 (user-triggered fetches) | 200 | 200 |
| ClaudeBot/1.0 (Anthropic) | 200 | 200 |
| PerplexityBot/1.0 | 200 | 200 |
| Bingbot/2.0 | 200 | 200 |
| Googlebot/2.1 | 200 | 200 |
| Chrome browser (control) | 200 | 200 |
The pattern held on every retry. Three consecutive GPTBot requests to fresh uncached URLs drew three 429s, each answered in roughly 0.26 seconds. Those 429 responses carry no x-litespeed-cache header at all, while normal responses are stamped hit or miss. The rejection happens above the cache and above WordPress.
This is documented behavior, not a misconfiguration on our end. Hostinger’s support article on server-level rate limits states: “Hostinger has implemented server-level rate limits for traffic originating from specific network ranges, such as Meta, AWS, and Microsoft.” The same article notes the limits “cannot be disabled for individual websites or hosting plans.”
The rule matches the user-agent string
Our test requests came from a residential connection, nowhere near OpenAI’s published GPTBot IP ranges. The string GPTBot in the user agent was enough to draw the 429. Hostinger’s article frames the limits around network ranges; on our server, the user agent alone triggers them. That detail is useful, because it means anyone can reproduce the block from a laptop.
The cache decides who gets blocked
A cache hit serves 200 to every crawler. A miss serves 429 to the two blocked agents. That inverts the usual risk profile: your busiest pages are almost always cached and safe, while the pages most likely to 429 are the ones you just published, the long tail nobody visits, and any URL with a query string. A new post is at its most invisible to training crawlers in the hours right after publish, which is exactly when your sitemap is telling crawlers to come fetch it.
The selectivity is why nothing looks broken
Both blocked agents are training crawlers. OpenAI’s crawler documentation says GPTBot “is used to crawl content that may be used in training our generative AI foundation models”, while OAI-SearchBot decides whether a site appears in ChatGPT search answers. OAI-SearchBot passed every test we ran. So did Googlebot, Bingbot, ClaudeBot, and PerplexityBot. Your rankings hold, your ChatGPT search visibility holds, and no search console has a reason to warn you. The blocked crawls are precisely the ones that never appear in a dashboard.
Why a 429 to a training crawler matters
HTTP 429 means Too Many Requests. RFC 6585 defines it as the server telling the client it has sent too many requests in a given amount of time, and well-behaved crawlers respond by backing off and retrying later. A page that answers 429 on every uncached fetch is a page a polite crawler learns to fetch less.
Whether that costs you anything depends on a choice you have hopefully already made in your robots.txt AI crawler policy. If you disallow GPTBot deliberately, as many publishers negotiating content licensing do, the server rule costs you nothing. If you allow it because you want your content and brand represented in the models that answer buyer questions, the core bet behind generative engine optimization, then the server is quietly overriding your stated policy. You opted in. The infrastructure opted you back out.
Hostinger’s article lists the collateral beyond AI training: link previews on Facebook, Instagram, and WhatsApp can fail to render, automations in Zapier, Make, and n8n can error out, HubSpot and Salesforce syncs can fail, and uptime monitors can fire false downtime alerts. To be fair to the host, the stated purpose is keeping sites responsive for human visitors during crawler spikes, and AI crawler volume is a real cost on shared infrastructure. The problem is the silence, since none of this is visible to the site owner without testing for it.
The five-minute test for your own site
Three curl commands. Run them from any machine.
# 1. A popular page, GPTBot user agent. Expect 200.
curl -s -o /dev/null -w "%{http_code}\n" \
-A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
"https://yourdomain.com/"
# 2. The same page with a cache-busting query. A 429 here is the tell.
curl -s -o /dev/null -w "%{http_code}\n" \
-A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot" \
"https://yourdomain.com/?t=$RANDOM"
# 3. Control: another cache-busted URL, normal browser user agent. Expect 200.
curl -s -o /dev/null -w "%{http_code}\n" \
-A "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36" \
"https://yourdomain.com/?t=$RANDOM"Read the results as a pair. If the bot user agent gets 429 and the browser user agent gets 200 on the same uncached URL, something above your CMS is filtering by user agent. Add a header dump (curl -sI) to see your cache layer’s hit or miss stamp; on LiteSpeed that is x-litespeed-cache, on Cloudflare it is cf-cache-status. Then confirm the scale in your server access logs by counting 429s per user agent. If you are behind Cloudflare, also check the AI Crawl Control panel, since Cloudflare adds its own bot-management layer with its own defaults on top of whatever your origin does.
What actually helps, in order
1. Test first, then open a ticket with the evidence. Hostinger’s own article says the limits cannot be disabled per site, but a ticket with measured output tells you whether your specific server applies the rule and whether anything has changed. The article also says the company is “currently refining these server-level protections”, so the answer may improve over time. Hosts respond better to a reproducible curl transcript than to a vague complaint about AI visibility.
2. Cache warming, the workaround that worked for us. Since cached pages serve 200 to everyone, the fix is to keep everything cached. A small script walks the sitemap and fetches every URL on a schedule; ours warms 288 URLs every three hours from a cron job, plus a re-warm pass after every publish. After deploying it, warmed pages answered 200 to GPTBot consistently. Gaps remain: URLs with query strings and uncached images still draw 429s, because the warmer never touches them.
3. Move the decision into infrastructure you control. On a VPS or dedicated server there is no host-level user-agent rule sitting above your config. A proxy or CDN layer you configure yourself can also serve crawlers from its own cache before requests ever reach the origin. Both options cost money and attention, so weigh them against what AI training inclusion is actually worth to your business. For most small sites, the warmer covers the bulk of the exposure.
Whichever route you take, watch the outcome rather than the plumbing. Track whether AI engines mention and cite your brand with a DIY visibility audit, a set of AI visibility metrics you check monthly, or one of the dedicated tracking tools.
The limits of this test
This is one host, one shared server, one measurement day. The matrix above describes Hostinger shared hosting behind LiteSpeed as measured on 1 September 2026, and the vendor says the protections are being refined, so the specific statuses may change. The transferable finding is the shape of the problem: a robots.txt Allow does not bind any layer above your CMS, hosts and CDNs make crawler decisions at the infrastructure level, and no SEO tool currently surfaces them. The only way to know what your server tells an AI crawler is to ask it as one.