https://rendercache.com/tags/ai-crawlers/ Recent content in AI Crawlers on RenderCache.com Hugo en-us Sun, 18 Oct 2026 00:00:00 +0000 https://rendercache.com/ai-crawlers-like-gptbot-and-claudebot-see-an-empty-shell-on-javascript-sites/ Sun, 18 Oct 2026 00:00:00 +0000 https://rendercache.com/ai-crawlers-like-gptbot-and-claudebot-see-an-empty-shell-on-javascript-sites/ <p>Take a developer docs site built with a JavaScript framework. Every page is a route in one app, and the content loads from Markdown files after the app starts. Search works. The pages look right. Then someone on the team asks an AI assistant about their own API and gets an answer that&rsquo;s a year out of date, or a shrug.</p> <p>One likely reason is in the server logs. GPTBot, ClaudeBot, PerplexityBot and the other AI crawlers have been fetching those pages for months. What they received was the app shell: a title, a script tag and an empty container. Most of these crawlers read the HTML they&rsquo;re given and don&rsquo;t run the JavaScript, so a page whose text only exists after rendering looks blank to them. Their capabilities differ and they change, but on a client-rendered site the safe assumption is that the words aren&rsquo;t reaching them.</p>