How AI Sees Your Page: Content the Crawler Can't Find
A product page has a detailed description in one tab, a spec table in another and two hundred reviews below – and when someone asks an AI assistant about the product, the answer is based on two sentences from the meta description. Intuition says display: none is to blame. That's almost never true: crawlers read HTML, they don't look at the rendered page, so hiding something with styles doesn't bother them. What bothers them is content that isn't in the HTML at all – rendered conditionally, loaded on scroll, inserted by CSS or baked into an image. I show exactly where it gets lost and how to check it with one command.