How long AI search engines take to pick up new content
Last updated
“How long until AI picks up my new page?” is three questions. How long until a crawler fetches it, how long until a search index holds it, and how long until an answer uses it. The companies give timings for the first two. Nobody gives one for the third.
What the companies say
| What you are waiting for | Who says | In their words |
|---|---|---|
| A robots.txt change to reach ChatGPT search | OpenAI | “it can take ~24 hours from a site’s robots.txt update for our systems to adjust” |
| A robots.txt change to reach Perplexity | Perplexity | “it may take up to 24 hours for our systems to reflect changes” |
| Google to crawl a new or changed page | “Crawling can take anywhere from a few days to a few weeks.” | |
| Asking Google again | “requesting a recrawl multiple times for the same URL won’t get it crawled any faster” | |
| Whether the page gets in at all | “Requesting a crawl does not guarantee that inclusion in search results will happen instantly or even at all.” | |
| A page sent through IndexNow | IndexNow | A 200 answer “only indicates that the search engine has received your URL” |
Sources: OpenAI’s crawlers, Perplexity’s crawlers, Ask Google to recrawl your URLs and the IndexNow documentation.
Gemini searches with Google Search, and Google’s AI answers use pages from its search index, so Google’s timings are the ones that matter for both. None of these companies says how long it takes before a page that is in the index gets used in an answer.
What to do while you wait
- Submit your sitemap in Google Search Console and in Bing Webmaster Tools. Ask for indexing once for each page that matters. Asking again does not help.
- Put the day each page really changed in the sitemap. It tells a search engine what is new.
- Use IndexNow if your host or site builder supports it, and read its answer as “received”, nothing more.
- Check with the search engine’s own tool. URL inspection says whether a page is indexed. Asking an assistant whether it knows your page does not.
- Leave weeks between two measurements, not days, and ask the same questions both times. How to measure.
What not to do
- Change a page’s date without changing the page. The date is a claim about the page.
- Resubmit every day. Google says it does not make crawling faster.
- Conclude from one quiet week that nothing works.
Wellread checks that your pages can be read, then asks ChatGPT and Gemini your buyers’ questions, so you can ask them again later and compare. What an audit is.