How Google Crawls Paginated Content and Infinite Scrolling

How does Google crawl long discussion threads? Is paginated content a good option? Can Google crawl infinite scrolling? Get insights from Google's John Mueller
Marie Aquino
February 22, 2022

In the Google SEO Office Hours last February 18, a user asked about paginated content and how Google crawls long discussion threads. For a long discussion thread that is split over multiple pages, for new comments posted, would it be better to add the comment at the end of the thread which could be around page 4 or beyond? Can Google crawl through all pages and get to the last pages, where recent comments have been added?

John Mueller responded that it would be up to the website owner which part of the comments would be prioritized. If something is on page four, then they would have to crawl pages one, two, and three first to be able to find the new content on page four. This would mean that it’s farther away from the main part of the website and from Google’s point of view, they would not give the content much weight. They would also probably not re-crawl the page as often as the added content is on the last pages and they have to go through a couple pages that could remain unchanged.

Whereas, if the newest comment should be the most visible one, then maybe it makes sense to reverse the order and show them differently – with the newest comments on the first page, rather than the last pages. If the newest comments are on the main page, it would be a lot easier for Google to re-crawl it more often, and it gives the content more weight in the search results.

Generally, it would be up to the website owner to decide on how to balance it.

The user then went on to ask if Google is advanced enough to handle infinite scrolling.

Mueller responded that Google renders a page at a fairly high viewpoint. They would render the page to see what the page would show and it usually would trigger some amount of infinite scrolling with the JavaScript method being used to trigger infinite scrolling. Whichever ends up being loaded there, that would be what Google would be able to crawl.

It really depends on how infinite scroll is being triggered and implemented, it could be that only two or three pages are being loaded and that’s what will be crawled and not everything.

He then went on to recommend using the inspection tool to see what Google would actually pick up on – if just a couple of pages or the whole content in infinite scroll.

Do you have discussion threads on your site? Do you have a forum site? Which do you think is a better method to implement, paginated content with new comments at the top, paginated content with new comments at the last pages, or perhaps infinite scrolling? Looks like all of them have their disadvantages and advantages and a balance between users and search is needed. Let us know what you think. Tweet and tag us at @seointel1

Check out the SEO Office hours episode here: