In the Google SEO Office Hours February 4 episode, a user asked about steps that could be done in order to get a site crawled and indexed. The user has noticed that the pages are showing up as either crawled but not indexed, or discovered but not indexed. What does these two particular crawl categories mean?
John Mueller responded that crawled but not indexed and discovered but not indexed does not mean anything in particular and that it is an early assumption to say that Google looked at the pages but decided not to index them.
Most of the time, when pages are crawled, it does not necessarily mean that they will get indexed automatically. The two categories of not indexed are similar. It can be tricky because Google does not index everything on the internet and that can happen on some pages of your site.
With regards to steps that can be done to help with getting pages indexed, Mueller recommended making sure that it is easy for Google to recognize the important content on a site. This can sometimes mean making less content and instead, making better, higher quality content. More of quality over quantity.
Another is internal linking. Internal linking is very important for Google to be able to understand what the important pages of a website are. For example, pages that are linked from the homepage are usually a sign that the pages are important and so Google should care about them more, also.
External links are also another factor. They see these links as other people think that these pages are important and maybe Google should see them as important, also.
Sitemaps and RSS feeds, from a technical point of view, also help Google understand better if the pages are new, or if they have had some recent changes, that they should be crawled again to see what has changed.
All these factors come together and it’s rarely that there is one trick that is missing to get pages indexed.
Another follow up question asked by the user is if these unindexed pages linking to other pages on the site can actually pass on juice from external links going to it, and if it can affect the rank of the site.
Mueller’s response was, linking to these unindexed pages can increase the chances of getting them crawled and indexed, but it also depends on the type of external links being used to link to the unindexed pages.
Links from social media usually have nofollow attached so they cannot really forward any signals there. Problematic links might also be ignored. If the link sources are good sources, they will probably go and crawl, and more likely index the page.
With regards to passing on juice though, if the page is unindexed, then it will not forward any value from any external links to other pages on the site being linked to in the unindexed page. If they decide not to index the page, then there is no destination for those links, and they could not do anything with those links for the rest of the website.
Getting pages indexed have been quite difficult and has been a complain for a lot of SEOs. It looks like the quality of the content is really important, plus, the overall quality of the site is also a factor when it comes to Google deciding whether to index a page or not.
Quality of pages is a higher priority compared to the number of pages and the hierarchy of pages seems to also be a factor, in the sense that for important pages, they should be linked from the homepage, in order for Google to determine that they are important pages that should be crawled. Aside from that, internal links are a must, coupled with external links, sitemaps, and rss feeds.
As Mueller mentioned, getting the pages indexed relies on a variety of factors coming together. Having issues with getting pages indexed? Maybe it’s time to look at your whole site, start pruning pages that are not important, and start building up important pages, to start.
Build up a process on how to get pages indexed, using the tips provided by Google, and hopefully, it would help you get more pages indexed, and in turn, help improve the rank of your site.
Watch the SEO Office Hours episode here: