In the Google SEO Office Hours February 4 episode, a user has been encountering issues with drops in crawl rate for his site when they moved to a CDN.
After the move, they have noticed that their response time have gone up from 300 millisecond to a second. From a crawl rate of 2 Million requests a day, it has dropped down to just 80,000 daily request. Did the move to the CDN impact the crawl rate?
John Mueller provided his insights on the situation. According to Mueller, from a ranking point of view, it should not change anything.
When changing hosts, what will happen on Google’s side is that the crawl rate will move into a more conservative area first and will crawl a bit less because of the bigger change in hosting – it could be a move to a CDN, a move from a CDN, or changing from one CDN to another CDN.
Over time, it could be a couple of weeks, maybe a month or so, the crawl rate will increase again to see where it would settle down. A drop in crawl rate when moving to a CDN or a change of a CDN can be normal. It does not necessarily mean that there is a problem.
If they were crawling two million pages of the site before, it’s unlikely that there would be two million changes every day. It is not necessarily that they would miss new content on the site, they would just try to prioritize again and figure out which of the pages actually need a recrawl on a daily basis. Drops in crawl rate is not necessarily a cause for concern.
An issue that Mueller find concerning is the change in average response time. This is because crawl rate is based on the average response time and server errors. If the average response time goes up significantly, the crawl rate will lower down.
From Google’s point of view, the response time is critical for the amount of crawls that is done per day because they try to limit the number of active connections that they have on the server.
If the response time is high, then they will run into the limit fairly quickly, whereas if they can quickly crawl, then there would not be a problem with the number of concurrent connections.
Valuable information on drops in crawl rate when moving hosts, how drops in crawl rate are not a cause for concern, and how server response times have an effect on crawl rate, too.
If you’ve been monitoring your crawl rate and have seen drops, hopefully these insights would be able to provide you answers and some areas to check into to be able to determine if there is an issue on your server.
Generally, it is not a cause for concern and should not affect rank.
Check out the Google SEO Office Hours episode here:
Before you can receive free updates, link building strategies or SEO tips you need to confirm your email right now.
(It’s easy)
Just go to your inbox, open the confirmation email from the SIA, and click the link.
And that’s it!
PS: If you don’t see a confirmation email, please check your spam/junk or promotions folders. Sometimes the confirmation message ends up there by mistake.
Obtenez nos 7 études S.I.A. les plus controversées qui feront trembler la tête même les SEO les plus avancés d’incrédulité.
De plus, nous vous alerterons lorsque nous publierons de nouveaux tests au public.
Obtenga nuestros 7 estudios S.I.A. más controvertidos que harán que incluso el SEO más avanzado sacuda la cabeza con incredulidad.
Además, le avisaremos cuando publiquemos nuevas pruebas al público.
Get Our Most 7 Controversial S.I.A. Studies That Will Make Even the Most Advanced SEO Shake Their Head in Disbelief.
Plus we will alert you when we publish new tests to the public.
