Screaming Frog Releases Version 16.6, URL Inspection API Integration

Screaming Frog Version 16.6: Romeo has now integrated the Search Console URL Inspection API to the tool, plus a number of smaller updates, security, and bug fixes.
Marie Aquino
February 4, 2022

Earlier this week, Google released the Search Console URL Inspection Tool API and looks like Screaming Frog has now integrated the new tool with its newly released update – Screaming Frog Version 16.6, codenamed “Romeo”.

The main feature for this update is the integration of the URL Inspection API that allows users to fetch current Google Index data that can be seen in Search Console.

With the intregration, you are able to connect the URL Inspection API in the SEO Spider tool and pull in data for up to 2000 URLS a day, per property, aside from the usual crawl data provided by Screaming Frog.

Upon connecting the URL Inspection API to your Screaming Frog account, the data will then be populated in the “Search Console” tab of the tool, along with the usual search analytics data provided in the tool.

New filters were also introduced by Screaming Frog, in order to help identify potential issues.

Here is a list of the new filters added and their meaning:

URL Is Not on Google – The URL is not indexed by Google and won’t appear in the search results. This filter can include non-indexable URLs (such as those that are ‘noindex’) as well as Indexable URLs that are able to be indexed. It’s a catch all filter for anything not on Google according to the API.

Indexable URL Not Indexed – Indexable URLs found in the crawl that are not indexed by Google and won’t appear in the search results. This can include URLs that are unknown to Google, or those that have been discovered but not indexed, and more.

URL is on Google, But Has Issues – The URL has been indexed and can appear in Google Search results, but there are some problems with mobile usability, AMP or Rich results that might mean it doesn’t appear in an optimal way.

User-Declared Canonical Not Selected – Google has chosen to index a different URL to the one declared by the user in the HTML. Canonicals are hints, and sometimes Google does a great job of this, other times it’s less than ideal.

Page Is Not Mobile Friendly – The page has issues on mobile devices.

AMP URL Is Invalid – The AMP has an error that will prevent it from being indexed.

Rich Result Invalid – The URL has an error with one or more rich result enhancements that will prevent the rich result from showing in the Google search results.

The integration of the API to the crawler should be able to provide better insights on the different pages of your site and help troubleshoot and find issues that needs to be addressed.

The data can also be exported via the Bulk export menu.

Aside from this API integration, the update also included a number of smaller updates, security, and bug fixes.

Check out the full details of the update here.