So with mobilegeddon raging around us we need a good way to bulk check all of our websites for mobile friendliness. A perfect job for Excel and SeoTools!

Google has released a tool that help test whether a website is considered "mobile friendly" or not. But only for one website at a time. What if we need to bulk check lots of websites to spot the ones we need to fix?

In this post I'll show you how we can create a SeoTools Scraper that check this for us in Excel.

The Google tool internally uses a JSON REST API whose result can look like this:

Json structure

We're after value with the JSON path $.ruleGroups.USABILITY.pass.

To build a new scraper, create a file myscrapers.xml under /scrapers/ in your SeoTools installation path with the following contents:

<?xml version="1.0" encoding="utf-8" ?>
  <ValueScraper Identifier="Google.MobileFriendly" Title="Google.MobileFriendly" Category="Onpage" SpiderScraper="true">
      <TextBoxParameter Identifier="Url" Title="Url" DefaultValue="" Required="true"/>
    <Fetch Url="{Url.UrlProperty.Absolute}">
        <IntervalBetweenRequests RandomFrom="1000" RandomTo="1000" IfSame="Host"/>
      <JsonPathMatcher Expr="$.ruleGroups.USABILITY.pass"/>

I've added 1000ms interval between requests to not piss Google off (so they won't block my IP).

If you have Excel open you need to restart Excel. We now have a new scraper:

Taskpane scraper in Excel

Since the Google API is quite slow you will need need to wait a while for the results. Excel might seem to freeze, but that's just SeoTools working in the background.

Update: As of SeoTools 4.3.5 the scraper created in this post is included in the release.

Do you have other ideas for scrapers? Let me know in the comments!