UPDATE 2015-02-27 Googler Doantam Pham replied to my discussion in the Google Webmaster Forums. As I suspected, the images were the cause of the disparity in the test results between the two sites. The problem remains, though. While I can get my site to pass the test, the tool does not mention the problem with the images. Once the tool was able to crawl the CSS & Javascript files (after I lifted the robots.txt block) the problem vanished. But other people may not be so lucky. I hope Google updates the tool to indicate when image sizes are a problem.
Today Google announced that effective April 21, 2015 their automated assessment of your site’s mobile-friendly usability will directly impact its mobile search results rankings. As a frequent mobile user I would normally welcome this news with open arms. The problem, however, is that Google’s mobile usability tool just does not work consistently enough to provide proper guidance on how sites should improve their usability.
Here are three screen captures that illustrate just one of the problems I have found with the mobile usability tool. I have two Websites, middle-earth.xenite.org and SEO Theory, which use the same WordPress theme and the same basic “robots.txt” file.
In this first screen capture we see that the Middle-earth blog renders very nicely on an Android phone using the Chrome browser. It would be tough for anyone to say that this is not a mobile-friendly theme because it’s a standard WordPress mobile-friendly set up. The menu expands for mobile users as Google suggests. I didn’t tweak anything in the theme to do that.
Both Websites use the Jetpack plugin to control mobile display. The SEO Theory blog does not use featured images. The Middle-earth blog DOES use featured images. I cleared my smart phone browser history and loaded the Middle-earth blog. The page loaded and completely rendered in less than 2 seconds. That is, I could scroll to the bottom of the page and see the four featured images within 2 seconds after clicking on the URL in the browser which (supposedly) did not have the site cached.
Having done what I can to prevent my own browser from clouding the results of the test, I decided to look at another Website (SEO Theory) using the tool. I took these screen captures as I was working my way through the test to ensure that I have a proper record of what Google is reporting today (Thursday, February 26, 2015).
Here is a capture of the problems that Google found with the site when I used their “new” Mobile Usability Test tool. The tool thinks the content is wider than the screen, which is not true. I cannot wiggle the page back and forth because it is fully rendered on the smart phone screen.
The tool thinks the navigation links are also too close together. That is also not true. However, notice that they say there are resources I have blocked via Javascript which may be causing the problem. These are standard WordPress scripts located in the wp-content folder. Now, I do block the wp-content folder from being crawled because, frankly, search engines expose far too many plugin installations by indexing these files. Google wants to help defend your site against hackers but it is one of the hackers’ most useful resources for finding vulnerable sites.
Still, are the wp-content files REALLY that important to the mobile usability tool? I submitted SEO Theory to the tool just to see what would happen.
Here is a screen capture of the Mobile Usability Tool’s assessment of SEO Theory. As you can see, the tool loves this site, which does not use featured images. And this site ALSO blocks the wp-content directory for crawlers. In both usability tests you can see where the tool is rendering the page without the benefit of the wp-content folder’s scripts.
Jetpack gives you the ability to disable the featured images, so I tried that. The site still failed the mobile usability test. And the tool still only provides limited feedback on why it doesn’t think the site is mobile-friendly. But it did detect that I am using WordPress, so I clicked through to their tips for WordPress users and found the article generally unhelpful because I’m already doing all these things (and more). The article even specifically mentions Jetpack as a good resource for WordPress users to install on their sites.
So what’s the problem, Google? Why is one site NOT mobile-friendly when the other is, and both use the same themes and same settings on Jetpack’s very simple mobile usability dashboard?
This is a HUGE problem for millions of Website owners because:
- Most of them are completely unaware of these issues
- Most of them have no idea of how to make a site mobile-friendly
- Most of them will never know why suddenly people have stopped visiting their blogs
The potential upside of this algorithmic update is that IT COULD BE BIGGER THAN THE 2011 GOOGLE PANDA ALGORITHM.
The potential downside of this algorithmic update is that IT COULD BE BIGGER THAN THE 2011 GOOGLE PANDA ALGORITHM.
I am already expecting to lose traffic to the Middle-earth blog because the last “Hobbit” movie has already left the theaters. The traffic is declining because people are moving on with their lives. But there are other Websites out there where this kind of nonsensical analysis from Google will have severe impacts.
This is a situation where Google needs to explain exactly why a Website is failing these tests because given the information they have provided, there is absolutely nothing I can do to make the blog pass the test. I’m already doing everything they recommend.
Your move, Googlers.
1. It is important that Google be able to crawl your CSS files (at a minimum) to understand when things become responsive through media queries and other declarations.
2. No index your plugins folder, but allow it to be followed. I’d leave the theme folder alone due to the first point.
3. It’s more of a pain to maintain, but see how many changes you can make through theme functions or CSS instead of plugins. As an example, setting the class around the featured image background to be fluid instead of fixed width in your stylesheet would eliminate the need for it to happen through Javascript or a plugin. width: 100% height:auto should keep everything in proportion, but scale up or down in a way that’s viewed as responsive.
That’s not just for header or featured images, but post images as well.
I appreciate the suggestions but since both sites are blocking crawlers from the CSS files it would seem that Google’s Mobile Usability tool does not need them to make its assessment.
But the greater issue here is that Google cannot possibly reach all the people who need to update their Websites. As of this comment, WordPress 4.1 has been downloaded more than 21 million times since it was released.
Those millions of Websites are at the mercy of Google’s flunky tool (and algorithm). That is the problem we must address here.
Michael,
You already know this but worth reiterating, security through obscurity isn’t security at all. If you’re worried about spammers or hackers finding your block, they are going to find it whether or not your wp-content folder is crawlable or not. Granted having it open might make it a little easier but they are going to find it anyway, at which point your site is going to be attacked – so the real solution here is to harden your WordPress installation not try and avoid getting spammed/hacked via a robots.txt half solution.
I think it’d be a useful exercise to work out what it takes at a dead minimum to get our blog using its current theme to pass the mobile usability test. Since you are intent on trying to keep some elements of your site private or out of Google, my first port of call would be to adjust your robots.txt to allow Googlebot to reach the relevant resources and to include HTTP meta noindex response headers via your .htaccess file for the directories or file types that you’d prefer were genuinely out of the Google index.
Seems like it’d be the best of both worlds, you’re site is mobile friendly and your themes/plugins aren’t disclosed via Google search. I appreciate that isn’t doesn’t address the issue that the testing tool being funky but if it works, maybe you could contribute these changes back to the WordPress community as generally good practice WordPress configurations to help reduce your blogs attack vectors being so easily discovered via search engines.
Al.
Alistair,
Thank you for your suggestion. This server is already hardened and other sites that I work on for friends, family, or clients also have anti-hacking protections. However, even if you can prevent the hackers from getting in the botnets can still bring down a Website so I prefer to keep the wp-content directory out of the search results.
But that still does not address the problem with what Google is doing. Their tool is broken, broken badly, and it’s going to be reporting a lot of false-positives.
While I can change my security strategies and update the sites that I manage between now and April 21 (if that proves to be what must be done) I still cannot do anything for the millions of other Websites that Google is putting into the wastebin. This is not about “how great my content is”. This is about what Google is threatening to do to the larger Web.
Aggressive marketers who have developed mobile-friendly sites will have a field day. This will be like taking candy from a baby because when the Google Mobile Usability tool FALSELY tells you that your site is not mobile friendly, and it cannot help you get out of the penalty box, you’re dead. I know how to play with Website code and make significant changes. The average mom-and-pop or college kid does not.
Maybe that will be great for the SEO industry. It will be the dawn of a new age for people who are now better equipped to provide technical SEO services. But that also means that the scam artists and egregious marketing tool vendors will have a whole new horde of victims to prey upon. This promises to be a virtual bloodbath. Google really needs to step up to the plate and fix this issue in a way that honestly helps Webmasters.
Reminds me of Google changing their Adwords terms of service in October and permanently banning thousands of Adwords Accounts that were and always had been in good standing — in error.
At least they can roll this back should they choose… They wouldn’t with the adwords accounts.
Hi there,
We have a similar problem where we have m. dedicated mobile pages that are properly marked up (using Google’s recommendations) with rel=”alternate” tags and canonicalisation to the desktop version of the page. However, for some sites the desktop versions of pages are identified when the mobile versions of the pages exist and are correctly linked through.
I tried contacting someone at Google and left feedback via the help function in GWT. However, nothing happens.
Cheers, Helmut
Helmut,
Sorry for not replying sooner. I am curious to know what the Mobile Friendly test tool tells you about these pages. Even if the error message is vague, it might help us provide better feedback to Google if you can share an example. Then again, maybe it will tell you something useful.
This information doesn’t surprise me, but I appreciate your detailed analysis. What do you think Google needs to do to restore users’ confidence in their system?
Articles like this prove even more why it’s not enough to design a great website and move on to something else.
Instead, a website needs to be constantly updated, improved, modified and tested if it is going to reach its full potential.
I hope that Google will strongly reconsider these changes if enough blog and website owners contact them. There is power in numbers, and things aren’t going to get better if people don’t make their voices heard.
Darrin, what I would like to see Google do is provide much more definitive information about what is triggering a failure in the Mobile Friendly test tool. Messages like “Content too wide” are not helpful. Also, while the need to open up the CSS and Javascript files was obvious to me from the start, if the problem is with an image (as in my case) or with an embedded video (as some people have reported) then the failure is not necessarily something that can be resolved.
Fortunately Googler Gary Illyes said at SMX West 2015 this week that if your page has only 1 error (especially regarding an embedded video) that may not be sufficient to hurt the page. He also said they were working to resolve the embedded YouTube video problem.