robotted resources (old, probably outdated)

Warning: This page is pretty old and was imported from another website. Take any content here with a grain of salt.

I see a bunch of posts about the robotted resources message that we’re sending out. I haven’t had time to go through & review them all (so include URLs if you can :)), but I’ll spend some time double-checking the reports tomorrow.

Looking back a lot of years, blocking CSS & JS is something that used to make sense when search engines weren’t that smart, and ended up indexing & ranking those files in search. For a long time now, that’s no longer the caseĀ  (happy to get reports if you do see it though), and nowadays it’s extremely helpful to be able to access CSS & JS files so that we understand what a page really looks like. Is it mobile-friendly? Is there content that we’d miss without JavaScript? Blocked CSS & JS files are some of the bigger problems that we run across in the meantime.

Search Console gives you tools to find blocked content across your site, to test individual pages to see if all embedded content is accessible, and to test your robots.txt files to find why a specific URL is blocked (and to submit an updated robots.txt so that we can crawl it ASAP). We’ve also documented this problem in various ways over time, for example in https://developers.google.com/webmasters/mobile-sites/mobile-seo/common-mistakes/blocked-resources (archive.org)

That said, we hear you that these messages aren’t as clear as they could be, and we’ll double-check your reports (here on G+, on Twiter, in the forums, etc) to see what we can do to improve that.

Original atĀ https://plus.google.com/+JohnMueller/posts/8iCsUmstaHy (archive.org)

Warning: This page is pretty old and was imported from another website. Take any content here with a grain of salt.

Comments / questions

There's currently no commenting functionality here. If you'd like to comment, please use Twitter and @me there. Thanks!

Tweet about this - and/or - search for latest comments / top comments

Related pages