Google Slows Web Crawler to Reduce Penalization of Blacked-Out Sites

  • Share
  • Read Later

Apparently Google’s protest of SOPA goes beyond a black Google Doodle. Its engineers have also slowed down the search engine’s web crawlers, meaning sites that are blacked out today in protest won’t suffer as much in Google’s rankings for their actions.

Pierre Far, webmaster trends analyst at Google U.K., explained it on his Google+ page:

Hello webmasters! We realize many webmasters are concerned about the medium-term effects of today’s blackout. As a precaution, the crawl team at Google has configured Googlebot to crawl at a much lower rate for today only so that the Google results of websites participating in the blackout are less likely to be affected.

Yes, the fact that Wikipedia’s ubiquitous, SEO-friendly entries now lead to black anti-SOPA pages won’t hurt the site much in the medium- or long-term. Far already sent out tips to webmasters on how to SEO-proof their sites — similar to how you’d prepare your site for a day offline for server maintenance — but it appears that even those who didn’t follow his advice will gain some measure of amnesty from Google’s spiders.

Of course, there’s no guarantee that a search engine such as Bing won’t penalize protesting sites (which include Mozilla, Craigslist, Boing Boing, Reddit and many others), meaning anyone who blacks out their site still risks being penalized in search, not to mention the revenue lost due to unclicked ads.

(MORE: Check out Techland’s SOPA coverage)

Disclosure: Time Inc. parent company Time Warner supports SOPA legislation.