Contact Us

Welcome to ineedhits Blog

Welcome to the ineedhits Search Engine Marketing blog, where we share the latest search engine and online marketing news, releases, industry trends and great DIY tips and advice.

Monday, April 10, 2006

Google’s Spider of Death

Posted by @ 10:24 pm

In one case, Google's spider 'killed' an entire website.
Do you really want Google’s spider of death to visit your website?

For most website owners, many of their search engine optimization efforts are aimed at getting Google’s spider to visit their website on a regular basis. It’s assumed that the more regularly the GoogleBot visits, the more up-to-date your search engine listings will be. And generally, regular visits by Google’s spider are a sign that your website is viewed quite highly by the search engine.

In a case of “be careful what you wish for”, a website owner experienced the full wraith of the GoogleBot recently. After engaging a web development firm to implement a content management system for their existing website, a month of painstaking development was swiftly undone when Google’s spider essentially killed the newly re-developed web site.

While the disastrous impact GoogleBot’s visit is alarming, it can’t be held totally to blame. It is fair warning for anyone with a poorly programmed site.

The spider is designed to follow each hyperlink on a page, including those with “Delete Page” in the title. The net result; GoogleBot systematically went through the site and deleted everything. Not all was lost, as the prudent back-up of the previous site was soon re-published to fill the void in the spider’s wake.

So if you don’t wish for GoogleBot to make a disappearing trick of your web pages, especially if you have a content management system controlling your website, then it might be timely to check your page links and ensure you’re not leaving your website open to the destructive power of Google’s spider.

Post To

Give your Google listing a boost.
7 day inclusion for only $19.95 – Money Back Guaranteed!

Rene LeMerle Rene is the marketing manager of - a global search engine marketing company. He also leads the marketing for - a web 2.0 style community for online and digital marketers. Rene has been in the industry since 1997 with much of that time spent helping businesses embrace the best of the internet and digital world.

View Rene LeMerle's profile

Discussion (3 - comments)

This is hilarious! A friend who worked at IBM in the good old days of room sized computers had a small placard from IBM on his desk, it only had one word on it: “Think”

This article doesn’t give any solutions for how to avoid this problem, so let me give a couple of very simple things to keep in mind:

1) Keep commands with disasterous consequences behind a login, or otherwise unspiderable place.

2) Use form buttons instead of links for thinks like edit, save, etc. Form buttons are generally not spiderable.

Using straight HTML links for content management editing might look nice, but they are an open invitation for this kind of disaster not only from Google, but any kind of web-whacking downloader software.

- Garnet, developer of Free Hive Wiki

By Garnet - April 11, 2006

Either this is a poorly written article, or it is claiming that Google’s spider can delete pages, or both. Sheer nonsense! No search engine spider can delete pages on your Web site — content management or not. Could a poorly written Web site cause Google to de-list a Web site, or substantially lower its listing position? Certainly. But no search engine has the power of deletion! This article smacks of a search engine urban myth.

By Randy Davis - April 11, 2006

uh, randy, i think you need to actually *read* the article above.

If a poorly written website has a link titled “Delete Page” that, when followed, deletes a page, google bot, or any spider for that matter, can surely follow the link causing the page to be deleted

By brien - June 26, 2006

Add Your Comments


Keep up to date with the latest from our blogs.

Subscribe to all blog posts

The Newsletter

  • New Posts
  • Popular
  • Comments


More in Search News (1609 of 1797 articles)

Well, by now you should all be well acquainted with Nancy Hackett, ineedhits' busy as a beaver marketing manager, from ...