Yahoo! recently announced the launch of a new faster, more efficient crawler. The new Yahoo! Slurp (as it’s known), is designed to navigate through the web quicker than its predecessor and in the process uses 25% less bandwidth. Its launch has been met by mixed reviews from website owners.
You might have seen far more activity from the Yahoo! Slurp in your website logs as Yahoo! has been running the old and new crawlers concurrently over the past few weeks. The extensive testing is designed to ensure that the new crawler lives up to its expectations.
The launch of the new crawler was announced via the Yahoo! Search Blog. The blog post provided by David Simpson and Priyank Garg states:
“In addition to crawling the Internet faster, our new crawler is more efficient at visiting websites. As a result, site owners will notice as much as a 25% reduction in the number of requests and bandwidth consumed by the crawler.
Now, with the new crawler in full production, we have turned off the old crawler and site owners should see a much lower crawl load without a loss in content coverage.”
This is good news for website owners, as each time the crawler visits your website it utilizes your bandwidth when indexing your web pages. For owners with small bandwidth website hosting, these savings will be appreciated.
That said, Yahoo! suggests that the new crawler will inevitably lead to some changes to rankings and reshuffling of pages within their results. So it is highly recommended that website owners check their listings in Yahoo! search.
Initial reviews about the new slurp have been mixed. While some website owners are happy with the drop in crawl load to their sites, others are not overly impressed with the changes to rankings and results evident from the new crawler. Give the new search results a whirl and share your thoughts on Yahoo!’s new crawler and results.