Yahoo! have been busy updating their search results index and search engine spider recently, with changes to their ranking algorithm and crawler frequency. This update is especially significant for webmasters as it will change how the Yahoo! spider crawls websites.
A driving force behind the update has been feedback received by Yahoo! from webmasters. Website owners were concerned about the large amount of IP addresses that the Yahoo! search spider used to crawl each site:
“Why is it that Yahoo! uses a different IP for every page request while spidering?
A lot of sites, such as forums, have a “guests” or “online users” count listed on their page. When Yahoo! uses 146 different IP addresses to spider 146 pages, it artificially inflates the online users count”
This suggestion received a huge amount of support from users prompting Yahoo! to make a change. From the Yahoo! search blog:
“We’re reducing our crawling machines by a significant amount with this update, so you might have observed a significant reduction in crawl load from us along with fewer machine IP addresses hitting you over the last few days.
The new crawler continues to improve comprehensiveness and freshness of coverage while also enhancing crawl efficiency, which reduces spurious load on websites.”
The new and improved search spider also features an updated search results index, with the resulting rank changes now appearing in search results. If your website has dropped its ranking with this latest update, try our new and improved Y-Boost service which improves your Yahoo! ranking with 5 guaranteed directory listings each month.