Brett Tabke drops the nuclear bomb of banning all spiders from WebmasterWorld. He explains here that heavy rogue spidering is the reason behind the move. Members worry in the thread that as pages drop out of search engines, it will become difficult to impossible to find anything at WebmasterWorld, which self-admittedly lacks good site search.
Brett figures he's got 60 days until pages drop from places like Google to get an alternative search solution in place. That seems optimistic to me. WebmasterWorld is a prominent site and should get getting revisited on a sub-daily basis. If search engines are hitting that robots.txt ban repeatedly, they ought to be dropping those pages in short order, or they aren't very good search engines. I mean, can you imagine the irony of Google and Yahoo getting pilloried on WebmasterWorld for taking so long to drop pages after they were told to do so after the ban was put into place?
A separate issue is the potential loss of search traffic. We have had the odd site from time-to-time declare that it might ban Google or Microsoft because of opposition to those companies, and we've certainly had companies ban all spiders for other reasons. But in one bold move, WebmasterWorld suddenly is about to become a big giant test case about what happens to a site if it cuts itself off from the oxygen of search results -- an incredible irony when so many come to the site looking specifically on how to gain more search traffic.
Realistically, any established site should be able to ride out having no search traffic at all. WebmasterWorld has plenty of people who will seek it out directly, plus referral links from other sites will keep traffic going and perhaps even growing. But search has been estimated to drive anywhere between 7 to 13 percent of new visitors to a web site, visitors who after they arrive continue to come back. I wouldn't want to roll the dice against losing them.
It'll be interesting to see if WebmasterWorld really sticks with this ban or seeks other ways of getting its content into the major search engine without spidering, such as via Google Base or Yahoo's paid inclusion programs, for example.
Source : http://blog.searchenginewatch.com/blog/051121-105154
No comments:
Post a Comment