I banned yahoo. you should too.

[Valkysas]

I just couldnt stand their damn search bots. for those that don't know, yahoo sends about 50 search crawlers to this site (and most others) on a daily basis to download EVERYTHING for inclusion in their search engine. and they've been ramping it up lately, which has slowed us down a lot. used to be they'd take a week to download everything, but instead this week they've been accessing everything on the site every day. every page of every topic and every use profile. we served out around 4GB of files to these bots yesterday. right before I did the ban, we had two pages of "who's online" dedicated to the damn things.

and if you're wondering how much bandwidth google's crawler uses, about 200MB a day. that's nothing. especially considering it doesnt come here every day.

If you have a site and want these stupid things gone too, ban the IP ranges 67.195. and 74.6. those are just the yahoo crawlers, no actual people will be banned. and don't do it through robots.txt, as yahoo ignores that.

In an age where yahoo no longer matters, I wouldnt think they'd do this kind of thing to webmasters, which only hurts yahoo more when people start banning them. people use google far more, so losing yahoo does not hurt us at all. people noticing "this search engine doesnt show me the content I'm looking for" kills yahoo.

performance should pick back up in a few hours.

[jvrlopez]

Re: I banned yahoo. you should too.

I'm gonna deny those ip ranges on my router's ACLs just to **** around.

[DYRE]

Re: I banned yahoo. you should too.

Wait.
You can't just stop them with robots.txt?

EDIT: Nevermind. I'm stupid and I don't read posts, apparently.

[Red Dragon]

Re: I banned yahoo. you should too.

In an age where yahoo no longer matters, I wouldnt think they'd do this kind of thing to webmasters, which only hurts yahoo more when people start banning them. people use google far more, so losing yahoo does not hurt us at all. people noticing "this search engine doesnt show me the content I'm looking for" kills yahoo.

Eh, Yahoo is still larger than Google. Not by much but they are.

Of course I know what you mean, more people use google search engine, but that might change as Microsoft is trying their best to buy it.

[Alzar]

Re: I banned yahoo. you should too.

Eh, Yahoo is still larger than Google. Not by much but they are.

Of course I know what you mean, more people use google search engine, but that might change as Microsoft is trying their best to buy it.

actually you're like very wrong by a lot

Search Market Share, US Market:
- Google 68.3%
- Yahoo 20%
- MSN 5.9%
- Ask 4.2%

Search Market Share, UK Market:
- Google 87% (google.com and google.co.uk)
- Yahoo 4%
- MSN 3.7%
- Ask 3%

May 2008
Source: http://tourismmarketing.wordpress.co...t-share-may08/

heaheha

[Goufunaki]

Re: I banned yahoo. you should too.

that's a lot of data for them to be going through. o_O

[Red Dragon]

Re: I banned yahoo. you should too.

Has it really changed that much in a year?

Geez.

EDIT: Wait a minute...

These are very powerful numbers, where nearly 70% of all US searches where powered by Google.

That's what I said.

[Alzar]

Re: I banned yahoo. you should too.

yeah, no one likes yahoo anymore

though i do use yahoo real estate and yahoo finance

[Karr Lord of Chaos]

Re: I banned yahoo. you should too.

yahoo drained all my resources the same month i did a large update. i was offline for the rest of the damn month because of them. they still piriodicly harrass my webspace though since i dont update much they just troll the forum bandwidth.