Anidb.info not respecting the HTTP RFCs ?

maxmillion · Post by **maxmillion** » Tue Aug 29, 2006 10:46 am

okay, I make it clear. I am not try to flamming or whatever it name...

here my problem
I use IE7 RC1 (and wish never use other browser than IE7 and opera), in Windows Server 2003 SP1.

I got a problem when I open any anidb.info page.
my IE7 will download .pl file instead of view it in IE7 window

I got some "official" answer from EricLaw from MSFT in this page :
http://blogs.msdn.com/ie/archive/2006/08/24/714977.aspx

I will paste his anwer :

Monday, August 28, 2006 11:00 PM by EricLaw [MSFT]
@Max: Do you have the Internet Explorer Enhanced Security Configuration package installed?

The Anidb.info server doesn't appear to be respecting the HTTP RFCs. It's returning GZIP'd content in response to all queries, including HTTP1.0 requests with no Accept-Encoding header.

I don't even care what is HTTP RFC. I just want my IE7 can open anidb.info with my IE7.. that's all..
please give me some explaination

note : please don't give me answer to use browser other than IE7 or Opera 9

maxmillion · Post by **maxmillion** » Tue Aug 29, 2006 10:52 am

never mind, I already find the answer

http://wiki.anidb.info/w/Can't access the website / browser wants to save animedb.pl

sory for bother.
you can lock this thread

Website · Post by **Rar** » Tue Aug 29, 2006 11:25 am

We could serve 406s to be nice to stupid clients (see also the Dildo thread in bug reports), but there is no requirement to do so:

RFC 2616 wrote:Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.

You might like to tell the IE team to get with the freakin' times and read the 7 year old HTTP 1.1 rfc.

Rar

Mensab · Post by **Mensab** » Wed Aug 30, 2006 4:52 pm

maxmillion wrote:never mind, I already find the answer

http://wiki.anidb.info/w/Can't access the website / browser wants to save animedb.pl

sory for bother.
you can lock this thread

Since 21.05.2004 AniDB enforces gzip compression of all text content.
Why you did it?
and why only few months ago its start working?

so, if any browser dont support gzip/deflate compression it will show only crap...

Website · Post by **Rar** » Wed Aug 30, 2006 5:52 pm

Presuming that the stupid and broken browser we're talking about is still IE, it's down to the fact its behaviour varies according to configuration, route, and version - and that enforcement this side has generally been permissive. Currently it is not permissive. If it's not IE you're worried about, consider the fact that zlib is going to be on any PC you can find, and is a prerequisite for the display PNGs. Use a real browser.

Rar

Mensab · Post by **Mensab** » Wed Aug 30, 2006 6:10 pm

alright...
i mean ANY program, which can access to internet and browse, get, download etc...

The Anidb.info server doesn't appear to be respecting the HTTP RFCs. It's returning GZIP'd content in response to all queries, including HTTP1.0 requests with no Accept-Encoding header.

what should i do if i want write script to get some search results from anidb.info?
now i need to burn my mind with zlib?

Website · Post by **Rar** » Wed Aug 30, 2006 6:23 pm

>>> import urllib2, gzip, StringIO
>>> print gzip.GzipFile(fileobj=StringIO.StringIO(urllib2.urlopen("http://anidb.info/perl-bin/animedb.pl").read())).read(50)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"

One (slightly squishy) line of code. Note that screen scraping is Bad and Wrong, anidb provides a UDP interface for all your automated access needs.

Rar

Post by **exp** » Thu Aug 31, 2006 1:14 pm

Mensab wrote:Since 21.05.2004 AniDB enforces gzip compression of all text content.
Why you did it?
and why only few months ago its start working?

so, if any browser dont support gzip/deflate compression it will show only crap...

We did it to reduce the http traffic anidb generates and because IE _CAN_ handle gzip'ed content, but still tends to request the uncompressed pages.

On some apache update somewhere on the way the enforcement got lost and was only recently reenforced.

BYe!
EXP