[FIXED] Hashing-Bug! & Hashing of x*9500KB-files

Bug reports for an official AniDB Client go here

Moderators: AniDB, AniDB API

Locked
wahaha
AniDB Staff
Posts: 1497
Joined: Sun Nov 17, 2002 3:33 pm

[FIXED] Hashing-Bug! & Hashing of x*9500KB-files

Post by wahaha » Mon Jan 12, 2004 10:49 am

It was news to me that edonkey/overnet<0.50 and emule use a slight modification of the described hashing-method to be compatible with a bug intoduced in early edonkey-clients (found it in this thread):
Files with a size that is an exact multiple of 9500KB are treated as if they had one more chunk when generating the hashset-md4 (9500KB -> 2 chunks, 19000KB -> 3 chunks and so forth)

As this seems to cause incompatibility, I looked at the hashes created by different clients and utilities:
~1 Chunk:

Code: Select all

"bug1-9500KB-1.avi", 9.727.999 bytes
| edonkey v61    : 431379dbd2cb7f3128f22527b391f5b6
| emule 0.41a.10 : 431379DBD2CB7F3128F22527B391F5B6
| emule+ v1 i    : 431379DBD2CB7F3128F22527B391F5B6
| ed/ov   v48.1  : 431379dbd2cb7f3128f22527b391f5b6
| overnet v50.1  : 431379dbd2cb7f3128f22527b391f5b6

| shareaza 1.8   : 431379dbd2cb7f3128f22527b391f5b6
+
| AoM 0.2.2.80   : 431379DBD2CB7F3128F22527B391F5B6
| AoM 0.3.2.160  : 01c0672f4bb6d7bbd9519c9e839dc6e4 (!)
| HashCalc       : 431379dbd2cb7f3128f22527b391f5b6
| ed2k_hash (0.3): 431379dbd2cb7f3128f22527b391f5b6
| ed2khash  (0.1): 431379dbd2cb7f3128f22527b391f5b6
| BitCollider    : 431379dbd2cb7f3128f22527b391f5b6

"bug1-9500KB.avi", 9.728.000 bytes
| edonkey v61    : d9597128599f247baad490949615714a
| emule 0.41a.10 : D9597128599F247BAAD490949615714A
| emule+ v1 i    : D9597128599F247BAAD490949615714A
| ed/ov   v48.1  : d9597128599f247baad490949615714a
| overnet v50.1  : 9697b5294681f8fcdf9b962d94991075
| shareaza 1.8   : 9697b5294681f8fcdf9b962d94991075
+
| AoM 0.2.2.80   : 9697B5294681F8FCDF9B962D94991075
| AoM 0.3.2.160  : 6d6155c880c27e302425b67c002bc778 (!)
| HashCalc       : 9697b5294681f8fcdf9b962d94991075
| ed2k_hash (0.3): (error)
| ed2khash  (0.1): d9597128599f247baad490949615714a
| BitCollider    : 9697b5294681f8fcdf9b962d94991075

"bug1-9500KB+1.avi", 9.728.001 bytes
| edonkey v61    : 715ec2ce751ab9309145640faed2b087
| emule 0.41a.10 : 715EC2CE751AB9309145640FAED2B087
| emule+ v1 i    : 715EC2CE751AB9309145640FAED2B087
| ed/ov   v48.1  : 715ec2ce751ab9309145640faed2b087
| overnet v50.1  : 715ec2ce751ab9309145640faed2b087
| shareaza 1.8   : 715ec2ce751ab9309145640faed2b087
+
| AoM 0.2.2.80   : 715EC2CE751AB9309145640FAED2B087
| AoM 0.3.2.160  : 715ec2ce751ab9309145640faed2b087
| HashCalc       : 715ec2ce751ab9309145640faed2b087
| ed2k_hash (0.3): 715ec2ce751ab9309145640faed2b087
| ed2khash  (0.1): 715ec2ce751ab9309145640faed2b087
| BitCollider    : 715ec2ce751ab9309145640faed2b087
~2 Chunks:

Code: Select all

"bug2-19000KB-1.avi", 19.455.999 bytes
| edonkey v61     : c5c7e280b4e9c35dc9df95f842654fb6
| emule 0.41a.10  : C5C7E280B4E9C35DC9DF95F842654FB6
| emule+ v1 i     : C5C7E280B4E9C35DC9DF95F842654FB6
| ed/ov   v48.1   : c5c7e280b4e9c35dc9df95f842654fb6
| overnet v50.1   : c5c7e280b4e9c35dc9df95f842654fb6
| shareaza 1.8    : c5c7e280b4e9c35dc9df95f842654fb6
+
| AoM 0.2.2.80    : C5C7E280B4E9C35DC9DF95F842654FB6
| AoM 0.3.2.160   : c5c7e280b4e9c35dc9df95f842654fb6
| HashCalc        : c5c7e280b4e9c35dc9df95f842654fb6
| ed2k_hash  (0.3): c5c7e280b4e9c35dc9df95f842654fb6
| ed2khash   (0.1): c5c7e280b4e9c35dc9df95f842654fb6
| BitCollider     : c5c7e280b4e9c35dc9df95f842654fb6

"bug2-19000KB.avi", 19.456.000 bytes
| edonkey v61     : 8c5c92e3f356c0d7813ab791e4e60236
| emule 0.41a.10  : 8C5C92E3F356C0D7813AB791E4E60236
| emule+ v1 i     : 8C5C92E3F356C0D7813AB791E4E60236
| ed/ov   v48.1   : 8c5c92e3f356c0d7813ab791e4e60236
| overnet v50.1   : f35f6333c234e7e99331039ae6a06979
| shareaza 1.8    : f35f6333c234e7e99331039ae6a06979
+
| AoM 0.2.2.80    : F35F6333C234E7E99331039AE6A06979
| AoM 0.3.2.160   : f35f6333c234e7e99331039ae6a06979
| HashCalc        : f35f6333c234e7e99331039ae6a06979
| ed2k_hash  (0.3): (error)
| ed2khash   (0.1): 8c5c92e3f356c0d7813ab791e4e60236
| BitCollider     : f35f6333c234e7e99331039ae6a06979

"bug2-19000KB+1.avi", 19.456.001 bytes
| edonkey v61     : fc6ce6fa3472e119cde592893180c74b
| emule 0.41a.10  : FC6CE6FA3472E119CDE592893180C74B
| emule+ v1 i     : FC6CE6FA3472E119CDE592893180C74B
| ed/ov   v48.1   : fc6ce6fa3472e119cde592893180c74b
| overnet v50.1   : fc6ce6fa3472e119cde592893180c74b
| shareaza 1.8    : fc6ce6fa3472e119cde592893180c74b
+
| AoM 0.2.2.80    : FC6CE6FA3472E119CDE592893180C74B
| AoM 0.3.2.160   : fc6ce6fa3472e119cde592893180c74b
| HashCalc        : fc6ce6fa3472e119cde592893180c74b
| ed2k_hash  (0.3): fc6ce6fa3472e119cde592893180c74b
| ed2khash   (0.1): fc6ce6fa3472e119cde592893180c74b
| BitCollider     : fc6ce6fa3472e119cde592893180c74b
As we can see, there are many utilities but also clients that "break the compatibility with this old bug", not only the newer edonkey/overnet-clients.
However, as emule outnumbers all the other clients, it isn't that easy to just decide that "they are wrong" and use only the "correct" hashes for AoM...

However, it might be good to focus on the hashing-bug in AoM first*... :?
*) Random guess: Files <= 9500KB don't hash correctly

PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW » Mon Jan 12, 2004 1:12 pm

Ugh. :x
Not what I wanted. :?

PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW » Mon Jan 12, 2004 1:20 pm

*) Random guess: Files <= 9500KB don't hash correctly
More likely would be 9500KB minus one hash buffer to 9500KB, this isn't my own code so it'll be iffy to fix. :(

PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW » Mon Jan 12, 2004 2:23 pm

Bug fixed, will make a release as soon as I'm sure I didn't break something else. ;)

Locked