AICH extended tag support [tracked]

DonGato · Post by **DonGato** » Wed Sep 15, 2004 1:32 pm

rowaasr13 wrote:
DonGato wrote:exp already stated why this won't be added soon to AniDB. So, we can stop discussing if it should or not be added for now.
Why? Continued discussion might reveal additional pros or cons.

Can you read again what I said?
I didn't say to stop discussing if it's useful or not AICH but about including it on AniDB now.

rowaasr13 wrote:
Aw3 wrote:The only weak place in "improved compressed stream handling" is impossibility of checking not full compressed stream (when there is a disconnection before receiving full zstream), but this isn't happen very often.
Heh... This happens to be most important difference in AICH vs ICSH for me. I often get lots of files from other sources than ed2k (like someone's else HDD, so I do mean "lots"). I use simple mod for eMule I written myself which can import data from local files into running downloads, thus allowing me to quickly fix files that doesn't match correct ed2k hash by redownloading only corrupt parts instead of entire file. Before I had to redownload entire 9,28 block for every bit of corruption and with AICH this amount is reduced dramatically, which ICSH, alas, can't do. Not that there are lots of people who do that, but still...

Moreover, AICH should work in all mods based on latest version of eMule and ICSH works only in e+ and mods that borrowed this feature.

You understand your case is 1 in a thousand (maybe even millions)?!
We're talking about the average problem here not your personal application to fix downloads not finished in another networks.

rowaasr13 · Post by **rowaasr13** » Wed Sep 15, 2004 2:03 pm

Read second paragraph after second quote - it was written especially to counter just this argument. AICH hash would help any official or official-based mod eMule with recovering from errors in ED2K DL.

DonGato · Post by **DonGato** » Wed Sep 15, 2004 2:33 pm

That doesn't mean they couldn't improve (correct) the old algorithm, or even made a better design for AICH. As exp and Aw3 said it's a little extreme and not justifiable as it was done.

rowaasr13 · Post by **rowaasr13** » Wed Sep 15, 2004 3:05 pm

Some problems mentioned in Aw3's explanation are not related to AICH, but are flushing problems instead and won't require changing AICH computation in future.

Other things he calls "problems" is speed/usage of SHA1. I don't really think those are problems either. Even if SHA1 takes more time to calculate, it is more secure, preventing falsification of data (you understand, of course, that CRC32 won't stand against data corrupted on purpose?). Also AICH gives generic solution for recovering any data stream, no matter if it is compressed or not or where it came from or if corruption was just a transmission error or was introduced on purpose. All you need is AICH root hash and at least one AICH-enabled client as uploader. Can ICHS do that? And, considering that hash computed only once at uploader and downloader don't even use it all, unless there are corrupt data, I don't really see how speed is a "problem" for AICH.

DonGato · Post by **DonGato** » Wed Sep 15, 2004 3:42 pm

I think the discussion is going off-topic so I will stop here.

If you want to discuss that go to any forum where an open thread for such thing is open or even open one yourself. We're at AniDB, not eMule nor eMule Plus forums.

As I said my opinion is that we should see if it proves to be useful and used by most other clients. That only time will tell... and that's my first answer. I don't know why I got dragged to this...

rowaasr13 · Post by **rowaasr13** » Wed Sep 15, 2004 4:08 pm

Because it is fun.

Well, actually, I agree with you.

Only thing I want to emphatize is that it should be added soon after it proves to be useful, since it is important for rare files.

Elias · Post by **Elias** » Thu Sep 16, 2004 10:53 am

And now we have comment fields for files - it can be used for placing ed2k link with AICH addition.

Post by **PetriW** » Thu Sep 16, 2004 10:57 am

Elias wrote:And now we have comment fields for files - it can be used for placing ed2k link with AICH addition.

Now, please don't do that.

Post by **nwa** » Thu Sep 16, 2004 11:01 am

I agree, the file comment field should be usef for other information

Skywalka · Post by **Skywalka** » Thu Sep 16, 2004 12:02 pm

No "useful" information should be put in comments. Comments are for comments, not for data.

Post by **nwa** » Thu Sep 16, 2004 12:29 pm

Skywalka wrote:No "useful" information should be put in comments. Comments are for comments, not for data.

really?
then why is there the following line written above the file description field?

only use for important infos, like errors in the file...

Post by **exp** » Thu Sep 16, 2004 12:44 pm

nwa wrote:
only use for important infos, like errors in the file...

the main point is that the field should not be used to write actual "comments" about a file, i.e. a review about the quality of the subs or
the encoding quality.
however adding usefull information there, i.e. additional sub/dub languages, important details, maybe a short description about the kind of corruption if the file is marked as quality: corrupted, censored/uncensored (for hentai), ...

Although hashes can certainly be considered information about a file I am not sure if those should really be added as file comments. I'd say better don't do it.
If there is really a large demand for a certain type of hash it's better to include it regulary into the anidb database instead of simply adding it to the comment field.

But ed2k links (or bt/direct links) do definetly NOT belong into the comment field. Any such links added to files will be removed.

BYe!
EXP

Elberet · Post by **Elberet** » Sat Sep 18, 2004 2:03 am

Question: Did I understand correctly taht when E+ has received a 180kb chunk through a compressed stream, it checks that chunk via CRC32 built into the compression layer?

To me, AICH does look like a very useful feauture that comes in handy in way more then one out of a million cases. Assuming that
(1) the position of a corruption within a downloaded file is completely random and
(2) there is absolutely no data available to locate the corruption, other then the ed2k hashset,
then a separate protocol that aids in locating and fixing corruptions would cause a couple hundred kilobytes overhead, but at the same time save you from redownloading 4.61MB of additional data (average).

So, unless I'm wrong about assumption (2), it appears to me that the most logical thing to do would be to add sort of a on-demand zidrav on top the ed2k protocol: A client that has detected corruption in a file creates a hashset using 64KB chunks from the corrupted 9.28MB ed2k chunk, sends that hashset to another client, waits until the remote client has verified the hashes against his copy of the ed2k chunk, receives the corrupted data block(s), patches its own ed2k chunk and runs the whole block through the ed2k hashing algorithm again... I _thought_ that this is more or less what AICH does, except that it also tries to prevent someone from breaking a file by injecting malformed data into the ed2k network.

wahaha · Post by **wahaha** » Sat Sep 18, 2004 9:21 am

Elberet wrote:I _thought_ that this is more or less what AICH does, except that it also tries to prevent someone from breaking a file by injecting malformed data into the ed2k network.

"thought"? - It does.
Zidrav has two modes:

corrupted file's hashset -> compared against correct file -> create patch
correct file's hashset -> compare against corrupted file -> create patch-request -> get requested parts.

As far as I understood it, AICH works more like the second variant of zidrav-patching, except that it doesn't request the hashset for the whole file but rather only for the affected chunk (+ other chunks' hashes neccessary to compare them against the root hash).

Le Eliminateur · Post by **Le Eliminateur** » Mon Feb 13, 2006 5:37 am

and now after more than 1yr since the introduction of AICH and it's extremely widespread use(i don't think anyone is using a non-AICH compatible client or emule version today), where does this feature request stands?