Corbin's crazy ideas for OpenAniDB

Want to help out? Need help accessing the AniDB API? This is the place to ask questions.

Moderator: AniDB

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Tue Aug 28, 2007 12:58 pm

don't get me wrong, i want to. the point is that blaming me/sqlite/encryption is a bad excuse at best.

you can "finish" aom without worrying about encryption at all. when the time has come to release you can just switch to the encrypted version. that way you can always test the latest official bulid without anyone having to keep a crypted version up to date. and if you need testers; there are no good reasons for not giving uncrypted versions to the mods

you might have something ready for release now but how would anyone know for sure? even if you said so i would find it hard to believe. i am however gullible enough to be persuaded


and for my bad manners, trashing this thread, your Awesomeness:
1) finish this and stick around before thinking about tcp api (not that you've been pushy)
2) set avdump at lowest pri. it's hardly needed and possibly only a waste of time

MostAwesomeDude
Posts: 38
Joined: Fri Jun 01, 2007 11:02 am

Post by MostAwesomeDude » Tue Aug 28, 2007 3:36 pm

epoximator wrote:and for my bad manners, trashing this thread, your Awesomeness:
1) finish this and stick around before thinking about tcp api (not that you've been pushy)
2) set avdump at lowest pri. it's hardly needed and possibly only a waste of time
Dude, it's all good. After all, in the last three weeks, I've tried to catch up to where you guys have been for over a year, which is a pretty big time difference. I started this because I was dissatisfied with AOM's performance on Linux, but I'm now in it because I enjoy coding stuff that I think others will like. I also think I understand a bit more how you guys operate and think, and why AOM was written the way is was.

Don't worry about "trashing [my] thread." We're all here to discuss the same thing: providing tools to enhance the ease of use of AniDB. As long as we're all working towards that goal, there's no problem with a bit of discussion and even less of a problem if we have disagreements. (If you think you're being a thread-trasher, don't ever visit LKML, the flamewars will burn yer face off! :wink:)

Anecdotally, I got a 320 NO SUCH FILE hashing a folder last night, and I wished that there was some way to automatically check the CRC32 without leaving the program, so my next target is support for the other unique hashes, as well as CRC32 for verifying and creqing.
~ C.

User avatar
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Re: Rar

Post by exp » Wed Aug 29, 2007 9:18 am

MostAwesomeDude wrote:
Rar wrote:
MostAwesomeDude wrote: - More hashes. TTH, MD5, SHA1, CRC32. CRC32 would be very useful to creqers, and could be used offline with a regexp that pulls a release group's CRC32 out from the filename. The others have been discussed at one time or another as being used to uniquely identify files in AniDB proper, and so I should probably implement them.
MD5 and SHA1 *do* uniquely identify files, and are used in some places *cough*winny*cough*. 2.5 has hashlib, I use pycrypto.
So fields 11 and 12 in fcode do uniquely identify? Sweet! That will make PyCrypto unneeded for hashing, since MD5 is gobs faster. Even without hashlib, there is still built-in MD5 and SHA1. I authored the Python version of TTH for SHA1, as well. That only leaves CRC32.
There might have been a misunderstanding there.
Hashes like MD5, SHA1 and TTH do have a very high likelyhood of uniquely identifying a file. However, as opposed to ed2k hashes, anidb does NOT enforce uniqueness for any of the other hashes atm.
Which means that due to copy&paste mistakes and similar issues there may be multiple files in the anidb database with, i.e. the same MD5 hash.
Though, as was already said somewhere else IIRC, we could of course simply enforce the uniqueness checks for all majoir hashes @ anidb file edit.

On your mylist export issue: We're planning to offer a real-time xml mylist export soon. It would allow clients to get an up2date XML dump of a users mylist data with one HTTP request.
However, the amount of included data which is not directly mylist related would be minimal (in order to reduce the size of the dump).

BYe!
EXP

Der Idiot
AniDB Staff
Posts: 1227
Joined: Fri Mar 21, 2003 10:19 am

Post by Der Idiot » Wed Aug 29, 2007 9:45 am

i'm fixing dupe hashes manually all the time. enforcing uniqueness might lead to tards manually changing them to work around that. it might on the other hand help to make tards realize the same file exists already as well.
so could turn out good or bad.

and as for uniqueness of ed2k. currently exp onyl enforces a full ed2k matchup. you cna easily add 2 ed2k hashes with different size, which leads often to me havign to merge them, because some tard added the file wrong first with the wrong size and then another guy added it properly

MostAwesomeDude
Posts: 38
Joined: Fri Jun 01, 2007 11:02 am

Re: Rar

Post by MostAwesomeDude » Wed Aug 29, 2007 10:43 am

exp wrote:On your mylist export issue: We're planning to offer a real-time xml mylist export soon. It would allow clients to get an up2date XML dump of a users mylist data with one HTTP request.
However, the amount of included data which is not directly mylist related would be minimal (in order to reduce the size of the dump).

BYe!
EXP
Squeeee!

Also, on files: I've put together a fun little regexp that will automatically check a release's CRC32 in the filename against the actual CRC32 of the file. Useful for verification and creqs. It'll be in next commit, whenever that is.

~ C.

Locked