Autocreqing avdump

Please report any sort of feature requests or bugs on the tracker instead of the forum! http://tracker.anidb.info

Moderator: AniDB

Locked
energist
Posts: 8
Joined: Wed May 12, 2004 9:29 am

Autocreqing avdump

Post by energist » Mon Jan 15, 2007 3:16 pm

I tried running new avdump on several files without any visible result. No creqs pending, file info in db is still the same, no file locks mentioned in wiki. So the question is: how does it work? Or is it working at all?

For example, I found this file in my collection (it is exactly the same as in anidb, all hashes correspond). Currently video/audio bitrates and length fields are empty for that file. Subtitle info is incorrect as well, there should be two softsubs tracks in there. So I typed in console something like 'C:\Program Files\avdump\avdump.exe -ac:energist:mypass PathToFile', it connected to anidb just fine, displayed hashes, codec info etc. without any error messages. But information in anidb is still the same and i don't see any creqs in 'my place'... So am I missing something here?

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Mon Jan 15, 2007 3:28 pm

yes, it works. there are atm 17k dumps pending, including one for that file. we don't want to proccess all that at once obviously. atm the server checks 500 dumps per day. if everything goes ok that number will be increased soon.

here's a simple diagram explaining what's going on:
http://wiki.anidb.info/wiki/images/b/bc/Autocreq.gif

energist
Posts: 8
Joined: Wed May 12, 2004 9:29 am

Post by energist » Mon Jan 15, 2007 3:33 pm

Thanks for the info. I suspected something like this, but better be sure.

User avatar
HiEv
Posts: 82
Joined: Fri Sep 08, 2006 3:34 am

Post by HiEv » Sun Jan 21, 2007 2:28 pm

At 500 per day that's about 34 days to get through those 17k dumps.

Can we get a progress meter or something so people can see about how long it will take their autocreqs to get processed?

Something as simple as a list of how many autocreqs are left for each day remaining to be processed would be fine, because then people could estimate how much time is left until day X is processed.

Also, would it be possible to have some dumps be processed automatically whenever the server's load is low? Getting caught up would probably help encourage the use of autocreqs.

Der Idiot
AniDB Staff
Posts: 1227
Joined: Fri Mar 21, 2003 10:19 am

Post by Der Idiot » Sun Jan 21, 2007 4:29 pm

17k dumps? lol?
45k dumps atm :P

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Sun Jan 21, 2007 4:57 pm

this is not about server load, it's about testing a new system.

it's 2k per day now, btw

egg
Posts: 769
Joined: Tue Nov 11, 2003 7:17 am

Post by egg » Sun Jan 21, 2007 6:43 pm

Do those numbers include duplicates or things that have already been locked?

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Sun Jan 21, 2007 7:23 pm

33K pending, 11,7K done

User avatar
nwa
AniDB Staff
Posts: 585
Joined: Sat Jun 07, 2003 10:51 am

Post by nwa » Wed Jan 24, 2007 2:27 am

files that already have a dump have a counter of how many times the file has been hashed, so it basically should show the number of users who have the file, which is good for validation... that is, of course, if epoximator fixed the matter with the counter increasing everytime you hashed the file again :P

User avatar
HiEv
Posts: 82
Joined: Fri Sep 08, 2006 3:34 am

Post by HiEv » Sun Feb 11, 2007 3:00 am

How are these auto change requests being processed? I've put in over a dozen from Jan. 20th to Feb. 7th, but so far only the one that I did on Feb. 5th has gone through so far. If this was a FIFO queue then something is going wrong.

One of the first ones I autocreqed was for this file on Jan. 20th (maybe 19th) and as you can see it still hasn't been processed.

Is this a bug? Are they still queued? Should I resubmit the older ones?

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Sun Feb 11, 2007 8:40 am

yes. no. yes. no.

mkv and ogm has been prioritized since avi has a mp3 xbr issue (maybe)

(atm: 23501 pending, 30567 done, 5013 unknown)

User avatar
HiEv
Posts: 82
Joined: Fri Sep 08, 2006 3:34 am

Post by HiEv » Mon Feb 12, 2007 12:10 am

Ah. Thanks. I wouldn't have guessed something like that.

Are you still doing 2,000 per day?

Also, is there any mechanism that removes duplicate auto change requests or are redundant reqs left queued?

Sorry for all of the questions, but this project has piqued my curiosity. :)

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Mon Feb 12, 2007 5:39 pm

no, actually, avi dumps are on hold until i'm sure about that issue (it's really not a serious one, but). so only new mkv and ogm dumps are done atm, which means far less than 2000 per day (we have reached a normal situation for mkv and ogm). the other formats will be done all at once (since there are relatively few of them) when... i feel like it..

remember that we are talking about dumps here, so there are no duplicate creqs. it can be duplicate dumps for each file, but that has only happened twice so far (meaning that our users have ok hw and no one has tried anything funny). anyway, when it happens, the autocreqer will use the most seen dump as basis for creqing. ie. "corrupted" dumps will not be confirmed by anyone else but the user who submitted it and therefore always lose against the real ones

User avatar
HiEv
Posts: 82
Joined: Fri Sep 08, 2006 3:34 am

Post by HiEv » Tue Feb 13, 2007 9:09 am

I just checked, and one MKV dump that I submitted back on 1/30/'07 hasn't been processed yet either:

http://anidb.info/perl-bin/animedb.pl?s ... fid=288541

Is there a reason for that or is it a glitch?

Also, you didn't specifically mention MP4 files. I have several that haven't been processed yet, so I'm guessing they're on hold in that "other" group?

epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator » Tue Feb 13, 2007 4:57 pm

oh, that's what you meant. yes, they are on hold because of pending manual/aom creqs at the time. there's a service for resetting that, but it's not enabled yet.

yes, other group at mp4 (800 dumps, btw)

Locked