I understand that the lack of manpower may be a problem; with thousands of lines of code to look through, it may be a daunting job to fix such a large problem.
Such a big job might be beyond the powers of eXp et al., or maybe not?
Since I have free time and programming skills, how about sending me the code segments that contain faults, and I can check through and fix the errors myself? I would be glad to help the cause and contribute to the excellent system you have built!
If lack of funds is the issue, I can send a specified amount in $USD via postal money order to an address, as well. I am very interested in seeing this problem fixed... What would be the dollar cost if I were to hire you to fix it?
Thanks.
4GB limit? [trackered]
Moderator: AniDB
well,
the problem is simply the dataformat which is currently used to store the size info.
it's a signed 4 byte integer field.
that limits the filesize to ~ 2^31
unfortunatly switching to unsigned 4 byte fields or 8 byte fields would probably be a real pain. don't get me wrong here, there are probably no code parts in anidb which would need to be changed for this.
however, switching to 8 byte fields would increase the database size and ram usage. I don't know how well perl would handle those either. I don't think that the few files which are larger than 2GB warrant this.
BYe!
EXP
the problem is simply the dataformat which is currently used to store the size info.
it's a signed 4 byte integer field.
that limits the filesize to ~ 2^31
unfortunatly switching to unsigned 4 byte fields or 8 byte fields would probably be a real pain. don't get me wrong here, there are probably no code parts in anidb which would need to be changed for this.
however, switching to 8 byte fields would increase the database size and ram usage. I don't know how well perl would handle those either. I don't think that the few files which are larger than 2GB warrant this.
BYe!
EXP
Here is the root of the problem: Size limit on files in AniDB / Invalid ID [OPEN]. Basically the system needs to be changed to do one of these EVERYWHERE the files size is used:
1) Change all of the number handling to string/float operations
2) Use a module to handle the larger numbers
3) Get the underlying perl to use 64-bit integers instead of signed 32-bits
an incomplete solution would only cause more problems than it fixes, so this is something that would need to be done carefully.
As far as working on the code, first you need to gain exp's trust and so far you appear to be off to a bad start. As a workaround currently I suggest you use the generic file to add the episode to your mylist for the time being.
[
I got interrupted and took a while to type this up and exp ended up responding first...]
1) Change all of the number handling to string/float operations
2) Use a module to handle the larger numbers
3) Get the underlying perl to use 64-bit integers instead of signed 32-bits
an incomplete solution would only cause more problems than it fixes, so this is something that would need to be done carefully.
As far as working on the code, first you need to gain exp's trust and so far you appear to be off to a bad start. As a workaround currently I suggest you use the generic file to add the episode to your mylist for the time being.
[
I got interrupted and took a while to type this up and exp ended up responding first...]Possibilities
Sounds a bit more complex. I am sure that in the near future, many files will be posted that are larger than 2GB. With the coming HD-DVD, Blu-Ray Disc, and other storage technologies such as HVD (Holographic Versatile Disc), files will only be increasing in size.
You could test the system and check to see what RAM/disk usage would be... size fields couldn't take up that much space, could they? Consider though that as time passes, it will only get more difficult to make the change, so why not switch now?
If you can implement 8-byte size fields or whatever size is required to fully support the ED2K maximum file size of 4,290,048,000 bytes in the AniDB by 31/03/2005, I will send a postal money order or wire the funds directly to an account of your choosing in the amount of $100 USD.
I am also open to negotiation on the cost of your time to do this job. My funding is flexible.
Looking forward to hearing your reply, and thanks for your consideration and time.
You could test the system and check to see what RAM/disk usage would be... size fields couldn't take up that much space, could they? Consider though that as time passes, it will only get more difficult to make the change, so why not switch now?
If you can implement 8-byte size fields or whatever size is required to fully support the ED2K maximum file size of 4,290,048,000 bytes in the AniDB by 31/03/2005, I will send a postal money order or wire the funds directly to an account of your choosing in the amount of $100 USD.
I am also open to negotiation on the cost of your time to do this job. My funding is flexible.
Looking forward to hearing your reply, and thanks for your consideration and time.
Well, as siad before AniDB is mainly fansub-centered and fansubbers tend to use codecs which give best quality-to-filesize-ratio. currently a normal fansub ep. ia about 170 MB to 250 MB. Dvdrips are usually encoded to xvids with 700 MB or 2x700 MB. Even with HD-DVD or other bigger storage-media this would not increase dramatically, as to some extend the eye cannot see the difference when watched on a screen/TV/beamer, as those limit the resolution. And as FS-groups usually keep an eye on filesize/bandwidth, I guess filesizes might triple some day, but not that much more in near future.
Remember: We do not catalogue 1 to 1 DVD-rips here.
Remember: We do not catalogue 1 to 1 DVD-rips here.
Re: Possibilities
Well,
right now switching to 8 byte integers is not ONTODO, however, should such large files really start to appear in numbers I'll rethink about making this change.
however, currently I don't think it's worth the extra strain on the server and the time it needs to implement & test.
however as anidb is constantly growing maybe we'll need to start gathering donations some day to handle the hosting costs.
I'd then certainly appreciate if you donate :o)
However, donations would not affect the priority of feature requests. :P
BYe!
EXP
right now switching to 8 byte integers is not ONTODO, however, should such large files really start to appear in numbers I'll rethink about making this change.
however, currently I don't think it's worth the extra strain on the server and the time it needs to implement & test.
sorry, I am doing this in my free time out of personal interest and I don't plan to change that.Isochroma wrote:If you can implement 8-byte size fields or whatever size is required to fully support the ED2K maximum file size of 4,290,048,000 bytes in the AniDB by 31/03/2005, I will send a postal money order or wire the funds directly to an account of your choosing in the amount of $100 USD.
however as anidb is constantly growing maybe we'll need to start gathering donations some day to handle the hosting costs.
I'd then certainly appreciate if you donate :o)
However, donations would not affect the priority of feature requests. :P
BYe!
EXP
-
epoximator
- AniDB Staff
- Posts: 379
- Joined: Sun Nov 07, 2004 11:05 am
