Page 1 of 2

Use NTFS ADS (for 0.6)

Posted: Fri Nov 12, 2004 11:18 pm
by Elberet
ADS are Additional Data Streams. They are a feauture of the NTFS file system on Windows XP (not sure about 2000) and later. These streams are stored alongside of the file and are persistent during the file's life, even if copied or moved to another NTFS volume. Regular file access, e.g. sharing it with a P2P program or watching it, do not interfere with ADS at all.

Now for the actual suggestion...

If a file known to AOM is renamed or moved to a different location (e.g. whenever I move a series to a different folder as soon as it has been completed), AOM needs to re-hash the file again and it will think that it's now a duplicate. The hashing is a fairly expensive operation, however. Hence my suggestion: After hashing a file, store the ed2k-hash and filesize in an ADS for the recently hashed file. After discovering a new file, check if an ADS with an ed2k-hash and filesize exists for the file and, if present, use that hash to identify the file (if the filesize matches). Possibly, also store the file's location where it was first hashed, and if present, check the known-files list for that file, check it's existence, and if not present, assume that the file has been moved and update the known-files list accordingly.

Posted: Fri Nov 12, 2004 11:35 pm
by PetriW
Ooo, I'll look into this asap. Thanks for enlightning me! :D

Posted: Sat Nov 13, 2004 1:27 am
by Elberet
You're welcome... :)

Posted: Sat Nov 13, 2004 8:07 am
by exp
this might be usefull, however there should be an option to disable it and a large onetime message popup to the user when the feature is first used.
bc files with additional datastreams will produce weird warning popups when one tries to move them to a non ntfs filesystem.

BYe!
EXP

Posted: Sat Nov 13, 2004 10:51 am
by PetriW
Yeah, it'll be in the first time startup instructions.

I'll probably store size, ed2k, crc32, md5 and sha1 in an xml file in the data stream (for others to read) along with an aom checksum of the values to prevent fiddling.

Posted: Sat Nov 13, 2004 3:43 pm
by PetriW
Oh this owns!
http://msdn.microsoft.com/library/defau ... /ntfs5.asp

Works in windows 2000 too!

Posted: Sat Nov 13, 2004 4:28 pm
by egg
PetriW wrote:Oh this owns!
http://msdn.microsoft.com/library/defau ... /ntfs5.asp

Works in windows 2000 too!
I don't have time to read it atm, but you might want to consider adding more information for native/3rd party tools to search for details in these fields... That way even if they are renamed they could still be found based on Anime name, ....

Posted: Sat Nov 13, 2004 5:54 pm
by PetriW
egg wrote:
PetriW wrote:Oh this owns!
http://msdn.microsoft.com/library/defau ... /ntfs5.asp

Works in windows 2000 too!
I don't have time to read it atm, but you might want to consider adding more information for native/3rd party tools to search for details in these fields... That way even if they are renamed they could still be found based on Anime name, ....
Could be a good idea, the space used would be insignificant.

Posted: Mon Nov 15, 2004 1:39 am
by Elberet
Well, if you store that information in the right way, Windows Explorer itself can make use of it since the author/topic/title information it lets you enter is stored in an ADS as well. :)

Posted: Tue Aug 30, 2005 5:49 am
by Rogelio-60626
Oh, My God!

Posted: Tue Aug 30, 2005 12:38 pm
by e-Viper
sounds promicing.

I would have been already happy with another color if the filename wasn't found at the location it was hashed

Posted: Tue Aug 30, 2005 3:03 pm
by Amour
I want it.

Posted: Tue Aug 30, 2005 3:41 pm
by PetriW
It's in the beta already.

Posted: Tue Aug 30, 2005 7:46 pm
by Amour
By the way, WinFS Beta is available:
http://www.vnunet.com/2141599

And where can I find AOM Beta?

Posted: Tue Aug 30, 2005 9:38 pm
by PetriW
Sadly I don't feel like paying a gazillion for a MSDN subscription, ADS is enough.