Page 1 of 1

[REQ] Xml Import

Posted: Sat Jul 28, 2007 3:20 am
by urban0p
I have alot of friends who would like to join the site however they have lists at other places. Is it possible to create an xml import feature similiar to the ed2k dump?

Here is an example of an xml snippet.

Code: Select all

<anime>
  <series_animedb_id>48</series_animedb_id> 
  <series_title>.hack//SIGN</series_title> 
  <series_type /> 
  <series_episodes /> 
  <my_id>238101</my_id> 
  <my_watched_episodes>26</my_watched_episodes> 
  <my_start_date>0000-00-00</my_start_date> 
  <my_finish_date>0000-00-00</my_finish_date> 
  <my_fansub_group>0</my_fansub_group> 
  <my_rated /> 
  <my_score>6</my_score> 
  <my_dvd>0.00</my_dvd> 
  <my_storage /> 
  <my_status>Completed</my_status> 
  <my_comments /> 
  <my_avg_ep_duration /> 
  <my_times_watched>0</my_times_watched> 
  <my_rewatch_value /> 
</anime>
So something that was able to read this file and add it to their mylist. What do you think?

Posted: Sat Jul 28, 2007 7:48 am
by exp
is that an xml structure you just came up with or is some other page/program exporting data in exactly that way?

One issue would be that mapping some fields to anidb could be pretty hard. Anidb just doesn't store some parts of that information and others might be used differently.

I.e. anidb's mylists work on a file level, not an anime level. Mapping is somewhat straight forward if generic files are used (no group). But even then your xml format only lists the watched episodes, not the collected ones. And there may be cases were the episode numbers in anidb and some other page do not match at all.

Using the group info too would probably be all but impossible. With group names/tags from another page the chance for a correct, automated mapping to anidb groups is pretty slim. On top of that even if the anidb group id were to be specified manually, many groups have multiple released files per episode...

So all in all, I'd say that some kind of automated import using generic files might be possible, but if you want file level accuracy it would be better to use an AniDB Client to hash your files and build your mylist that way.

BYe!
EXP

Posted: Sat Jul 28, 2007 10:35 am
by urban0p
That code was Generated from Myanimelist. What would be an acceptable compromise? True I understand now that the db mapping is done at a file level instead of an anime level. However it seems too much for people with existing lists of over 500 animes to transfer their list manually. Normally I would say that your solution with AOM is fine. As I do use that myself. However many people have their content on external sources. Maybe you can suggest some other means of importing. I have looked at many places for the perfect solution of xml to ed2k however that does not seem to have been done.

Waiting Patiently
Urban0p

Posted: Sat Jul 28, 2007 2:57 pm
by epoximator
it could be done with the UDP API using generic files. the main issue would probably be title mapping, but it would probably work ok in most cases. not all of the data in that example can be imported, though (as exp said).

if you send me more data to work with i can write a simple java client for importing

Posted: Sat Jul 28, 2007 8:00 pm
by exp
Maybe a special import page on the website would be better for this. An UDP API based approach would have to sent one UDP packet per file.
If someone wants to import a 500 anime mylist with an average of 24 eps per anime that would lead to 12.000 packets.

Same goes for the matching of anime names, that would be easier with direct sql access. Users also wouldn't have to download any client software that way.

BYe!
EXP

Posted: Sat Jul 28, 2007 9:48 pm
by epoximator
actually, you can add more than one file per packet with the upd api...

MYLISTADD aname=Seikai no Monshou&gname=zx

Posted: Sat Jul 28, 2007 10:10 pm
by Der Idiot
you could just do both ;)

Posted: Sun Jul 29, 2007 8:26 am
by exp
I can look into the animedb side of this, however I would need some data for testing.
urban0p, can you send me an export of a large myanimelist export? (just pm me an url where i can download it)

BYe!
EXP

Posted: Sun Jul 29, 2007 12:33 pm
by urban0p
Exp: Can I send it to you via an attachment? If I can could you send your email addy to my pm.

Cheers
Urban0p

Posted: Mon Jul 30, 2007 7:57 am
by exp
a first implementation of this is finished (though not yet available on the main server).
however for testing purposes we'd still need some large example exports.

BYe!
EXP

Posted: Thu Aug 09, 2007 6:02 pm
by imokie
exp wrote:however for testing purposes we'd still need some large example exports.
I saw on their forum that they have an AniDB MyList import feature: http://myanimelist.net/import.php
Maybe you could import some AniDB lists there and then export them again. Should be the easiest way to create some example exports. ;)
(At the moment that import page gives some PHP errors, so I'm not sure if it works.)

Posted: Sun Aug 12, 2007 2:50 am
by urban0p
Just used this feature, apart from the 50 limit of uploads. Out of 500 anime titles it only didn't pick up about 40. Excellent work. I suggest for further improvement that the unknown title animes should be listed in a seperate box aswell so that you can manually add these titles and cut and paste to a text file.

Once Again Excellent Feature