Change of the voting grading system for animes [DONE/DENIED]

old granted and denied feature requests

Moderator: AniDB

User avatar
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato » Wed Mar 23, 2005 1:17 am

1 - unwatchable
2 - hopeless
3 - very bad
4 - bad
5 - decent (neither bad nor good)
6 - fine
7 - good (recommended)
8 - very good
9 - great (must watch)
10 - sugoi (masterpiece)


I wouldn't say 6 values are not enough to rank the animes you like, but maybe people have other ways of ranking things. IMO we should normalize the way people vote (the way they rate things) instead of diverting anymore. I thought the text explanation we added was for that but people (like you does) still vote 7-9 only for animes they like. :(

Andemon
Posts: 117
Joined: Thu Oct 14, 2004 4:12 pm

Post by Andemon » Wed Mar 23, 2005 3:39 am

I'm diligently following the ranking descriptions, and as the result, about 80% of my votes fall in 6 - 8 range. That's far too large percentage per single rating...

For example, I've rated both Ai yori Aoshi and Fushigi Yuugi with 7, but I damn well know that they don't deserve same rating -- but IMO they don't deserve 6 or 8, either. If I could, I'd rate the former 7.5, and the latter 6.5. It'd be far more accurate that way.

...but for all I know, it might be too difficult to make such change with the current version of AniDB -- so it's probably pointless to discuss that. -_-;

Furthermore, I really can't agree with what DonGato said. There aren't six votes for anime you liked -- there are three. Looking at the voting guide, it's obvious to me that 5 and 6 aren't for anime you like -- they're for mediocre anime. ...and based on same source, I don't think that any anime deserves 10. "...it changed your mind, gave you a whole new perspective on life and broadened your horizons"? Nope. I've never watched any such anime, and probably never will. All that's left is 7-9; 7 and 8 for good anime, and 9 for great.

That's just not enough to rate accurately.

I do agree though that normalization could be useful. Change the definition of 10 so that it isn't so unattainable -- that'd help a bit.

Elandal
Posts: 31
Joined: Thu Jan 02, 2003 9:14 am

Post by Elandal » Wed Mar 23, 2005 4:59 am

I believe I might have used 7.5 and 8.5 if they had been available to differentiate a little more in the 7-9 range. But already a 10 point scale is a lot.. If I had given out 7.5's and 8.5's, they wouldn't really be that accurate. When I look at my votes, sometimes I feel that I should lower votes of some animes and raise those of others, while at other times I feel about the exact opposite.

What's wrong with 3 star scale? 5 star? 7 point? 10 point? 100 point?
Obviosly with 3 star scale you're only going to get three kinds of results. You're not going to compare good anime to another good anime - they both got 3 stars for being good.
On 5 star scale you'll have "great" added over three stars (and "bad" below "not good"), and can fan out the votes a little more.

I believe 7 point scale would be close to optimal. Would leave 5 => 4, 7 => 5, 9 => 6, 10 => 7 on the upper end. If something is better than "not bad, not good", is it already good enough to recommend? I've used 6 for cases where eg. I've got a few good chuckles out of something but that's it. That anime would live just as well with 5 - it wasn't good enough for me to tell someone else about it. And 8 is just for those of the upper end of 7 that you don't think deserve a 9 - that is, you don't start harassing people about them until they watch them, but are the first ones you'd recommend to people after they've seen everything you've forced upon them (10's and 9's).

10 point scale is what we have now. It's not balanced symmetrically but rather there are more grades on the positive than negative side. For me, this is just fine - if I don't like something, I'm not likely to watch it to the end unless it's short. And if I don't complete it, I won't give it a permanent vote - how could I not having seen it completely? Still, I think there are more than enough low grades.. Doing away with one of them would be just as good. Even just two grades below "not good, not bad" would be fine: "bad" and "hate it".

So, how about grading systems with more choices - like 19 point grade (1-10 with half-points)? I doubt the low half-points would be of any use really, so it would add a lot of useless grades on that side.. In the mid-grades (4.5 & 5.5) I doubt it'd make any difference. If it wasn't good or bad, would I /care/ enough about it to try to compare it against other "not good, not bad" animes to differentiate them by half-points? And as I noted, in the good to great area, the half points would depend on my mood and I'd just want to shuffle them around forever, sometimes feeling that something is rated too high or low, while at other times wanting to reverse my changes again. In the extreme high it might make a difference: 9.5 would probably be even useful..

100 point scales? You know how they are.. the lower order digit is quite random and useless :roll:


So, summarizing my opinions:
7 points is minimum to work with, 10 points may have some problems but gives a few more grades to work with, more than that and differences by one grade are mostly meaningless. Thus, the 10 point scale we currently have is likely the best compromise.


And yes, most votes will be in 6-9 area. Most of us like most anime and thus will give at least "fine" for them, and most of us want to keep 10's in some check.
What I would like to get is a per-user vote distribution. A graph on my vote page showing how many of each grade I've given - one just like that on anime rating pages, but containing all my permanent votes instead of all users' votes on one anime. After that, start eliminating influcence of users based on their vote graph: if your vote distribution diverges too much from "acceptable" distribution, none of your votes will be used to calculate anime ratings.

User avatar
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato » Wed Mar 23, 2005 5:26 am

You can like or dislike a thing. Saying something is decent is far from being something you dislike. You might not like it a lot but you still like it instead of disliking it.

I think Elanadal pretty much summarized what I think. I would do the same as he recommends. Have a 7 point scale as the lower part is crowded with two much options, but agree that the current 10 point scale is a good compromise.

BTW, that 10 changed my life comment is pretty stupid. Mostly nothing in my life do that, but it changed my opinion/view of anime... that's for sure.

Jcubed
Posts: 3
Joined: Tue Mar 22, 2005 11:28 pm

Post by Jcubed » Sat Mar 26, 2005 2:12 am

Jcubed wrote:If you're willing to put some work into it there could be a profile option for whole numbers (defualt), .5, and even for .1. I know I'd put the .1's to use in the 7'ish area and DeathWolf said he would too.
If EXP is will do this it would make everyone happy, I'd be grateful if he was willing to and I think it would be the best solution for all parties.

A further clarification of why most votes of mine (and as far as I can tell many other people) end up being 6+, usually 7-9. I have a pretty limited amount of time I can spend watching anime (I spend more time encoding =/) and so do most people, so it's not that I don't think there are a bunch of 1-5, I definitely do, it's just if after a few eps I consider something a 1-5 I'll stop watching it and move to something else, hence unless it's a movie or 2 ep or smaller ova you won't see any 1-5 votes from me. If there's no way I can even watch half the anime I want to see why would I waste my time with something I that's not at least a 7 to me?

User avatar
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Post by exp » Sat Mar 26, 2005 1:38 pm

well,

just to let you know.
the current rating system is internally based on a 100 to 1000 scale. (1 => 100, 10 => 1000)
therefore it would be very easy to add additional X,X (7,5) or even X,XX (7,25) votes.
(my guess would be 3 minutes of work)
the question is if we want that :P

BYe!
EXP

Andemon
Posts: 117
Joined: Thu Oct 14, 2004 4:12 pm

Post by Andemon » Sat Mar 26, 2005 2:15 pm

How difficult would it be to implement what Jcubed suggested?
Rating scale selectable from profile options -- Normal (current) and Extended (+.5) would probably suffice. That way basically everyone should be satisfied, right -- those who prefer the current one can keep on using it, and those of us who would like more choices would have 'em. I mean, if it's not lot of work, then why not just implement it?

'course, I am biased towards extended rating scales (...been using Alexlit and Gamespot for ages, so I suppose I'm used to such scales) -- but I can't see why more choices would be bad, especially when you don't *have* to use them.

Then again, on the other hand, it'd require some modifications to Anime Hint, which would mean more work for egg -- wouldn't it? ... -_-

User avatar
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato » Sat Mar 26, 2005 3:21 pm

I don't think that should be done (having profile options). As exp said the question is if we want/need it or not. If there is no such trouble with the code and it's a mere cosmetic change I'm not against even if I don't like it that much.

But adding a profile option would mean to add another check to the database and another if in the code that is already troubled by CPU usage. We should avoid any unnecessary increase on the load.

User avatar
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Post by exp » Sat Mar 26, 2005 9:38 pm

DonGato wrote:But adding a profile option would mean to add another check to the database and another if in the code that is already troubled by CPU usage. We should avoid any unnecessary increase on the load.
that would actually not be a problem.
we're talking about one bit in a 32 bit integer here (which is allocated and cached in memory anyway and so far not fully used)

BYe!
EXP

User avatar
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato » Sun Mar 27, 2005 12:44 am

As I don't know the design of the site I couldn't say for sure. In that case then I would like that option.

Devil Doll
Posts: 49
Joined: Sat Mar 26, 2005 1:29 pm

Post by Devil Doll » Sun Mar 27, 2005 9:03 pm

Very interesting discussion...

During the last two days I entered most of my anime stuff here, all of which I had to convert my rating from another site (where I use some percentage value) to this 1-10 range.
After reading the explanations of the rating values (prominently linked in the website) I had no problems in doing so.

The distribution of my 70 votes (avg. vote 5.97) is as follows:

Code: Select all

10  9  8  7  6  5  4  3  2  1
 3  8  8  6 14 14  8  6  3  0
Actually I was not willing to use the "1" value due to the description but I used every other rating level without hesitating. Naturally many animes have good and bad elements thus 20% in each of the "6" and "7" column doesn't seem awkward to me... looks as if I'm just fine with the current system.

I don't see any downside about mapping both a "high 7" and a "low 7" to a 7. I can't strictly order all those 70 animes; my percentage ratings originated from giving ratings for certain aspects (art, animation, characters, music, series story, episode story) and assign weight factors to these aspects - which can lead to a very interesting anime getting downgraded for certain aspects that I dislike, relatively to an average anime without significant flaws. When mapping my ratings to the 1-10 scale here I became aware of this and didn't just divide my percentage ratings by 10, I tried to focus on the most important aspect (thus now giving a "10" to an anime that I had originally downgraded for too much violence, for example).
I also tried to give "objective" ratings, i. e. look at "value" instead of "personal satisfaction" - only to find out that certain animes that I like a lot but was doubtful about their "value" were consistently rated very highly by the database average! Consequently I renounced this idea and started giving purely subjective ratings, and more often than not they turned out to be mainstream: For those animes with an average below 7 I have not a single rating higher than 7, and for those with an average above 8 I have not a single rating lower than 5 (except for one entry that my "2" rating lowered to 7.99, with 649 votings).

What I do like about the system of only 10 values is the "graph" feature of the website providing wonderful normal probability curves in most cases, telling you whether you're voting "mainstream" or not. This would probably not work with a percentage scale, at least not for animes with less than 1000 votes.
Looking at those graphs I did notice that those animes with an exceptionally high number of "10" ratings also have a remarkable number of "1" ratings, such as Kimi ga Nozomu Eien or Shin Seiki Evangelion. Then again, while both being in my top 10 anime series these animes are controversial in a way... would it make sense to somehow display the "degree of controversy" of an anime, such as the standard deviation of all ratings for this anime?

Jcubed
Posts: 3
Joined: Tue Mar 22, 2005 11:28 pm

Post by Jcubed » Thu Mar 31, 2005 11:32 pm

exp wrote:that would actually not be a problem.
we're talking about one bit in a 32 bit integer here (which is allocated and cached in memory anyway and so far not fully used)

BYe!
EXP
PLEASE go for it! I see very little drawbacks such a move would cause and for people like me a large gain. ^^

Skywalka
Posts: 889
Joined: Tue Sep 16, 2003 7:57 pm

Post by Skywalka » Sat Apr 02, 2005 10:58 pm

Please EXP introduce a new scale from 1 to 20 so we can get rid of this discussion.

It pops up every two or three months with the same stuff discussed over and over again.

If some people really need that many divisions so be it.

If anyone complains they want the old system back we can tell them just to devide the number by two. Or we do it for them if they don't get that 11 is the same as a 5.5 rating on a 1 to 10 scale. Duh!

^^;;

Der Idiot
AniDB Staff
Posts: 1227
Joined: Fri Mar 21, 2003 10:19 am

Post by Der Idiot » Sat Apr 02, 2005 11:09 pm

check the news :P
it's already done

kidan
Posts: 319
Joined: Thu Feb 13, 2003 9:13 pm
Location: .DE

Post by kidan » Sun Apr 03, 2005 2:33 am

Hmm, doesn't work for ep/(group) votes yet.
For groups I guess it's not that useful, but I could immagine rating an episode 6.5)

But, I'm quite happy already, as I can now balance some anime pretty fair. (I've put the new scale to some good use already)

Locked