Voting Rating with +, ++, +++, ...

Forum for discussing AniDB rules & standards. No small talk!

Moderator: AniDB

fahrenheit
AniDB Staff
Posts: 438
Joined: Thu Apr 08, 2004 1:43 am
Location: Portugal

Post by fahrenheit »

i admit i missuse the system, i only use the upper half of the scale to cast my votes, in a 5 point voting system, this is to say that gambit is totaly right, users have their own voting range, but they still kinda followed the old meanings, the +'s don't add much to the system, we still vote the old way in chii.
I view the votes as personal thing, not a public thing, ence my use of a 5 point scale.

just my 2 cents
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

my list will always be the one implemented, hehe.

no, users weren't happy with it, they were used to it. important difference. i know all about that since i'm responsible for many changes around here

to me, you are pretty much arguing for removal of the labels, only to end up with the wrong conclusion.
new and exp. users vote differently -> exactly why we shouldn't pretend that the numbers have meaning. i've seen "how could anyone vote less than 7 for this" (and similar) several times, were the user obviously has mapped some meaning to it and expects everyone else to follow. i would say that the labes actually make the difference between new/exp users larger, not less.

i also think you are underestimating "new" users. even a monkey know how to evenly distribute between 1-10. it's not like they can't, more like they don't want to, if that's the case
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

Assuming you`re talking to me, let me respond :)
epoximator wrote:no, users weren't happy with it, they were used to it. important difference.
You claim people weren`t happy with it, but used to it. Perhaps that`s true, I can`t say for sure, but I honestly haven`t seen many people complaining about them. Perhaps they weren`t happy, but they didn`t feel a need for changing it, otherwise there would be a lot more people saying something about it. Looking through this thread, I don`t exactly see many people encouraging the label-less idea though. Wouldn`t this thread be a decent indication of what the people want?
epoximator wrote:new and exp. users vote differently -> exactly why we shouldn't pretend that the numbers have meaning. i've seen "how could anyone vote less than 7 for this" (and similar) several times, were the user obviously has mapped some meaning to it and expects everyone else to follow.
And how exactly does having a no-labeled voting method make this easier? "How could anyone less than ++++ for this?" would be the new reasoning, if you do it with the + way.
People have personal opinions, so of course people would still complain about others not voting according to what they believe it`s worth it. I don`t think that has much to do with the labels or ratings 1-10, really. If people see others voting differently than them, they will complain anyway. Whether you use an ABC-grading, 1-10 or a number of + signs, the result will be the same when others grade it differently. So I think that`s an issue by itself, not an issue concerning whether or not it has labels.
epoximator wrote:i also think you are underestimating "new" users. even a monkey know how to evenly distribute between 1-10. it's not like they can't, more like they don't want to, if that's the case
Never underestimate tards :P No, really - I also believe that everyone should be able to distribute between 1-10. But like you said, want and can are two different things. But that`s all beside the issue of having labels or not, imo, because either with or without, people vote the way they want.

You still keep the issue of someone clicking the 6th radiobutton/voting a 6 while someone else clicks the 8th/votes an 8, while still having a quite similar feeling about the show though. I think it`s better to regulate this by giving meaningful comments/labels to the options. Whether they`re numbers, plus-signs or radiobuttons, I don`t care (although I do like the numbers :P)
And once again, once you watch more, your voting-sense changes as well. So eventually your votes will differ from how you used to vote, so you already create the gap between your current self and your previous self, so how would a different voting system solve that?

I admit that the old labels were subjective. How about more neutral labels then? i.e.
1 = worst
2 = very bad
3 = bad
4 = below average
5 = slightly below average
6 = slightly above average
7 = above average
8 = good
9 = very good
10 = best
where you take a 5.5 as the absolute average which you can`t choose, due to the fact 10 doesn`t have a middle.

Bottom line (I think)
Okay, perhaps the labels weren`t that good after all. But I rather see the people give input for new labels which fit the grades better rather than replacing it with an entirely different system without labels. That`s not criticizing your system, that`s criticizing the fact that they`re without labels ;)
Last edited by Gambit on Thu Apr 19, 2007 10:22 am, edited 1 time in total.
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

Gambit wrote:You claim people weren`t happy with it, but used to it.
my point was conservatism. like you said, neither you or i can say for sure what the general opinion is. (not that it matters)
Gambit wrote:And how exactly does having a no-labeled voting method make this easier? "How could anyone less than ++++ for this?" would be the new reasoning, if you do it with the + way.
it's not supposed to make anything easier. it's about not fooling anyone into believing something. that is; "7" = "good". the comment should be: how can naruto be rated higher than narutaru?
Gambit wrote:But that`s all beside the issue of having labels or not, imo, because either with or without, people vote the way they want.
yes, they do and they should. but they should not think that the rating is directly correlated to their vote value or some label
Gambit wrote:So eventually your votes will differ from how you used to vote, so you already create the gap between your current self and your previous self, so how would a different voting system solve that?
it's not supposed to solve that either. you obviously have to update your old votes as you watch more. how this could be done more easy is another discussion. (vote history is planned, as a side note)
Gambit wrote:I admit that the old labels were subjective. How about more neutral labels then?
better, but i still don't think they're needed or good for humanity
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

epoximator wrote:
Gambit wrote:You claim people weren`t happy with it, but used to it.
my point was conservatism.
Well, I can live with that explanation :)
I agree it`s not good to be too conservative, but I also don`t agree of applying changes without interaction with others about it. Of course, some open interaction can only be achieved through forcefully implementing something and waiting for feedback. So I hope you like the feedback here ;)
epoximator wrote:it's not supposed to make anything easier. it's about not fooling anyone into believing something. that is; "7" = "good". the comment should be: how can naruto be rated higher than narutaru?
On a scale of 1-10, with 10 as the best possible rating, I think everyone would consider a 7 as something which equals 'good'. There are still differences of course, with 'above average', 'recommended' and all, but it`s just an indication I don`t think you should follow blindly. But it equals what most people instinctively label it - no matter what it is called.
As for the comment, that`s simply subjectivity. You cannot do anything about that. If you like Naruto more than Narutaru, then feel free to vote so. But don`t push that opinion on others, I say.
epoximator wrote:
Gambit wrote:But that`s all beside the issue of having labels or not, imo, because either with or without, people vote the way they want.
yes, they do and they should. but they should not think that the rating is directly correlated to their vote value or some label
It all boils down to this, heh? ;) But fact is, that there will always be some relation; whether it is a label, a value or a radiobutton from left to right, people always have their own vote values for a rating method. Lousy enough, those never match with others perfectly. This is why I prefer to see labels so people can at least feel a match with their own ratings. As a guideline, not a hard rule it has to match with.
epoximator wrote:
Gambit wrote:So eventually your votes will differ from how you used to vote, so you already create the gap between your current self and your previous self, so how would a different voting system solve that?
it's not supposed to solve that either.
Ah, good to see you agree with me :)
epoximator wrote:you obviously have to update your old votes as you watch more.
My opinion is that you should only update your votes once you`ve rewatched it or changed your opinon. You don`t suddenly change your opinion of 30 shows at once, right? So why making that easy to update?
epoximator wrote:(vote history is planned, as a side note)
Image
epoximator wrote:
Gambit wrote:I admit that the old labels were subjective. How about more neutral labels then?
better, but i still don't think they're needed or good for humanity
Perhaps not needed, but preferred would be a better word ;)
In the worst case, perhaps we can even make it an option in the preferences ... "labels or not", which seems to be a solution for most user-based issues.
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

yes, i had to be forceful here, i believe. it's just not enough people who cares, or at least; they don't appear before they miss something. which is also the reason why most feedback is negative and conservatism is a step ahead. it somehow reminds me of 'The Little Red Hen'.

you missed my point again, it seems. subjectivity is ok. comment all you want, just don't expect me to understand 7. the label system feeds ideas like this; i voted 7=good, rating is 7.XX and can be translated back to good, because it's defined in a list. if you want to make a statement just compare A with B, ie. narutaru>naruto. it's much more useful than narutaru>7

and again, users can and will have whatever system they want to. we just don't have to feed them with something and then believe we've reached uniformity, or accomplished anything at all

as i see it, anidb has been pretty special with the labels. we've lived in our own little happy world where everything is simple and plain... AND THEN evil epox came and took our virginity. it's weird that numbers usually stands by themselves everywhere else (mostly).
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

Many other places have an explanation what the different grades represent though, especially those using 1-100.

I've not complained about this as I've seen it as pointless, the grades were removed pretty much without asking around, from what I saw it was even split in the mod channel for and against.
That you put those plus signs there instead just showed exactly how poor thought through the change was, even if it was planned for a long time.
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

ehehe, if you'd bothered to actually read what i've written so far you would know that it was intentional. besides that, i can't really blame anyone for not getting my points

(and... this was not planned for a long time. the only thing i can remember was someone suggesting to drop the numbers and only use labels, just to get raped by a bird)
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

I've read every post of the thread and it hasn't changed what I thought about this change 2 months ago, it was a poor thought through hackjob.

I think a change along these lines have been discussed to and from for like 3 years by some. ;) But it wasn't changed as they couldn't get exp to do it.

Anyway, most important thing for me was that the + signs were removed and they have been.
I do think you're wrong in not many other places having a description of each vote, plus the more important/serious the grading the more defined they are. Any serious paper/site/show/whatever with multiple people voting needs to have definitions on what it means to vote a certain way unless a system is put in place to even out the votes (which we have, hence why this change is not too bad).


We could always implement a Famitsu style voting system though!!!! :twisted: :twisted: :twisted:
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

replacing the labels with + wasn't a hackjob, it wasn't even a job. it was supposed to be provocation and it seemed it worked.
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

Sigh, yes I guess that's how you people work.
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

epoximator wrote:yes, i had to be forceful here, i believe. it's just not enough people who cares, or at least; they don't appear before they miss something. which is also the reason why most feedback is negative and conservatism is a step ahead. it somehow reminds me of 'The Little Red Hen'.
I don`t really get your reference, but that`s probably my problem. I don`t think it was really bad to provocate by just changing something and watching what kind of feedback you get. It was just the lousy + sign which made at least me react that fiercely :P
epoximator wrote:you missed my point again, it seems. subjectivity is ok. comment all you want, just don't expect me to understand 7. the label system feeds ideas like this; i voted 7=good, rating is 7.XX and can be translated back to good, because it's defined in a list. if you want to make a statement just compare A with B, ie. narutaru>naruto. it's much more useful than narutaru>7
I don`t think I missed your point, it is just that I don`t fully agree with you :P You want to let a user interpret a 7 (radiobutton, number, +- sign, whatever, as it boils down to a number anyway) as whatever he wants, whether it be a good or an above average or recommended or whatever label you can imagine without having to confirm that to any predefined label AniDB sets it to, am I right? You want to give people the freedom to determine how much value a 7 bears. Tell me if I`m right in summarizing your idea of ratings :)

Your suggestion for using comparisons I`m not completely satisfied with. While I understand it might be more natural to compare them with titles, it would directly link a title being better/worse/exactly the same as another title, which I personally don`t like. I mean, I voted a 9 for Koi Kaze, but I also voted a 9 for Full Metal Panic! for instance, but I would never compare Koi Kaze to FMP! as in Koi Kaze > FMP, because I don`t believe they are comparable. But that`s a personal opinion ;)
epoximator wrote:as i see it, anidb has been pretty special with the labels. we've lived in our own little happy world where everything is simple and plain... AND THEN evil epox came and took our virginity. it's weird that numbers usually stands by themselves everywhere else (mostly).
In a range of 1-10, it is common sense that 7 would mean 'good', or something like 'above average'. That`s why other sites don`t explain them as such. It is just a personal interpretation what kind of (mental) label one adds to it, but fact is, nobody would call a 7 bad, or excellent. What is true, is that there is an entire range between i.e. 'good' and 'above average'. While you are correct in saying that numbers stand by themselves normally, I don`t see what the problem is with having a predefined label attached to it. But that is -imho- simply a difference in preference between you and me.

@PetriW You mean you don`t work like that with AoM? :)
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

Gambit wrote:@PetriW You mean you don`t work like that with AoM? :)
Don't think so! ;) I've heard rumours about me not working on AoM and in that case it'd be impossible for me to do anything such!
Last edited by PetriW on Wed Apr 25, 2007 12:57 pm, edited 1 time in total.
PetriW
AniDB Staff
Posts: 1522
Joined: Sat May 24, 2003 2:34 pm

Post by PetriW »

I just had this horrendously stupid "only PetriW could ever think that up" idea where.... What if the users median vote was labeled "average"! (This is not really a suggestion, just a random thought, it would need some beating at least.) Then if some tard voted 10 for all anime, then 10 would be labeled "average", and other votes would have similar descriptions based on what was the current median vote for that user. ;)
I guess for me "average" would be 6.5 or 7 or so.

No idea if it'd work, make sense, cure cancer, lead to world peace or solve anything whatsoever. But it was a cute thought at least. :P
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

@PetriW - Kind of what vivafruits started here
vivafruits wrote:How about instead of subjective terms like "recommended" you just say the voting distribution you'd like to see?
and where I suggested this?
Gambit wrote:If you want to use percentages, I think it`s better to base those on your own voting list instead, so you can relate to your own votes when casting new ones. That should keep your votes consistent.

Let me take that apart for you by making use of a user having casted five 10-votes and will be casting one 8-vote.

So you`ll end up with translating one`s vote to a 'pre-defined list', like a static 1-10. Those five 10-votes (user-values) will get translated as a '5.5' (general average on the server) and once you cast a 6, it will re-calculate those votes and you`d get a new set of votes: 5x10 + 6 = 55 / 6 = 9.1 (approximately).
So a 9.1 vote will become your median vote. Which means those 10s are suddenly translated to a 6 (approximately) on the server, whereas your 8 will become a 5 (approximately).
Well, that should work. It would only cause a stressful situation on the server which has to re-calculate all votes-per-anime into a new average, due to all kind of users who voted for that particular title casting new votes for other titles, which influences their vote-value for all other titles they already voted for.
Cache and cronjobs should be able to spread that, but I don`t know how much it would influence the stress on the server by using this.

Every new vote from a user
1) would cause the server to re-calculate the new median vote for that user.
2) would cause the server to re-calculate the general vote for all anime the user voted for before casting this vote, since the user`s median vote has changed and his previous vote might not have the same value as it used to be; it can suddenly be higher than average or lower than average.
3) could make things harder on i.e. the AniDB Hint, which basically feasts on what users vote.

I don`t know how much extra stress this causes on the server, but although I like the idea, it might be a bit too hard on the server and the availability of AniDB.
Last edited by Gambit on Fri Apr 27, 2007 6:41 am, edited 1 time in total.
Locked