Voting Rating with +, ++, +++, ...

Forum for discussing AniDB rules & standards. No small talk!

Moderator: AniDB

Amour
Posts: 640
Joined: Fri Oct 08, 2004 5:19 pm
Location: France

Post by Amour »

It was better before... Now I don't know what is the “recommanded” level.

By the way, shouldn't we change the category weight system the same way? I dislike the mix of * and + together.
vivafruit
Posts: 17
Joined: Mon Jan 19, 2004 4:44 pm

Post by vivafruit »

How about instead of subjective terms like "recommended" you just say the voting distribution you'd like to see? For instance:

0: Top 1%
9: Top 5%
8: Top 10%
7: Top 25%
6: Top 50%
5: Bottom 50%
4: Bottom 25%
3: Bottom 10%
2: Bottom 5%
1: Bottom 1%

Or if you don't like arbitrary numbers, something like:

0: +
9: ++
8: ++++
7: +++++++
6: ++++++++++
5: ++++++++++
4: +++++++
3: ++++
2: ++
1: +
NetKnight
Posts: 2
Joined: Sat Mar 24, 2007 8:11 am

Post by NetKnight »

I agree the new + system for voting does not work.
suppy
Posts: 151
Joined: Mon Jan 31, 2005 10:37 am
Location: A House of the Azath

Post by suppy »

I've been staying out of this conversation mostly because I understand both sides of it, and I agree with both sides - the one that doesn't want the descriptions there, and the one that does, and the reasons I agree with both are:

For the ones that don't want the descriptions there:
They are arbitrary and have no meaning. You can vote according to any scale you wish; the voting system will adjust.

For the ones that want the descriptions there:
They were something you could use to remember how good exactly you thought something was, whether it really was good enough to recommend to friends or if it was just something plain with nothing special in it and so on - the descriptions helped you decide where to put your vote. Now you haveto do it on your own, and you can't be sure you'll vote according to the same scale every time for all the years you plan to use AniDB.

I must say that I sympathise more with the second view; since it doesn't really matter according to the first view.

Anyway, I have something constructive to add here as well:
You can still find the old descriptions on the wiki if you use the history function to view an older revision of the Votes page.
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

so.. you would check your vote list, see 7, check teh dropdown, see "good", and think ok, i though it was "good" at the time.. must recommend it to my friends. yeah, makes lots of sense... hehe (you're probably thinking of the wiki, but that would've been less fun)

if we really want to remember we should just add a private comment field to votes. i want that myself.

i think vivafruit's idea is the best so far
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

epoximator wrote:if we really want to remember we should just add a private comment field to votes. i want that myself.
If we`re heading that way, implement the history of votes too then, perhaps even limited to i.e. 3 votes (1-original, 2-previous vote, 3-current vote), since one sometimes wants to adjust one`s votes after a while :) It should give you an indication about when you voted and what your thoughts were about the series later on.

vivafruits` suggestion is indeed 'technically correct' but doesn`t look too appealing. If you want to use percentages, I think it`s better to base those on your own voting list instead, so you can relate to your own votes when casting new ones. That should keep your votes consistent.
As for the + system viva shows, I think that one`s confusing and I don`t support that one. People consider the +es as 'more == better' so when 5 and 6 have more than 8, they start to wonder.
vivafruit
Posts: 17
Joined: Mon Jan 19, 2004 4:44 pm

Post by vivafruit »

Gambit wrote:People consider the +es as 'more == better' so when 5 and 6 have more than 8, they start to wonder.
My first response was, "no one is that stupid," but then I realized that this is the internet we're talking about. :(
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

vivafruit wrote:
Gambit wrote:People consider the +es as 'more == better' so when 5 and 6 have more than 8, they start to wonder.
My first response was, "no one is that stupid," but then I realized that this is the internet we're talking about. :(
Remember the word 'tards'? ;) The more you deal with them, the better you make them 'idiot-proof' in order to avoid their questions ;) That`s why we invented the descriptions in the first place, so it wasn`t prone to misunderstandings.
meis
Posts: 10
Joined: Sun Oct 16, 2005 1:54 pm

Post by meis »

the voting is fine... is just that some stupid kid, fanboy, biatch who screwing it by voting 1, can vote 0.5? =.=
egg
Posts: 769
Joined: Tue Nov 11, 2003 7:17 am

Post by egg »

You will never have a controlled voting system. Even if people vote properly, there will still be inconsistencies in the voting results.

The larger the range in voting, the more subjective the actual voting will be. The way to make more consistent voting (and to make high/low end voting have less of an impact) is to have a smaller voting scale. By default anidb has a scale of 1-10, but it can be even more granular with .5 increments or even 0.01 increments. This means that differences between for instance a 7 and an 8 are not consistent. At amazon they have a 5 star voting system which is something like, I hate it, I dislike it, It's OK, I like it, I love it. It cuts down on your freedom, but it makes the voting more consistent. Some places only have a 3 or 4 star type system, and others only have a thumbs up, thumbs down.

Another thing, unless people randomly watch anime, there will never be a perfect distribution. People are more likely to watch anime that are appealing to them, so people will generally watch things they are likely to enjoy, so votes at the low end will generally be sparse.

As far as the +'s, I think that they do not help. A star type system can work with a relatively low number of stars (up to 5), but with the pluses, there are too many. As far as vivafruit's idea, you cannot try to force people into an artificial distribution, it will not work. If people only watch things that they like, but force their votes into a perfect distribution, then they would vote 1 on the thing they liked the least, although they still liked it and would not have given it a 1 on a regular scale.

I think that words need to give people guidelines on what their votes mean. If you take a professional survey, they usually at least tell you what the end points mean, sometimes they also give guidelines on the in between points. Having particular meanings for particular votes though is more appropriate for small ranges (like Amazon). For a larger range like we have here, it is probably better just to label the end points.

Rate this on a scale of 1 to 10, where 1 is "I hate it" and 10 is "I love it". Given this, you would expect that people who vote properly would probably vote 6 or 7 most and with more higher votes than lower ones (because people are more likely to watch what they like). But it should be an honest opinion of how much they like it.

With the previous wording voting 10 or 1 almost taboo, although there should be relatively few of these, the previous wording was too restrictive, and I had found myself artificially changing my votes to fit into the defined meanings sometimes.

The more complicated you make it the harder it is for people to vote properly. If you try to imply artificial meanings to particular votes over a large range then you are tampering with the votes. If you try to enforce distributions over the votes you are tampering with the votes (since people do not necessarily watch an even distribution of anime). To gather opinions you need a simple system, either a smaller range, or an simple to understand range only defined at the end points. The current + system, with so many pluses is not something people are used to seeing, so is not simple to understand.
fahrenheit
AniDB Staff
Posts: 438
Joined: Thu Apr 08, 2004 1:43 am
Location: Portugal

Post by fahrenheit »

totaly agree with egg
epoximator
AniDB Staff
Posts: 379
Joined: Sun Nov 07, 2004 11:05 am

Post by epoximator »

"you hate/love it", "best/worst", "better/worse", ... ?
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

egg wrote:I think that words need to give people guidelines on what their votes mean. If you take a professional survey, they usually at least tell you what the end points mean, sometimes they also give guidelines on the in between points. Having particular meanings for particular votes though is more appropriate for small ranges (like Amazon). For a larger range like we have here, it is probably better just to label the end points.
I agree with that. Which was the entire idea of using descriptions in the first place, I believe. Although we didn`t think that far, but only considered it logical ;)
egg wrote:With the previous wording voting 10 or 1 almost taboo, although there should be relatively few of these, the previous wording was too restrictive, and I had found myself artificially changing my votes to fit into the defined meanings sometimes.
What exactly did you found restricting?
As for 'artificially changing your votes', I believe it`s best to start off with the descriptions while later on you would be casting votes based on your own interpretation of the grades. Once you know how to handle votes, you can also change your votes around to match your own interpretation of the votes, which means being flexible with the descriptions.
egg
Posts: 769
Joined: Tue Nov 11, 2003 7:17 am

Post by egg »

Gambit wrote:
egg wrote:With the previous wording voting 10 or 1 almost taboo, although there should be relatively few of these, the previous wording was too restrictive, and I had found myself artificially changing my votes to fit into the defined meanings sometimes.
What exactly did you found restricting?
As for 'artificially changing your votes', I believe it`s best to start off with the descriptions while later on you would be casting votes based on your own interpretation of the grades. Once you know how to handle votes, you can also change your votes around to match your own interpretation of the votes, which means being flexible with the descriptions.
I don't remember the exact situation, but I do remember thinking things like, I would probably give it an X, but according to the description I should give it Y. After a while, yes, I did ignore the labels and voted how I wanted. I don't remember the wording of those descriptions, but I remember thinking they were not evenly distributed. 1 and 10 were basically not supposed to be used, and the other wording somewhat pushed things around a little. I guess I don't like it unless you have balanced wording for both sides of the scale (epox has started a list). If you use balanced wording though, you have to be careful as you push towards the center though, with a scale of 1-10, the middle is 5.5 which is not available by default. That is why with a scale of 1 to 10 I prefer to only label the ends. Also, thinking back to surveys I have taken, the interface for voting is usually a series of bubbles to fill in with the wording above it. If we make the voting a series of radio buttons rather than a drop-down, then the interface itself will accomplish the meaning of the pluses.
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

I don`t disagree with 1-10 having only labeled the ends. But how much different was that from the old way of voting? It had some extra labels and new people were happy with them since it was a decent help to see how they could vote. Experienced users stuck to their own set of vote-ranges and ignored most of the labels. Which doesn`t really differ all that much from the original set of labels, imho - it was just described differently. It differs perhaps 1 in a range of 1-10, at max.

A new users sees a 7 as a pretty good show, worth recommending to someone who wants to watch a series. An experienced user sees a 7 as a good series which is above average. An 8 is considered good, instead - and a recommendation worthy.
A new user will probably see an 8 as a really good show, which is definitely worth recommending to someone of his own level of experience.

Fun sidenote, most new users generally vote 1 higher than experienced users. So if a new user votes an 8 for a show, the experienced user will vote it a 7. (No, I don`t have actual numbers for it, it`s just my experience with other watchers) So eventually, the votes will always differ a bit.
I generally look at the votes for anime on AniDB as a vote - 1, to determine what the value is of the show - which is, usually, spot on.

When you have both new and experienced users mixed, one of them will always disagree with the votes. So, yes, of course experienced users will not identify with the labels made up 4 years ago anymore. But new users still do.

As for getting rid of all labels, I still think they`re useful for newer users. If you want to change the labels, feel free to come up with other ones and I`ll gladly give feedback :)

What kind of list does epox have, btw?
Locked