Vote normalization

DonGato · Post by **DonGato** » Wed Mar 23, 2005 1:23 am

That's why I said "I care...", meaning people differ in what they prioritize, so I don't thing giving different weight to each category in relation to a fixed taste would help.

Rafal · Post by **Rafal** » Wed Mar 23, 2005 2:43 am

I'm sorry, I don't think you've understood my suggestion, I don't suggest giving a different weight to each category, I suggest giving them no weight at all towards the overall rating of an anime. In this way the overall rating is not bound to these categories so comedies don't have to suffer from a low score for 'story' for instance, and it will also help give a more accurate picture of the reviewer's overall opinion on an anime while still retaining the reviewer's thoughts on each different aspect of the anime.

DonGato · Post by **DonGato** » Wed Mar 23, 2005 5:28 am

Then I don't understand what you recommend to do with the overall value... let the user set it?! as your example leads me to think you want to put some weight in some of the categories. Please explain it in another way then.

OnegaiNL · Post by **OnegaiNL** » Wed Mar 23, 2005 6:14 am

i vote by looking at the story, but ofcourse i also watch anime that nearly have a story... but i vote those on a different way... usually how i felt watching it... if i felt very good and i want more... i would vote it pretty high already lol (well not 10 ofcourse but somewhere around

Rafal · Post by **Rafal** » Wed Mar 23, 2005 9:33 am

DonGato wrote:Then I don't understand what you recommend to do with the overall value... let the user set it?! as your example leads me to think you want to put some weight in some of the categories. Please explain it in another way then.

Yes, I actually meant the overall score should be set by the user itself on a 20 (.5) point rating scale. I thought that was clear, sorry.

DonGato · Post by **DonGato** » Wed Mar 23, 2005 10:21 am

Now that makes sense.

kamenoko · Post by **kamenoko** » Wed Mar 23, 2005 6:43 pm

Rafal wrote: I don't know if anyone has suggested this idea before, but how about disregarding the highest and lowest 10% of votes (or 5%) as to eliminate all the reactionary 'hate-votes' and the mindless fanboy praise. I've seen this system implemented on a Japanese bishoujo game site and it seems to work pretty well there.

I did, I figured it was the simplest, and easiest solution to implement. It's just a bit of extra math when calculating an average. Well, a query to sort, a query to get how many votes have been cast, then a couple queries to disregard the top/bottom 5/10%, then an average taken.

But, exp seems to be ignoring this one, so it may be a while before anything (if anything) is done about this.

hhaamu · Post by **hhaamu** » Wed Mar 23, 2005 9:45 pm

I'll vote for the 5% filtering system, as it seems the simplest (and implemented the fastest, if ever. Yes, I'm being sceptical).

kamenoko wrote:But, exp seems to be ignoring this one, so it may be a while before anything (if anything) is done about this.

Oh, he tries very hard to ignore all the posts in here

Ultima · Post by **Ultima** » Wed Mar 23, 2005 10:30 pm

Hmm, I'm against filtering because I think it would upset a lot of users. It is one thing to normalize the votes, it's another to cast out users' votes by shaving the top and bottom by using a certain percentage.

I still think weighted votes would be the best and most effective way to normalize the votes. The DB has grown quite a bit, and many anime have a lot of votes. This wouldn't work well if the user base was small, but it is large, so weighing user votes should have a great effect now. I think every vote should be counted (whether it be fanboy or anti-fanboy or whatever), but I think they should be weighted according to how much anime the user has voted for. This would weed out the users that just register to vote for a few select anime and encourages users to vote for more anime.

OnegaiNL · Post by **OnegaiNL** » Wed Mar 23, 2005 10:45 pm

only filter out the obvious only 10 and only 1 voters or other numbers lol

Rafal · Post by **Rafal** » Thu Mar 24, 2005 11:24 am

Ultima wrote:Hmm, I'm against filtering because I think it would upset a lot of users.

Well they can cry me a river.

I think normalization is a little more important here than upsetting a few users who don't want to risk having one of their votes not counted towards the total statistics (oh noes! Now they won't count the 1 I voted on Rurouni Kenshin!). One or two of my own votes will probably not count either but I'll happily 'sacrifice' those for the sake of normalization. At least I know that one of the 'other side' will be taken down with me as well.

Anyway, I don't see what's wrong with taking out the extreme deviations, they do this all the time in experimental research as well. It's just for the sake of statistics and normalization, I really don't see how this can affect someone personally.

hhaamu · Post by **hhaamu** » Thu Mar 24, 2005 1:23 pm

We seem to have an ongoing race of FMA fanboys and RK OVA 1 fanboys right now. They are switching places every day. Both have gotten at least 30 hatevotes in the past month. Seriously, something must be done about this.

<speculation>I can imagine someone who hails FMA as the best anime ever checking anidb and seeing it's not #1 but something called "Tsuiokuhen" is. Never having seen it, he votes a 1 for it just because he thinks FMA must beat it.</speculation>
The names could just as easily be reversed, or one could add Beyond the Clouds to the equation.

Someone ought to ask EXP or someone else with access to make a few manual SQL queries, something along the lines of "select from users where user.vote(979) == 10 and user.vote(73) == 1" and post some numbers/anonymous statistics here to confirm whether this guess's far-fetched or not.

Post by **exp** » Fri Mar 25, 2005 9:53 am

there are 40 such users.
however, only 2 of them have the same email address.
7 have <=5 votes on anidb.

BYe!
EXP

Ultima · Post by **Ultima** » Fri Mar 25, 2005 7:04 pm

Rafal wrote:
Ultima wrote:Hmm, I'm against filtering because I think it would upset a lot of users.
Well they can cry me a river.

I think normalization is a little more important here than upsetting a few users who don't want to risk having one of their votes not counted towards the total statistics (oh noes! Now they won't count the 1 I voted on Rurouni Kenshin!). One or two of my own votes will probably not count either but I'll happily 'sacrifice' those for the sake of normalization. At least I know that one of the 'other side' will be taken down with me as well.

Anyway, I don't see what's wrong with taking out the extreme deviations, they do this all the time in experimental research as well. It's just for the sake of statistics and normalization, I really don't see how this can affect someone personally.

I think every vote should count. Imagine having an election process that shaved the top and bottom votes just to make the outcome "normalized". What kind of twisted logic is that? Are you suggesting that people aren't entitled to their opinion simply because they think the anime deserves a really low or really high rating? Would users vote lower than usual (resulting in even more skewed average anime ratings) to make their votes count more? These are things I wouldn't subject users to. This is a free database. Your opinion is as valid as anyone else's. If we do weighted voting, you can always increase your vote weight by watching more anime, vote for more anime, and increase your repretoire of anime.

It isn't about "oh noes they didn't count my '1' vote for Kenshin". It is more like, "What the fuck, I think that anime deserves a 1 (or whatever rating), and now you are ignoring my votes because I think it deserves it? What kind of voting is this where if you happen to not like the anime much or like the anime a lot your vote may or not be counted at all?"

egg · Post by **egg** » Fri Mar 25, 2005 9:07 pm

Ultima wrote:I think every vote should count.

I almost agree. As I said at the beginning of this topic:

egg wrote:
Votes should from "legitimate users" should not be normalized, when enough users that vote, things will average out, every series has individuals that vote really high/low.

Votes from dummy users created just to vote should be filtered out.

IF votes are filtered and/or normalized then on the anime page, the raw votes average (the one without the filtering/normalization applied) should be listed as well.

IF a user has their votes filtered there should be a clear indication on the myplace and myvotes screens with [a link to] the list of requirements to have their votes counted in the overall statistics.

So, although I agree all votes from regular users should count, I don't think it is fair for someone to create 100 users and vote 100 times. So what I propose is to make voting requirements to ensure that you have experienced enough anime to have a meaningful vote, something like the anime hint requirements. But, it is important that users know they are filtered and know how they can change that and raw results are displayed somewhere. Since the users would have to have a minimum number of votes and/or items in mylist it would make it harder for people who create multiple users just to vote...

BTW, you can't just filter out users/votes by email address. In some cases a user has forgotten their login and created a new one. Also multiple people can share the same email, my wife and I have different AniDB accounts (we vote differently), but we could have the same email address in there...