Normalized ratings/reviews?

shaydwyrm · Post by **shaydwyrm** » Fri Aug 08, 2003 9:33 pm

Meh! Microsoft tried to do that with Windows, and I'd rather not want to see AniDB go down the same way, being all bubbly, candy and tele tubbies, with assistants, wizards and hint popups everywhere.

As a linux user, I completely agree. On the other hand, I would never dream of selling a piece of software that wasn't as rigorous and intuitive as I could possibly make it.

I'm also just a huge fan of cool math tricks like normalization

Sure, it's probably not a high priority issue, and we could probably just let it lie, but wouldn't it be cool if the system dynamically adjusted itself for each voter's bias? So if some fool doesn't care to read the descriptions, or votes 10 for everything 'cause he thinks it's funny, or can't count beyond 6, then he just gets smoothly incorporated into the statistics without hurting anything? I think that would be badass.

Again, not a very high priority issue. Maybe I'm just too much of a math dork, and it would be better to leave things as is. I'd rather see a wishlist first in any case

In terms of the review feedback issue, Elberet makes a good point, but I worry about adding too many statistics. I would rather keep the interface as simple and clean as possible.

zaufany · Post by **zaufany** » Fri Aug 08, 2003 9:47 pm

@Elberet
If spam isn't a problem for you, go look at AnimeNfo. Only 5 "Best anime ever all 10's" is too much for me.

idiot-proof?
Does we need idiots on AniDB or ed2k? Anime is mainly for intelligent people. We should better improve Anime Hint than make a normalization.

shaydwyrm · Post by **shaydwyrm** » Fri Aug 08, 2003 10:05 pm

Edit: Never mind, screw it, I don't know why I keep pushing this. There are much more important things to work on. In fact, you guys should probably move those much more important things to other threads.

Elberet · Post by **Elberet** » Fri Aug 08, 2003 10:31 pm

zaufany wrote:If spam isn't a problem for you, go look at AnimeNfo. Only 5 "Best anime ever all 10's" is too much for me.

You might be right, I believed that most of the spammers would be driven back by the requirement to register as a user before being able to post a review. Then again, at AnimeNFO, the only way to cast a vote on an anime is by writing a review, while here at AniDB reviews and votes are separated from each other. After all, I hardly see any real junk reviews around here.

It also depends on what you consider spam... Spamming in the sense of posting a large mass of reviews isn't possible due to the simple fact that you'd have to register a separate user for each spam review.

Elberet · Post by **Elberet** » Fri Aug 08, 2003 10:52 pm

shaydwyrm wrote:As a linux user, I completely agree. On the other hand, I would never dream of selling a piece of software that wasn't as rigorous and intuitive as I could possibly make it.

Hehe, true. Lord knows, if the only text editor on the world was vi, people would actually have to stop switching their brains off at the same time they switch their computers on.

Then again, I shouldn't complain so much since I do use Windows XP on my work computer. I'm just too lazy to mess with Linux all the time. I have it on my server and my laptop since I'm developing there, but I sure as heck don't want it on the big puter and spend 50% of my time customizing my X - since that's surely what I'd end up with. -_-;

Gambit · Post by **Gambit** » Sat Aug 09, 2003 8:45 am

shaydwyrm wrote:Edit: Never mind, screw it, I don't know why I keep pushing this. There are much more important things to work on. In fact, you guys should probably move those much more important things to other threads.

Heh, at least we have something to discuss now

Since there aren`t that many other topics, why not trying to get a proper solution for what you suggested? If people don`t like the discussion anymore, don`t participate

But as long as someone thinks he has a point, he should defend it - like we did with the review-system. It *might* get approved after a while if it actually makes enough sense.

Elberet wrote:Then why not just rate reviews in the same way as animes, giving them "grades" ranging from 1 ("it's not a review, it's a joke - a lame one") to 10 ("hits the nail on the head AND is fun to read") ? But then I guess, wouldn't that be overkill?

A possibility would be to replace the 'YES/NO' option with an ... um ... combobox (damn, I really haven`t progged for ages...) so we can add a small description to the vote. I.e.:
Yes - The review answers most of the questions.
Neutral - The review is good, but lacks something.
No - The review doesn`t answer my questions.

Perhaps adding an 'I just don`t agree with the opinion on this anime!'-option would be an excellent way for stupid people to click on, so they won`t actually vote for the review if they just want to bash that reviewer for criticising an anime. But I think that might make it too confusing

Elberet · Post by **Elberet** » Sat Aug 09, 2003 11:18 am

Gambit wrote:combobox (damn, I really haven`t progged for ages...)

combobox = combination of dropdown box and edit field. you can select an item from the (dropdown) list or type in your own text.
dropdown box = ordinary list but with height 1 and only a single selection.

Anyhow, if it's going to be a dropdown, there shouldn't be a problem with introducing the four ratings: Great! - Ok - Incomplete - Bad!

Gambit · Post by **Gambit** » Sun Aug 10, 2003 10:38 am

Yep, dropdown box, that was the one I was looking for

But adding a little text would be the best option to display a rank if there are more than one.

Elberet · Post by **Elberet** » Sun Aug 10, 2003 11:09 am

Wouldn't that make the dropdown box a little too wide?

Iceman[grrrr] · Post by **Iceman[grrrr]** » Sun Aug 10, 2003 12:56 pm

well it doesn't mind if it is wide

it can be as wide as the review text

Elberet · Post by **Elberet** » Sun Aug 10, 2003 1:33 pm

Oh, I was thinking that it might be moved to the underused right border if it was a little more narrow.

Post by **exp** » Thu Sep 04, 2003 11:39 am

hm,

so basically you are requesting a change to a voting range of 1 to 10 @ display (as for animes & co)
and two additional voting options @ reviews
in a dropdown with a short info text beside the vote? (as for animes)
?

BYe!
EXP

Elberet · Post by **Elberet** » Fri Sep 05, 2003 12:24 am

No. A 1-10 rating would probably be too complicated and everyone would complain that they don't know how to vote.

My suggestion would be to:

- change to a voting range of -2 to +2 (-1 is excluded).
- sum(votes)/count(votes) will be somewhere between -2 and +2 and measures the quality of the review.
- a quality <0.5 means the review was disapproved. (For comparision with today's voting style, a quality of 1.0 would mean that everyone clicked "Yes".)
- reviews with qualities >1 are not only acceptable but very well written. this is visualized by adding little stars, Amazon-style.

So much for the technical part. For the display:

- remove the Yes/No links.
- add a dropdown on the right panel besides each anime with a caption saying "Rate this review".
- the dropdown box contains four options: "Great review, interesting to read. Super!

", "Good review, answers all important questions.

", "Bad review, contains errors. :-/", "It's not a review! >:-O". These options correspond to the four values +2, +1, 0 and -2 (in that order).

This assumes that a +1 rating is "normal" and +2 means the review was exceptionally well written. Using text instead of numbers makes sure that users will understand this concept and that they are supposed to rate a good review "+1" while reserving "+2" for something special.

Advantages of this:
- less confusion about votes.
- spam reviews are filtered out quickly.
- great reviews get an extra reward.

Of course, this doesn't have anything to do with the topic of this thread, since shaydwyrm was suggesting to interpret the votes a user casts based on their past voting behavior, i.e. when a user rates all animes "10", this user's votes are only counted as "8", while when a user only votes between 4 and 6 normally, an "8" would be counted as a "10".

Gambit · Post by **Gambit** » Mon Sep 08, 2003 6:29 pm

Elberet wrote:Of course, this doesn't have anything to do with the topic of this thread, since shaydwyrm was suggesting to interpret the votes a user casts based on their past voting behavior, i.e. when a user rates all animes "10", this user's votes are only counted as "8", while when a user only votes between 4 and 6 normally, an "8" would be counted as a "10".

I`m kinda hesitant about that. Even though I like the idea, I wonder how hard it`ll be to implement that without any bugs. How many anime do you have to vote before the system will understand what your behaviour is? And what if you only vote for stuff you really liked/disliked?
I think we shouldn`t tamper too much with these values, but let people actually rate it with the ideas they have about their vote.

Although I hate it when people vote 1 (I won`t name him), just because he thinks the anime is overrated and wants to lower the score with that vote. Or when he stopped watching a series because it became uninteresting, he also votes a 1.
But that`s just a person`s inability to make use of the voting system, imo.

Elberet · Post by **Elberet** » Wed Sep 10, 2003 11:04 pm

I agree... normalizing votes is a good idea in a public environment where unregistered visitors may cast votes. Around here, it makes no sense. In fact, it will cause problems when new people sign up and start adding their favorite animes to their mylists. So the first few votes will probably always be between 8 and 10, simply because new users are likely to vote first for the animes they like best.

Not to mention the technical aspect...

Unless a normalized vote is stored permanently as is, a query to determine the average vote would have to also find all votes for all animes for all users who voted for the anime, or something like that. I don't think SQL can do those calculations in a single query...