Review Moderation & Trusted Users/Reviewers [DONE]

old granted and denied feature requests

Moderator: AniDB

Locked
Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

Post by Elberet »

If it's ideas you want, I can be of service. Here's another one I've been pondering:

All users who currently have submitted at least, let's say 5 reviews and have a minimum total word count of at least 4000 words and an average approval rating of at least 5.5 are considered "trusted" reviewers. When a new review is posted, the db randomly selects 19 trusted reviewers and one mod. These users are then shown a notification the next time they visit the page and are brought to the newly posted review and are given a chance to approve or reject the review.

Voting modus:
- each trusted reviewer has 1 vote, the mod has 2 votes.
- an option [approve / reject] must have at least 5 votes to win.
- an option must lead by at least 1:3 votes to win.
- if all eligible voters voted - or three days have passed - and no option leads by at least 1:3 votes, simple majority wins.

In other words, if the first five people (or the first three and the mod) approve the review, it gets posted immediately. If the first three users reject it and the next 9 approve it, approve leads with 3:1 votes and wins immediately. The other 12 users are not considered and will neither see the notification anymore nor will they know that they were selected.

Since this new system replaces the review voting, new trusted reviewers are selected by the number of reviews that were approved: if the total number of reviews minus the number of denied reviews times two is >= five, a user is considered "trusted".

I always wonder why you need mods for everything when the community can handle things like this just as well. :mrgreen:
Guest

Post by Guest »

Hey,

After reading and carefully analyzing this thread, I think the review rating system should stay. But changes of status should be given after a certain point.

For example:

1. Avg. Review Rating - this is a crucial number, as at least 25% of the accounted votes are honest (since there are several - inclduing all of us - that actually vote based on the review's worth).
2. Total # of votes - As avg. review rating is dependent upon this variable, it must be used to determine effectively how good the reviewer is.

Setting things together, why not create a status that enables those of us who add 3-4 reviews a month that are informative, not spamming or spoiling, and substantial for all requirements to immediately be "trusted?" For example, I have 270 votes as of 11-20-2003...and I have a few reviews that I worked extremely hard on, only to get 0's or #'s > than 10. Of course I'm picky, so I would bring this number to mind.

votes: 200, avg. review rating: I'm aiming at 5, but can be higher or lower. = status: Trusted Reviewer

With that in mind, the reviewing system can still provide a fair judgement for those who work on their reviews. This should have the place where the "Rating: <number>" appears be replaced in green bold Trusted Reviewer.

Everyone who has lower than 200 votes or an average of 5.00 will still have their reviews rated with a number. Using him as an example (gulp, don't hurt me), Gambit has almost 800 votes and approval of 8.16 - it's obvious he knows what he is doing. Looking at most approved/disapproved reviews, this will bring Gambit's Berserk review back to the top, because he would have this "status." Those who spam in general as I've seen have low averages anyways, besides only writing at the most 2-3 reviews. The only way someone could fall through the cracks is if they write good reviews and get bad enough scores not to achieve a 5...there are at least a decent number of honest voters to help such a person get a "5," and thus they reach that status.

These trusted reviewers will still have the same ability level as all other reviewers - they will just have a their reviews posted and organized alphabetically, or chronologically. Though right now it seems there are only 4 uber-status (including me, nwa, gambit, and themug)...there are a few determined people who will eventually reach this level, too - so its not too easy, but not unattainable.

Last, the only ones who should know about this number are those who end up reading it in this thread I suppose (and of course, it can be deleted) - keeping the number (200 as an example) secret will keep competition at a minimum, as nobody will know when they get the status.

If this idea is acceptable, there are only two worries:

-How to determine if a trusted reviewer won't spam. I suppose they are "trusted" for a reason - however, I can only say that there aren't enough guarantees for everyone. Also, a method for status revocation should be developed.

-Elite group chaos. Like kamenoko mentioned, an elite group has bad circumstances...I would imagine if the competition is kept minimal, the elite group will become more of a "reward" than a goal.

~requiem
requiem
Posts: 42
Joined: Thu Aug 07, 2003 10:27 pm
Location: koko ni inai doko demo ni iru...

Post by requiem »

Ugh...it didn't occur to me until after I posted that I didn't log in...sigh :cry:

Well, just so you know, it was really me.

Edits:

When I refer to the avg. review rating, it's implied as "avg. approval rating" in the "myreviews" section for each user. The number changes constantly (as it did for the sake of Gambit)...

Also, when I meant "trusted reviewers would have their reviews listed alphabetically or chronologically, I'm also implying that they would be separate - perhaps the first ones to be displayed in the review section - and be organized collectively as I stated.
Last edited by requiem on Fri Nov 21, 2003 5:16 am, edited 1 time in total.
nwa
AniDB Staff
Posts: 585
Joined: Sat Jun 07, 2003 10:51 am

Post by nwa »

well...I'm also backing up requiem's suggestion as some of the ideas there are mine :lol:

still..this remains a suggestion like all the others...basically as I see it...ppl are starting to be against the review rating system cause of the stupid fanboys who vote no without any real reason...
so as said...Gambit's Berserk review would move to the top since he has the "status" and that's all that really matters isn't it :P
and as the limit is 200 user votes...I have 198 so I wouldn't reach it just yet..so there are only 3 uber-reviewers right now :D
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato »

Trusted reviewers could be a nice addition. :)
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

Elberet wrote:All users who currently have submitted at least, let's say 5 reviews and have a minimum total word count of at least 4000 words and an average approval rating of at least 5.5 are considered "trusted" reviewers. When a new review is posted, the db randomly selects 19 trusted reviewers and one mod.
I like that idea, but I have a few things to say about it.
I like the 'randomly picking' of trusted reviewers. But I don`t like the '5 reviews make a reviewer trusted'. What requiem says is more to my liking. 200 votes and at least 5 reviews with an average of at least a 5.5 makes someone trusted.
imo 100 votes and 5 reviews with an average of 6 will already be enough. Now this would be nice, but currently there are a lot of people voting for reviews who are only agreeing with the praising of the anime, not the review itself, which makes the rating not really usable for the moment.
requiem wrote:-How to determine if a trusted reviewer won't spam. I suppose they are "trusted" for a reason - however, I can only say that there aren't enough guarantees for everyone
Even though it might sound a bit nasty, I would suggest that the moment someone gains the state of trusted, his/her name would be shown to a bunch (5? 10?) of trusted reviewers who vote to accept new member as a trusted reviewer. This way you`ll also have a bit of control about reviewers who have a good rating, but bad reviews due to the way how people vote for opinions, not for reviews.

If that person gets rejected, he`ll get a second chance as soon as he has made 5 more reviews (considering his rating is still over 5.5) and will still be able to gain the trusted reviewer-state.

I don`t see how the mod would have to have 2 votes, after all, a mod is also just a regular 'trusted reviewer' according to the system. I wouldn`t want my vote to have more weight than the votes of other people. After all, in a community like this I`d like to see it more democratially ;)
Besides, all AR members are mods, which is pretty useless, since none of them is active in AniDB except for EXP and BennieB - and they don`t bother with reviews. So the 'mods' should only apply to AniDB active mods, not all mods. And in this case, you might even look at the mods who actually care for reviews. I doubt EXP, SilverWordz or ElNipp bother reading reviews at all, so they`d never vote for them and would be a useless vote.

About an elite-group, as long as there is no competition, I don`t mind having an elite-group. It`s more of an open group, keeping an eye on the quality. As soon as there is someone who is able to do the same by writing good reviews, he`s supposed to be allowed to the group as well. If you`ll only become an elite if you do your work really good, in what way is that bad? That`s why I don`t mind having an elite-group, if you want to call it that. I`d rather call it quality control instead.

Recommendations should be an option too, considering the current rating system. Someone who makes really good reviews, but gets a horrible rating since his/her opinion is the opposite of all others, is still a good reviewer. As soon as a trusted reviewer notices that, (s)he should be able to recommend that person to the other trusted reviewers, so that person could be accepted if (s)he applies to all rules (xxx votes, x reviews) but then the rating would be overlooked and the acceptation would be the responsibility of a random number of trusted reviewers.
Last edited by Gambit on Sat Nov 22, 2003 9:25 am, edited 1 time in total.
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato »

I don't like now that Trusted Reviewers are selected by Trusted Users... Who will chose Trusted Users?... and why they are trusted?
nwa
AniDB Staff
Posts: 585
Joined: Sat Jun 07, 2003 10:51 am

Post by nwa »

DonGato wrote:and why they are trusted?
because after 200 user votes their average review rating is above 5.5
DonGato
Posts: 1296
Joined: Sun Nov 17, 2002 9:08 pm
Location: The Pampas, The land of the Gaucho!
Contact:

Post by DonGato »

nwa, you're talking about trusted reviewers that I don't object... and I'm talking about trusted users that can vote trusted reviewers. Or did I get the wrong idea about Gambit post?

Trusted reviewers are ok when they are rated by all the user base, not a minimal 'elite' group. ;)

Elite = I don't like (even the name)
Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

Post by Elberet »

I would guess he merely slipped up a bit and "trusted users" is actually identical to "trusted reviewer".
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

Elberet is right, I just used the word Trusted User instead of Trusted Reviewer. I`ve changed all users -> reviewers now.
Sorry for the confusion :oops:
DonGato wrote:Trusted reviewers are ok when they are rated by all the user base, not a minimal 'elite' group.
That`s not much different from what I mean; the trusted reviewer will still be chosen by all the users, but the trusted reviewers will have to look into that potential new trusted reviewer. It is possible that he just persuaded people to vote good for him to gain that status, or because people like the praising (while his reviews are not good at all). Then a number of random chosen trusted reviewers will check if the person is really worth to become a trusted reviewer. This way you`ll prevent cheating.
I don`t think this makes the trusted reviewers an elite, because the trusted reviewers would like to welcome more people - after all, the more people who are able to make good reviews, the better :) But that only goes for people who actually make good reviews. It`s no use having this system when just everyone with reviews where they`re only praising an anime to get good ratings can become a trusted reviewer. imho.

Note to self: don`t post such stuff in the morning when you`re in a hurry to leave.
exp
Site Admin
Posts: 2438
Joined: Tue Oct 01, 2002 9:42 pm
Location: Nowhere

Post by exp »

interesting discussion, please keep in mind that someone will have to implement that :P

btw. a simple revoking of your vote for a review is now possible.

BYe!
EXP
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

That`s nice at least :) I believe it was even the origin of this thread :P
Elberet
Posts: 778
Joined: Sat Jul 19, 2003 8:14 pm

Post by Elberet »

Then someone please rename the thread into "Review Moderation & Trusted Users/Reviewers" please. ;)
Gambit
AniDB Staff
Posts: 555
Joined: Sun Oct 06, 2002 11:21 am

Post by Gambit »

* Gambit uses his new mod-forum powers and changes the threadtitle.

Damn, it worked 8O
Locked