Make anime hint factor in temporary votes [DONE]

[Guest]

Alright, so I decided to be a goody-goody and change all of the permanent votes for the shows I hadn't seen all of to "temporary" votes. To my dismay, this made anime hints pretty much useless, as the thing is now recommending animes that I've already watched and disliked.

[Gambit]

An option 'Hide Temp votes by me' would be useful there instead.
Or you just act like I do and vote only for series you`ve finished watching (which is pretty much the only reason I see for a vote anyway ... I disagree with temp votes, but I guess there`s no way to make people follow my own principle).

[wahaha]

To me, a temp vote means either:
a) "It's ongoing and I'm watching it. My opinion is: (vote)" - in that case it's useless to show this anime as a recommendation.
b) "I watched some (but not all) eps of the anime and discarded it." - which makes it even less helpful to re-recommend this anime

[ For case b), a function that explicitly ignores one's temp-votes might indeed be useful for finding out personally underrated animes. ]

But at least by default anime-hint shouldn't distinguish between temp- and permanent-votes, and I second that

[Gambit]

That`s why I indicated that a button 'Hide Temp votes by me', so you can see if a series is highly recommendated while you dropped it after 1 episode. It might be an indicator to regrab it and give it a second chance after all
That button would give YOU the decision whether to include them or not, which seems more useful to me

[egg]

Also, Temp Votes should be used in the logic of finding matches. I did some testing, a permanent vote does change the hints, but a temporary vote of the same value does not. This is partucularly useful in the cases that match wahaha's b) option and other individuals had dropped a particular anime and had given it a low score, but if their scores are temporary those low scores wouldn't count and it might get recommended to you.

[yidaki]

I voted a temporary 2 on AMdriver after having seen only one episode, and I'm not going to see any more either, it really sucks.
I see that both as a warning to others, and (if it only worked) as a hint to the hint function that I dont want stuff like AMdriver

[egg]

ATM the anime hint page tries to get a list of possible anime recommendations by first comparing all YOUR anime votes with the anime votes of all other anidb users.
This is used to create a weight (or score) for each user in relation to you.
The weight is a number calculated by adding 10 for each anime the other user voted exactly the same as you did. And adding 5 is the other user voted only one rating different (i.e. 8 instead of 9).
If the difference between votes is 3,4,5,6,7,8 or 9 a value of 2,4,6,8,10,12 or 15 is substracted from the weight for that user.

The resulting weights for the users are compared to the min. weight value from the input box. All users with a weight < min. weight are ignored from here on.

Now the votes of all remaining users for animes you haven't voted fore are checked. A score per anime is generated by adding the weight of the user*((vote-5)/5) per user vote.

Here is a new proposal for voting logic:
1) Using Temporary Votes for User Weight
For calculating a weight for a user, although it is important to look at temporary votes, it does not seem to be appropriate to count a temporary vote like a permanent vote.

For the user rating I would suggest that if at least one of the votes (yours and/or the other users) is temporary, you divide the modifier by 2. That is if you both voted the same, but one of them was a temporary vote, then it would add 5 instead of ten. This way there is still a modifier, but it does not make as big of a difference.

2) Using Temporary Votes for Anime Score
Similar to #1 divide the weight of the user with temporary scores in half like #1. This way they have an affect on the final score, but do not count as much as permanent votes.

3) Getting Best Rated Anime
One thing that I have noticed is that the anime hint will tend to suggest animes that have more users rather than other animes that might be higher rated, but by fewer users. An anime with a lot of people rating it a 7 can get a higher score than one with a few users with similar voting histories rating the anime as 10. This ends up becoming a self fulfilling process if a lot of people use the anime hint to determine what they want to download.
Here is an extreme example:

Code: Select all

User  Weight  A1 Vote  A2 Vote  A3 Vote
----  ------  -------  -------  -------
u1    200     10                7
u2    150      9                7
u3    100                       8
u4    100              10       8
u5    100              10       8
u6    100              10       9
----  ------  -------  -------  -------
Score         320      300      400

Now in this scenario, I would probably prefer to have A1 listed as the best hint, then A2 and finally A3. Unfortunately since A3 has more users though, it comes out with a higher score.

I tried a number of things to try to make the score independant of the number of users, but still give higher scores for users with higher weights. Here is what I came up with:

Code: Select all

Variable Definitions:
puw = Weight of a user with a permanent vote
pv  = permanent vote for a user for that anime
tuw = Weight of a user with a temporary vote
tv  = temporary vote for a user for that anime
nu  = number of users that voted on that anime

current logic is:
sum(puw * ((pv - 5) / 5))

Adding Temporaries based on #2 would be:
sum(puw * ((pv - 5) / 5)) + sum((tuw / 2) * ((tv - 5) / 5))

Now putting in an averaging to make it independant of the number of users:
(sum(puw * ((pv - 5) / 5)) + sum((tuw / 2) * ((tv - 5) / 5))) /
(sum(puw) + sum(tuw / 2))

Finally take into consideration the overall weight of the users doing the voting:
(sum(puw * ((pv - 5) / 5)) + sum((tuw / 2) * ((tv - 5) / 5))) /
(sum(puw) + sum(tuw / 2)) *
((sum(puw) + sum(tuw)) / nu)

Note, the final temporary weights are NOT halved, this would artificially reduce the score.

I know this is a little complicated, but overall, I think it is fair. I have run it through a few scenarios and it appears to work the way I think it should.

BTW, in the example above, using this formula, the scores would be 160 for A1, 100 for A2 and 67 for A3.

Any comments (PM me if you do not want to post here) would be appreciated.

[exp]

i just updated animehint slightly.
and i also updated the anime hint decr @ forum
http://www.anidb.net/forum/viewtopic.php?p=1021#1021
it was kinda out dated

note: this does not mean i do not like your solution i just think we should discuss it a little. and the newly introduced avg. scale value will also allow ppl to play a bit with the new feeling animehint will get with your requests.

also note that only your temp. votes are taken into account atm, not the temp votes of other users.

BYe!
EXP

[Elias]

The main difference is dividing score by number of users scoring for anime? This produces now strange results in hint, placing on top animes very unpopular.

Example:

Code: Select all

User  Weight  A1 Vote  A2 Vote  A3 Vote A4 Vote
----  ------  -------  -------  ------- ------
u1    200     10                7 
u2    150                        7      10
u3    100                        8 
u4    100                       8 
u5    100              10       8 
u6    100              10       9 
u7    100              10       9 
u8    100              10       9 
u9    100               9        9 
----  ------  -------  -------  ------- -----
Avg          10       9.8               10 

A1 will get top score, even if it was scored only by one user, and hundreds other even not voted for this title. And there will be A4 voted by only one another user to 10, A5 voted by another, and many more such titles will take top of list, and A2 - rather popular with high, but little below 10 score will be far away, and A3 - most popular will be never appear. So animes having low number of users should have been downed final score to avoid selecting only rarest title.
Maybe like this
<5 users - -2.5
<10 users - -2
<15 users - -1.5
<20 users - -1
<25 users - -0.5

Instead of number of users better will be percent of total number users taken to hint (<5%, <10%).

And from other direction, animes with large number of users should been pushed up a little.
>100 users +0.25
>200 users +0.5

[exp]

I am not sure i get your point.
As I see it you're talking about the avg. scale parameter which can already be set freely by any one using anime hint.
At least it should take care of your "only one 10 vote but still first ranked" example bc with a value 0<x<1 the score of an anime will also increase with the number of ppl voteing for it.
Or did i miss something there?

BYe!
EXP

[Elias]

Previous method (without dividing by number of votes) granted animes with more number of votes, even if those votes where not highiest (popular animes are higher than less popular but with better votes),
now when dividing by number it prefers animes which have very low number of users voting, but voting high (so if there will be anime scored by user which has high weight, and if this user votes 10, this anime will be as first on the list, even if none of other users voted for this anime).

If it is really true:
animescore = score/(numberofvotes*avg scale)
than avg. scale does really nothing.
Lets compare this example: 10 users with height 100 each, only one of them voted for anime A1 with 10, and for anime A2 5 voted 10 and 5 voted 9.
Its gives:
score for A1= 100 / 1 * avg = 100 /avg
score for A2 = 900 / 10 * avg = 90 / avg
Changing of avg changes final score, but in same manner for all animes, so position on list will be the same.

So i wanted to make final place of anime depending not only from vote, but also in some way from number of users scoring (if number of users is low, than it mean rather accidental high vote, if number of users scoring is high, than average score is more appropriate - and additionally this anime should get some boost from its popularity). Maybe i overtricked a little to much.

Maybe changing it to:
animescore = score/(numberofusers - avg.scale*(numberofusers-numberofvotes))
where numberofusers = number of users taken for hinting
or better numberofusers = max(number of votes for one anime)
For avg.scale=0, it gives:
score/numberofusers (like old method, only all scores lowered in same way)
For avg.scale=1:
score/numberofvotes (like new method, average score per vote)
And for values between gives something different, i have checked it for some examples and it looks maybe not ideal, but better than current.

[Gambit]

Personally I just tweak the min. weight until I have 10 (or a bit more) people left for the Hint. I tend to believe that those 10-20 people with the closest matches are more suitable to provide a good listing for me than if I take 500 people into account which are not so close to my taste than the 10-20 who are.

But, Elias does have a point ... if the #19 on my list has voted 10 as the only person on my list for an anime, it`ll be listed as the #1 anime with an average of 10 (if I understood it correctly). But, with limiting this with the top 10-20 people, these 'flaws' should be removed pretty easily since those users are people who have pretty/very similar taste compared to you. Thus, if one of them likes an anime a lot than he`s the only one with a 10, it might as well be something you like as well. I mean, he`s not in your top 10 for nothing

Or is that just me?

[egg]

[Edit: I just tried it out, and it is working differently than I stated, for 0.25 the scores are about 4 * the value of 1, but different animes are listed... There is something else going on that I am missing. My statements were made more by the formulas posted than actual experimentation (which is difficult to tell because you don't know all of the scores that are used in the calculations... [Edit Edit: Actually things do work the way I expected, except when there is only one vote, see post below for more details.]]

I think Elias has the wrong logic in the table. He is assuming that the votes are just averaged. In actuallity, the value that is averaged is the modified vote (vote-5)/5 which is a value -0.8 to 1 times the weight, so higher rated users will get higher scores.

Here is a table with my calculated scores [updated for single votes]:

Code: Select all

User  Weight     A1 Vote  A2 Vote  A3 Vote  A4 Vote  A5 Vote
----  ---------  -------  -------  -------  -------  -------
u1    200        10                7                 10
u2    150                          7        10       10
u3    100                          8
u4    100                          8
u5    100                 10       8
u6    100                 10       9
u7    100                 10       9
u8    100                 10       9
u9    100                 9        9
----  ---------  -------  -------  -------  -------  -------
avg. scale 0.00  200      480      640      150      350
avg. scale 0.25  200      384      284      150      700
avg. scale 1.00  200      96       71       150      175

Now if you like the old system, which promotes more popular animes, just set the avg. scale to 0, and you can see A3 gets the highest score, by far.

The new system overall lists them by highest weighted top votes, so if the user you match the best votes a 10, then it shows up at the top. A user with a lower weight voting 10 will get a lower score than the higher ones.

A3 - most popular will be never appear.

Why do you say that? It will have a lower score (which I think it should) because it has lower votes, but it should still be there. BTW, if you want popular rather than higher rated animes, you can use 0 in the avg. scale and that is equivalent to the old logic, and that favors popular anime.

Also:

Now the votes >=8 of all remaining users for animes you haven't voted for are checked.

I believe this means that it will only calculate scores for animes that have at least 8 votes. This means that in the above example, A3 would be the only anime that would have a hint.

At least it should take care of your "only one 10 vote but still first ranked" example bc with a value 0<x<1 the score of an anime will also increase with the number of ppl voteing for it.
Or did i miss something there?

I think you missed something there (although I think it is a great improvement). Basically there end up being two different possible scores, 0 for the old logic and any other number for the new logic. If you use a non-zero value, then the scores are always in the same order and have the same ratios, (in the above example A1 score/A4 score = 1.33 regardless of the avg. scale). The reason is the avg. scale just ends up being a multiplier that gets multiplied by every score.

animescore = score/(numberofvotes*avg scale) =
score/numberofvotes*1/avg scale

So, if the average scale is 0.25

score/numberofvotes*1/avg scale =
score/numberofvotes*1/0.25 =
score/numberofvotes*4

[Edit: there is an exception, animescore=score if numberofvotes == 1]

Although this does a great job of averaging the weights and the scores, it can penalize an anime if a lower weighted user votes with a 10 (see A5 compared to A1). The thing is with a minimum of 8 votes, this shouldn't be too much of an issue.


With the current system, I would change the avg. scale to a drop-down box scale by: [popularity|highest votes] and that is equivalent to the current 0 and 1 avg. scales.

I also think that the formula needs to be changed to NOT penalize an anime for getting an additional 10 vote from a lower weighted user, but I need to play with the formulas a bit before I can make a recommendation.

[Elias]

Personally I just tweak the min. weight until I have 10 (or a bit more) people left for the Hint.

When is only 10 people left, better method may be just watching through their votes list.

I also think that the formula needs to be changed to NOT penalize an anime for getting an additional 10 vote from a lower weighted user, but I need to play with the formulas a bit before I can make a recommendation.

I agree. But finding good formula may be little problematic.
For avg=0 it should give const1 * score, for avg=1 const2 * score/votes

Till now the best i have find is:
score*((1-avg)*votes*votes + avg*users)/votes

For your example it started working different than with avg=0 or avg=1,
when avg was about 0.75 (results (positions on list) were different). But maybe for not so extreme examples will it work better with lower values.
I will try finding something better.

[egg]

OK, I have gone through with actual numbers from the hint and confirmed the logic, there is an exception to the rule...

If there is only 1 vote, it ignores the avg. scale and lists the same value (if the weight was 200 and the vote was 10, then the score would be 200 regardless of the avg. scale). This has the affect of changing the order. When the avg. scale is a low value, a number of the other scores woule be above 200, so it would be towards the bottom. When the avg. scale is high, the other scores would be less and the single vote will show up at the top.

So in this case, if the 8 minimum votes worked, then this would not happen.

My suggestion for making the scale useful and to reduce the penalty of additional 10 votes reducing the score is:
animescore = score/(1+numberofvotes*avg scale)

This makes the scale a true scale, a value of 0 would be like the old logic and highly favor the number of users that have an anime. A value of 1 on the other hand would be like the new logic and would end up with lower scores only if there are a few users with that anime. Adding one to the denominator doesn't make much of a difference if there are 10 users with the anime, but makes a huge difference if there is only one user.

Going back to the previous example, here are the scores:

Code: Select all

User  Weight     A1 Vote  A2 Vote  A3 Vote  A4 Vote  A5 Vote
----  ---------  -------  -------  -------  -------  -------
u1    200        10                7                 10
u2    150                          7        10       10
u3    100                          8
u4    100                          8
u5    100                 10       8
u6    100                 10       9
u7    100                 10       9
u8    100                 10       9
u9    100                 9        9
----  ---------  -------  -------  -------  -------  -------
avg. scale 0.00  200      480+     640*     150      350
avg. scale 0.25  160      213+     197      120      233*
avg. scale 0.50  133      137+     116      100      175*
avg. scale 0.75  114+     101      83       86       140*
avg. scale 1.00  100+     80       64       75       117*

As you can see, the first (*) and second (+) place animes change as you go down the scale, so then people would just have to find the scale that suits their needs.

If you want the denominator to be no larger than the number of votes you can do this:
animescore = score/(1+(numberofvotes-1)*avg scale)
but I prefer the previous method after playing with the numbers.