Sunday, November 27, 2011

Re: rating manipulations on www.vim.org have taken place

Excerpts from Bram Moolenaar's message of Sun Nov 27 20:40:23 +0100 2011:
> It's going to be very difficult to automatically distinguish these
> down-ratings from what happens to a really bad script that gets posted
> and deserves these down-ratings.

Well - close to impossible because its we who must decide on what should
be "valid" and not - you can't distinguish bots from humans without
spending hours on analyzing behaviour of HTTP clients - user agent
strings can be changed trivially.

> Since it doesn't happen frequently, we can write a simple PHP function
> to enter a date range and script number and remove the negative votes in
> that date range. Simply removing the negative votes from the table has
> the problem that the total count for the script still has to be updated,
> which is a hassle to do manually.
And the sum of total counts. Of course writing a PHP interface for it
would be possible. But how to catch those issues?

You can trivially write a bot which does this kind of voting:

$r = rand(100)
if $r < 8 then vote_neutral()
if $r < 12 then vote_up()
else vote_down()

and it'll be very hard to recognize this kind of abuse.

If there is a bot - bots have time. They can vote slowly submitting
5 votes a day.

Alternatives? Force login? ...
Probably not an option - unless we think less but higher quality votes
are better.

I'd replace that voting system by usage counts if possible. Eg allow
users to submit the plugins they have installed. This would also reflect
the current state of popularity rather than historical data.
If we asked authors to add simple

" vim_script_voting_id: MY_PLUGIN_NAME

~/.vim/plugin/*.vim could be grepped and submitted - eventually forcing
logins.

If abuse happens - users could start to "follow" usage counts of other users.
Then abuse is close to impossible. Eg you like plugins of authors A,B,C
and user X then only follow their plugin usage counts.

I don't know how well such would work in practise - its an idea only.

In an older thread we've already discussed the option to decay votings
slowly - new version may no longer suffer from issues of older versions.

Marc Weber

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

No comments:

Post a Comment