Friday, November 25, 2011

Re: rating manipulations on www.vim.org have taken place

On Fri, 25 Nov 2011, Marc Weber wrote:

> Excerpts from Benjamin R. Haskell's message of Fri Nov 25 04:52:14 +0100 2011:
>> That's what you could do if you're assuming that the ratings are for
>> the benefit of the plugin authors. I'd rather the ratings be useful
>> for users.
> Let me explain it to you: Its plugin author writing scripts.
> Thus its plugin authors serving the needs of users.
> If you setup a simple efficient feedback loops (the way github does)
> you'll get nice system moving forward improving on its own.

If you want a feedback loop the way github provides, use github. That's
what a lot of script authors do (including you, from what I recall).

Vim.org has a ratings system.


> Its like telling to babyies: "You're not useful to me - so I don't
> even try to teach you standing up and how to walk."
> You don't see that it's children serving your needs when you're old
> (unless you suicide). From this example its easy to understand that
> "little boys/girls" need guidance to be helpfull to the community.
>
> That's my understanding about open source.

Following your analogy, plugins are authors' babies, not users' babies.
Users have no connection or obligation to plugin authors. If someone
else's baby can't walk, it's not my job to teach it, it's theirs.

Providing feedback is nice, yes. And the right thing to do. And the
reason open source succeeds when it does. Back to the analogy: society
as a whole is better off when people are willing to help other people's
babies.


>> without leaving a comment, they'll likely not leave a rating.
> Then they don't understand feedback loops. Then they should not vote.
> Becaues its *you* benefting as user if you read "downvoting because
> script-id 326 gets the job done much better".

Vim.org's ratings are more like amazon.com's star ratings. Comments
aren't required with star ratings, and the reviews that accompany the
ratings are much more useful than the ratings alone. So you're right
about that: rating + feedback is better. But, both serve a purpose.


>>> Its ok to downvote for a reason such as "contains executable". But
>>> its bad to have people provide negative rating and not knowing why!
>>> The website should allow some minimal feedback.
>> Allow? Yes, great idea. Force? No.
> I agree its debatable. Why do I prefer comments? Because I can
> validate and proof them. I can't judge votings without comments - so I
> surely am in favour of forcing comments.

Yes, I also agree it's debatable, and that comments are preferable.

But, as I mentioned in another thread recently, I'm probably the wrong
person to ask about how to improve vim.org's script-hosting, since I
rarely install anything directly from it.

Bringing this back to a bigger picture: I think it would take a
tremendous amount of effort to improve the scripts site to the level of
something like github. And, personally, I think that effort isn't worth
it, since the option already exists to simply use github. Even the
effort to maintain the scripts site's current level of usefulness is
increasing (cf. recent increase in spam scripts, and the downvote issue
from this thread).

I'm somewhat curious as to whether people share my opinion of vim.org's
script hosting. Whenever it was first deployed, I'm sure it filled a
need. But, today, there are any number of free, easy-to-use places to
host scripts (github, bitbucket, Google Code).

Does the scripts site have much reason to continue?

I guess one clear benefit is that the scripts site is a nice way to have
a "stable" release, vs. a "development" release on github. So, maybe
for that reason alone it makes sense to continue to devote effort to
maintaining/improving the site.


>> He wrote a message stating it was fixed on Sept. 2.
> I've found the followings scripts which got dowvoted after Sept. 2.
> I searched for scripts having 10 or more down votings in sequence
> since Sept. 2.
>
> SCRIPT_ID / downvote count / time range
> 3695 / 40 (2011-10-23 from 09:09 till 09:23)
> 2140 / 117 (2011-10-29 01:34:49 - 2011-10-29 02:07:53)
> 1435 / 100 (2011-10-23 08:28:59 - 2011-10-23 10:05:42)
> 670 / 17 (2011-10-23 09:05:14 - 2011-10-23 09:09:59)
> 294 / 191 (2011-10-23 07:43:33 - 2011-10-23 11:43:03)
> 122 / 134 (2011-10-23 08:22:55 - 2011-10-23 09:07:22 )
>
> SCRIPT_ID / NAME (AUTHOR) => voting result
>
> 3695: commentary.vim : Comment stuff out; takes a motion as a target (Tim Pope) => 28/103
> 2140: xoria256.vim : Soft pastel gamma on dark background, same appearence in {,g}vim (Dmitriy Zotikov) => 249/245
> 1435: HiMtchBrkt : withdrawn (Charles Campbell) => -36/131,
> 670: VisIncr : Produce increasing/decreasing columns of numbers, dates, or daynames (Charles Campbell) => 785/648
> 294: Align : Help folks to align text, eqns, declarations, tables, etc (Charles Campbell) => 1452/712
> 122: Astronaut (Charles Campbell) => -57/169
>
> 4 times Charles Campbell
> 1 time Tim Pope
> 1 time Dmitriy Zotikov
>
> comment: I cleaned up commentary Aug 20 21:07:23 2011 so it happened again.
> As example I attached relevant data for 3695, see below. And even for
> www.vim.org I can't believe users voting the same plugins every 4
> secs?
>
> If Bram fixed the issue on Sept 2. then its very likely that someone wrote a
> script or some other magic is going on I can't imagine - maybe search engines
> do follow forms as well?

Yes. With these new data post-2011/09/02 it seems clear the problem
wasn't simply the downvote link being crawled. (How do you have access
to that data?)


> If so why didn't it happen more often?

Before changing to POST, the effect would only be from:

1. The bad link gets spidered (so, 1 downvote)
2. Many people search for that plugin, then click the bad link (then,
many downvotes)

I'm not sure, though, why it would still happen after the change to
POST.


> http://googlewebmastercentral.blogspot.com/2008/04/crawling-through-html-forms.html
> .. says it may happen and might have happened in the past.
> A fix would be robots.txt for google if this was the case.
> But then - why should google bot run that many queries if there are 3 options only ?!
> Doesn't make sense to me. Passing invalid data to web apps will yield "internal
> server errors" very often.

One potential issue is that Googlebot started crawling more forms with
method="POST" recently.

http://googlewebmastercentral.blogspot.com/2011/11/get-post-and-safely-surfacing-more-of.html

But, there doesn't seem to be a nice way to prevent it with the current
setup, because the voting page is the same as the script information
page. If instead of using the same page, the form with name="rating"
used action="/scripts/rating.php", then rating.php redirected the user
back to the /scripts/script.php?script_id=NNNN page, /scripts/rating.php
could be added to robots.txt.


> Additionally the following plugins still have 20 down votes in sequence within 6 hours:
>
> [trimming data for scripts 515, 2002, and 3695]

Do you have a way to correlate the voting data with access logs that
contain the User-Agent? It'd be interesting to see if it lined up.

--
Best,
Ben

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

No comments: