Ask Your Question
4

could askbot use percentage-based values for karma (and other) thresholds?

asked 2010-10-18 09:55:44 -0500

niles gravatar image

updated 2010-10-18 09:57:23 -0500

I've been thinking again about the karma thresholds for user permissions, and the numerical values attached to badges . . . I know these are configurable, but I wonder if it would be useful for the defaults to be percentages of the total site activity, rather than specific values. Then new communities using askbot wouldn't have to re-set the thresholds as they grow (if they even bother to lower them in the first place). Of course there would have to be some absolute minimum values, for when the site is brand new, but I think these should be small.

Here's one proposal: Let U be the top 50% of users (that is, the set of their karma values). Then set the following thresholds to gain the following priveleges:

  • open and close own questions: 100 or minimum of U
  • retag other's questions: 200 or top 20% of U
  • edit community wiki questions: 200 or top 10% of U
  • edit any answer: 400 or top 5% of U
  • delete any comment: 1000 or top 2% of U

Since a user may fall outside of a given percentage (as the community grows), there should be some (lower) threshold to lose each privelege. It could be zero, if you think that people should never lose a privelege once they've gained it, but the minimum value of U would probably be a more reasonable threshold. Also, U could be recalculated monthly rather than, say, daily or hourly, to make the numbers fluctuate less. Or, if you want to take this idea to an extreme, U could be recalculated every t days, where t is 10% of the site lifetime.

A similar principle could be applied to the badges: getting the "nice question" badge could require having your answer upvoted by 10% of the total number of users.


As a last remark, I think that the particular thresholds should still be configurable; the new feature here is being able to set them by percentage, rather than by absolute number only, and then setting the defaults that way (better for new askbot communities). (And, by the way, I haven't actually looked at a single line of askbot code, so forgive me and let me know if this is already possible. :)

edit retag flag offensive close merge delete

Comments

I think for the re-tag operation, instead of removing tags, there could be a second row of moderator added tags. If there is a duplicate (similar name), perhaps it could be left in place, but grayed to indicate change recommended.
ccanonc gravatar imageccanonc ( 2010-10-18 10:59:01 -0500 )edit
Is the delete operation really a hide operation? It's good to keep data around for auditing purposes.
ccanonc gravatar imageccanonc ( 2010-10-18 11:00:39 -0500 )edit
For the edit-any-answer operation, I think the edited should be highlighted and annotated so that it is clear that it is a moderator change. Otherwise, users will complain about things which they did not say/type.
ccanonc gravatar imageccanonc ( 2010-10-18 11:02:02 -0500 )edit

3 Answers

Sort by » oldest newest most voted
3

answered 2010-10-18 11:06:35 -0500

ccanonc gravatar image

In addition to my comments above, I think another useful metric of positive activity is a moving average of the first derivative of karma with respect to time, and possibly other variables with respect to time (Recent Karma Velocity!). A moving average, however weighted, has some advantages (as long as you have that sliding window of time, it keeps people incentivized to keep contributing.) Perhaps a first step towards this would be to show the moving average of the first derivative (with site configurable time interval of course) in the karma history. Then perhaps draw up some proposals based upon observed behavior.

edit flag offensive delete link more

Comments

Right now I'm wondering how page views are counted. Questions with high view counts have a corresponding proportionality of popularity, however, it is often the answers that draw the views(!). =)
ccanonc gravatar imageccanonc ( 2010-10-18 11:47:06 -0500 )edit
The views are counted in a "humanized" way. If you've seen the page as it is your consequent views are not counted. If I modify the page and you look at it again, that view will be counted. Also robot visits are not counted as far as it is possible.
Evgeny gravatar imageEvgeny ( 2010-10-18 12:59:09 -0500 )edit
Are non-logged-in users counted?
ccanonc gravatar imageccanonc ( 2010-10-18 13:16:28 -0500 )edit
Yes, anonymous users are counted per-session.
Evgeny gravatar imageEvgeny ( 2010-10-18 13:42:59 -0500 )edit
So, if someone cleared their cookies, they could drive up views? wget could be directed to do this infinitely.
ccanonc gravatar imageccanonc ( 2010-10-18 13:54:46 -0500 )edit
2

answered 2010-10-18 14:29:50 -0500

Evgeny gravatar image

updated 2010-10-18 14:33:05 -0500

This can be experimented with, the easiest approach will be to re-assess the barriers automatically about once a day. It's a lot easier than hard-coding the logic into a whole bunch of tests and assertions of type assert_user_can_do_X.

If anyone is interested to make this work - one can write a management command that analyzes the karma of users and adjusts the settings accordingly.

Management commands are located in directory askbot/management/commands.

Available minimum rep settings are in askbot/conf/minimum_reputation.py.

Say if you want to update setting MIN_REP_TO_VOTE_UP you could code like:

#import askbot live settings
from askbot.conf import settings

settings.update('MIN_REP_TO_VOTE_UP', some_new_value)

The management command is then run via cron job:

python manage.py reassess_karma_barriers

It's best to try and see how this will work. It's not obvious how people will react to having some privileges come and go based on their "karma" standing relative to others.

edit flag offensive delete link more

Comments

I'm sure over time, incrementally you will find a close to optimal solution.
ccanonc gravatar imageccanonc ( 2010-10-18 14:42:31 -0500 )edit
1

answered 2012-07-28 11:09:30 -0500

todofixthis gravatar image

updated 2012-07-29 10:49:52 -0500

This is a very interesting idea, but you would need to exclude a significant subset of the user base when calculating U.

Based on my own observations of various Q&A sites including StackExchange, askbot.org and my hosted site, the median karma/reputation amount (i.e. minimum amount of karma to be in the top 50%) tends to be 1.

For example, on askbot.org, as much as 60% of the user base have only 1 karma — it would take a lot of participation by people who posted once and never returned to raise the median karma above 1.

On really successful sites such as StackOverflow, the ratio is even less favorable; over 92% of StackOverflow users have 1 reputation (see this post for an explanation of how to interpret the chart).

To work around this problem, you could simply change the calculation to be the median karma of users who have more than 1 karma.

For askbot.org, if you were to exclude all 1-karma users from the median calculation, you'd end up probably with a U value between 50 and 60 (as of the time this was posted, although it would be interesting to see if this changes over time).

edit flag offensive delete link more

Comments

The problem with reassessing karma limits that way is that with time it will be harder and harder for the new users to reach the barrier.

Evgeny gravatar imageEvgeny ( 2012-07-28 11:41:01 -0500 )edit

I think you're correct on that, but depending on how quickly new users join vs. established users gain karma, the increase might actually be in line with what an admin might assign manullay. For example, looking at StackOverflow's stats after several years of (arguably massively successful) operation, the median reputation score of all > 1 rep users is only between 500-1000.

todofixthis gravatar imagetodofixthis ( 2012-07-29 09:41:48 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2010-10-18 09:55:44 -0500

Seen: 220 times

Last updated: Jul 29 '12