First time here? Check out the FAQ!
4

could askbot use percentage-based values for karma (and other) thresholds?

I've been thinking again about the karma thresholds for user permissions, and the numerical values attached to badges . . . I know these are configurable, but I wonder if it would be useful for the defaults to be percentages of the total site activity, rather than specific values. Then new communities using askbot wouldn't have to re-set the thresholds as they grow (if they even bother to lower them in the first place). Of course there would have to be some absolute minimum values, for when the site is brand new, but I think these should be small.

Here's one proposal: Let U be the top 50% of users (that is, the set of their karma values). Then set the following thresholds to gain the following priveleges:

  • open and close own questions: 100 or minimum of U
  • retag other's questions: 200 or top 20% of U
  • edit community wiki questions: 200 or top 10% of U
  • edit any answer: 400 or top 5% of U
  • delete any comment: 1000 or top 2% of U

Since a user may fall outside of a given percentage (as the community grows), there should be some (lower) threshold to lose each privelege. It could be zero, if you think that people should never lose a privelege once they've gained it, but the minimum value of U would probably be a more reasonable threshold. Also, U could be recalculated monthly rather than, say, daily or hourly, to make the numbers fluctuate less. Or, if you want to take this idea to an extreme, U could be recalculated every t days, where t is 10% of the site lifetime.

A similar principle could be applied to the badges: getting the "nice question" badge could require having your answer upvoted by 10% of the total number of users.


As a last remark, I think that the particular thresholds should still be configurable; the new feature here is being able to set them by percentage, rather than by absolute number only, and then setting the defaults that way (better for new askbot communities). (And, by the way, I haven't actually looked at a single line of askbot code, so forgive me and let me know if this is already possible. :)

niles's avatar
808
niles
updated 2010-10-18 09:57:23 -0500, asked 2010-10-18 09:55:44 -0500
edit flag offensive 0 remove flag close merge delete

Comments

I think for the re-tag operation, instead of removing tags, there could be a second row of moderator added tags. If there is a duplicate (similar name), perhaps it could be left in place, but grayed to indicate change recommended.
ccanonc's avatar ccanonc (2010-10-18 10:59:01 -0500) edit
Is the delete operation really a hide operation? It's good to keep data around for auditing purposes.
ccanonc's avatar ccanonc (2010-10-18 11:00:39 -0500) edit
For the edit-any-answer operation, I think the edited should be highlighted and annotated so that it is clear that it is a moderator change. Otherwise, users will complain about things which they did not say/type.
ccanonc's avatar ccanonc (2010-10-18 11:02:02 -0500) edit
add a comment see more comments

3 Answers

1

This is a very interesting idea, but you would need to exclude a significant subset of the user base when calculating U.

Based on my own observations of various Q&A sites including StackExchange, askbot.org and my hosted site, the median karma/reputation amount (i.e. minimum amount of karma to be in the top 50%) tends to be 1.

For example, on askbot.org, as much as 60% of the user base have only 1 karma — it would take a lot of participation by people who posted once and never returned to raise the median karma above 1.

On really successful sites such as StackOverflow, the ratio is even less favorable; over 92% of StackOverflow users have 1 reputation (see this post for an explanation of how to interpret the chart).

To work around this problem, you could simply change the calculation to be the median karma of users who have more than 1 karma.

For askbot.org, if you were to exclude all 1-karma users from the median calculation, you'd end up probably with a U value between 50 and 60 (as of the time this was posted, although it would be interesting to see if this changes over time).

todofixthis's avatar
1.3k
todofixthis
answered 2012-07-28 11:09:30 -0500, updated 2012-07-29 10:49:52 -0500
edit flag offensive 0 remove flag delete link

Comments

The problem with reassessing karma limits that way is that with time it will be harder and harder for the new users to reach the barrier.

Evgeny's avatar Evgeny (2012-07-28 11:41:01 -0500) edit

I think you're correct on that, but depending on how quickly new users join vs. established users gain karma, the increase might actually be in line with what an admin might assign manullay. For example, looking at StackOverflow's stats after several years of (arguably massively successful) operation, the median reputation score of all > 1 rep users is only between 500-1000.

todofixthis's avatar todofixthis (2012-07-29 09:41:48 -0500) edit
add a comment see more comments
2

This can be experimented with, the easiest approach will be to re-assess the barriers automatically about once a day. It's a lot easier than hard-coding the logic into a whole bunch of tests and assertions of type assert_user_can_do_X.

If anyone is interested to make this work - one can write a management command that analyzes the karma of users and adjusts the settings accordingly.

Management commands are located in directory askbot/management/commands.

Available minimum rep settings are in askbot/conf/minimum_reputation.py.

Say if you want to update setting MIN_REP_TO_VOTE_UP you could code like:

#import askbot live settings
from askbot.conf import settings

settings.update('MIN_REP_TO_VOTE_UP', some_new_value)

The management command is then run via cron job:

python manage.py reassess_karma_barriers

It's best to try and see how this will work. It's not obvious how people will react to having some privileges come and go based on their "karma" standing relative to others.

Evgeny's avatar
13.2k
Evgeny
updated 2010-10-18 14:33:05 -0500, answered 2010-10-18 14:29:50 -0500
edit flag offensive 0 remove flag delete link

Comments

I'm sure over time, incrementally you will find a close to optimal solution.
ccanonc's avatar ccanonc (2010-10-18 14:42:31 -0500) edit
add a comment see more comments
3

In addition to my comments above, I think another useful metric of positive activity is a moving average of the first derivative of karma with respect to time, and possibly other variables with respect to time (Recent Karma Velocity!). A moving average, however weighted, has some advantages (as long as you have that sliding window of time, it keeps people incentivized to keep contributing.) Perhaps a first step towards this would be to show the moving average of the first derivative (with site configurable time interval of course) in the karma history. Then perhaps draw up some proposals based upon observed behavior.

ccanonc's avatar
363
ccanonc
answered 2010-10-18 11:06:35 -0500
edit flag offensive 0 remove flag delete link

Comments

Right now I'm wondering how page views are counted. Questions with high view counts have a corresponding proportionality of popularity, however, it is often the answers that draw the views(!). =)
ccanonc's avatar ccanonc (2010-10-18 11:47:06 -0500) edit
The views are counted in a "humanized" way. If you've seen the page as it is your consequent views are not counted. If I modify the page and you look at it again, that view will be counted. Also robot visits are not counted as far as it is possible.
Evgeny's avatar Evgeny (2010-10-18 12:59:09 -0500) edit
Are non-logged-in users counted?
ccanonc's avatar ccanonc (2010-10-18 13:16:28 -0500) edit
Yes, anonymous users are counted per-session.
Evgeny's avatar Evgeny (2010-10-18 13:42:59 -0500) edit
So, if someone cleared their cookies, they could drive up views? wget could be directed to do this infinitely.
ccanonc's avatar ccanonc (2010-10-18 13:54:46 -0500) edit
add a comment see more comments