5

If I got it correctly SE tries to provide communities of experts on very specific topics which are able to provide qualitatively high answers to questions in the respective field.

But if a Beta of a proposed site is "healty" or not is among others measured in terms of visits/day and questions/day in absolute numbers. While qualitative measures like %-of-answered-questions and #-of-answers-per-question are good measures of a site of high quality, the inclusion of the quantitative measures on the beta stats might provoke the complete opposite:

Fixed measurements like "15 questions per day on average is a healthy beta" is maybe not reachable for a smaller community. Now lets say they nevertheless try to build a fruitful SE site with perfectly fine questions and answers, but as they see the red numbers of the quantitative measurements they get worried about the future of their site and participation drops, some might even start to ask useless questions, just to push the stats. Now somebody comes up with a proposal throwing together everything slightly related, just to build a critical mass of users (just look at the discussions about the AI, MachineLearning, etc. proposals to see what I mean). But even though the stats look better this time, the actual quality of the new site is lower, because it is just several communities using the same site without any overlap - every community lives on the site in its own tags, but because the site is so unfocused and messy because of throwing everything in one big pot it fails to attract the real experts for each field, who possibly would have participated to a more focused site in their field.

So what I'm trying to say is - I don't think it is productive to anchor the healthyness of a site to those quantitative measures, because they really depend on the absolute number of members of the community in the real world and not on the quality and actual "healthyness" of the site and might just scare off people and lead to unfocused proposals with lower quality.

And as those two measures don't seem to be crucial in the end anyhow, I propose to drop "visits/day" and "questions/day" from the beta stats before they do any more harm and introduce new measures which emphazise quality instead, maybe something like "number of tags without wiki" or "percent of reviewed questions". Maybe some of you have some good ideas of even better measures of quality.

2
  • You seem to be asking to drop one set of quantitative measures and take up yet another, rather than finding "quality" measures. I can line up a boatload of users with one crap answer that happened to get a tenuous upvote, and that doesn't prove anything about the quality of the site.
    – jonsca
    Jan 6, 2012 at 21:21
  • I don't think they are other example of "quantitative measures", as they are not connected to the size of the community in the real world, while "visits/day" and "questions/day" clearly are. But I'm not asking to use exactely these measures anyhow - they were only some first suggestions I could come up with. This post is about finding good quality measures and you are welcome to suggest better ones.
    – tobigue
    Jan 6, 2012 at 22:33

2 Answers 2

1

I don't think its necessarily bad to have quantitative measures of success. What does matter is the proper threshold for "success". Eventually - and I think this time has already come and gone - the sites passing into beta are going to be less trafficked sites. And at that time, I think SE will have to determine if having a cluster of smaller, quieter secondary sites is "worth it" to them, for whatever goals they have in mind.

Back when the Healthcare IT beta was starting to slow down, I posted this picture:

enter image description here

Thats a picture of the number of "Questions/day" for each of the sites in beta at that time. Only one of them was at the "healthy" threshold, and it was brand new - since then, it too has declined down into the mass of sites in the 1-3 range. I stated in that thread that by those metrics, the best a beta site can hope for at this point is "Okay", and most won't even make that.

What remains to be established is whether or not Stack Exchange wishes to keep these sites around, and in what form.

0

The solution cannot not be to remove those measurement quantities, but to change AREA51 requirements for smaller sites/topics. Think about this. You don't change much by measuring it another way. In the end you want a successful site, therefore you need experts and attention from inside the SE community. Non-Programmers hobbies like philosphy, history, economics don't work here pretty well, while broad topics with many questions, they gain few attention.

You should rather watch those numbers as minimum requirement to establish and obtain a necessary community dynamics during the first beta months. Otherwise these sites will often fail and you cannot rescue them, just waiting that the site will get more attention by experts doesn't work if you look up sites mentioned above. You need a minimum of competition of a minimum amount of experts or this will simply be a average internet board on topic X like there are many dead ones in the WWW. Thats not the aim of the SE owners, but competing and exceeding Quora, Reddit. And for Physics, Programming, Language... STRONG Programmer's hobbies this works/ed

With current AREA51 system and commitment requirements and average type of SE user I see no chance that you can establish here for example similar quality of Q&A concerning cultural and humane sciences in near future. They noticed this and started merging e.g. Neuroscience and Psychology, but therefore lowering the quality of this merged site. That's the downside, but in the end SE maybe now popular enough to attract anyway enough SE newbies but experts in their field from outside to keep such sites healthy. I'm not sure this works, probably you earn a higher quality than on reddit, but lower than on Quora for this non-tech topics where SE is lacking experts. That's my guess. But it is obviously a downside to have a very broad site with less experts and contradicts the former AREA51 system of gathering experts for a well-defined clear topic. I made a call to unite user from different proposals to join biology.SE, as it looks better to me to not broaden the scope of a proposal but attract commiters from overlapping similar proposals to this one to start it. I think this is the better way, as you get more experts from inside of SE, while from outside, when the beta is launched, there will come much more non-experts to the otherwise broadened and merged proposals.

If a site gets closed, doesn't seem to be determined by these numbers, but a minimum of experts and good Q&A on wikipedia level on those sites, which will grow over the years or sites like atheism, which had such a poor quality and low dynamics that time wouldn't change anything, as they never reached the quality & quantitiy treshold to realize some more good content. That seems to me the line of reasoning when closing betas.

A site that yields only all 2-3 days a superb question (your quality), is like a rss-feed you subscribed to but ends in the recycle bin of your attention. That may work with a blog, but SE is thinking in bigger numbers. And experience has shown, that such a low frequented sites will not be joined by experts who want to be triggered or contribute their views and answers when they look up a site of the few they surf freqently as having less time. A single blog has to offer daily articles to attract commentators, in a blog-network like scienceblogs authors can rely on other bloggers to keep commentators, everything else ends in the nirvana of WWW ;) To me these 2 measures are more important for final quality after some beta-months than the other 3 all together because of these nonlinear group dynamics. No quality without minimum quantitiy and more the development of growth and community size and dynamics measured by these numbers is making SE guys assuming a long term future than current status quo of absolute red/green numbers. The quality and attention on philosophy.SE is currently not really high enough to attract experts, but in the end, closing this site and relaunching it a second time wouldn't significantly raise the quality as less students of humanities on SE and for hobby-enthusiasts the level is probably ok. It will always be the ratio of non-experts to expters on a SE site that defines the quality, but there will also be always much more interested enthusiastic non-experts vs. true experts commiting for topics around cultural sciences. You would have to start a complete new SE network with high-quality sites about culture or philosophy which would only attract people interested in these topics but of course very seldom high-end programming. There are a lot german bulletin philosophy boards with a quality SE will NEVER achieve because of this ratio of user types and background. IMHO this really becomes clear if you compare systems and user types on current boards, same as Quora or Reddit will never reach the quality on programming answers.

Area51 guarantees no high quality, only sufficient and lasting community dynamics to generate further growth and questions. Thats the main point of those measures. The end-quality of your proposal will be mainly determined by experts already hanging around on SE before you even proposed it. The amount of professional theoretical physicists or CS theorist vs. mainly interested students on biology.SE is overwhelming and without top-notch questions at the very beginning it becomes harder and harder to attract experts and professionals from outside with good content above a popular science level. You cannot really invert this starting quality by any type of democratic community managment.

Did become rather long, but that is my final view of limits and possibilities of the SE system, after observing it for a while. Like everywhere, there are pros and cons

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .