56

Proposal: Software Recommendations

We're going to let this site go ahead, and see what you all can do with it. While we discussed the topic, we had to take some time to identify our chief concerns with recommendation & shopping type questions. The number one problem that we agreed upon was pretty simple:

They tend to overrun a site.

This entire site is dedicated to this type of question, so this is obviously a moot concern. With that said, the second chief concern was quality - so we do want to lay down some ground rules when it comes to asking and answering this type of question - and then close with something everyone might want to think about. We're honestly excited to see where this goes, and will support you to the best of our ability.

Asking recommendation questions

It's important to keep your questions as narrowly-scoped and specific as possible, because this is what's going to steer the types of answers that you receive. This is best illustrated by example, starting with titles:

What's the best IDE for Python?

This will be closed immediately, as it does not:

  • Explain in detail what you hope to get out of an IDE in terms of features, license or cost
  • List any applicable frameworks that you might be using, or other possible compatibility quirks with your workflow
  • Indicate any that you had previously tried, and why you didn't like them
  • Do anything else to encourage answers to be as specific as possible
  • Obviously, 'best' without context is never going to work - in fact please try to just avoid the word in titles. If you ask about an IDE for Python on Linux that has (features), we're pretty certain people aren't going to recommend the worst.

What Python IDEs support Django?

This is a much better question, as you've really narrowed the possible answers with some additional criteria. If you can add more criteria in the question body, then you've got the makings of something that is useful, and very unlikely to receive a bunch of mostly link answers.

What Windows Python IDE supports Django and ships with a pink color theme?

... I think you get the point, more specifics means a better question. Ideally, questions will on average have only 5 or so answers that actually address the question constraints to the best of their ability.

You must also be prepared for answers that simply inform you that no single thing meets your needs, but goes on to recommend things that cover what you've identified to be the most important. Hence, make sure you're clear about what you must have out of something, and what would simply be nice to have but not absolutely essential.

Good questions describe specific needs, contain bullet lists of constraints and inform anyone that cares to answer as to which constraints are the most important.

When it comes to libraries, show your ideal use of a prospective library. What features should it provide? What kind of interface would you like in ideal circumstances? Does it need to be able to compile even under certain compiler settings, or platforms? Do you have a size limit in lines of code, allocated memory, or both? Just like in the IDE example above, it's very helpful when you indicate how important these constraints are.

Answering recommendation questions

We need to put just as much work into answering these questions as folks put into asking them. Answers that contain little more than a link will be deleted without question, comment or other ceremony. Additionally, our policy on excessive self-promotion will be even more heavily enforced on the site. If you're asked to stop promoting a product that you're affiliated with by a moderator or community manager - you need to stop, or you'll likely be asked to leave the site. Spam filters will be adjusted accordingly - so be careful.

If you work on a product, be it free, gratis or proprietary - it's fine to recommend it to people opportunistically provided that you include full disclosure of your involvement with the product. If more than a small percentage of your posts mentions your product, we're probably going to need to have a talk.

Good answers on this site will be in the form of honest testimonials that share first hand experience with something and why it meets the needs of the asker.

You should answer questions when you have first hand knowledge of things that satisfies at least most of the constraints given in the question. You should be ready to talk about your experience with it, what you liked about it, what features it does (or perhaps doesn't) have, within the context of the question. You should also talk about any potential quirks that you can think of, and even things that you found to be a down side of the product. In short, share your experience with something, not simply your knowledge that it exists.

Answers that do not even come close to meeting the constraints described in the question should be flagged as not an answer, and will be removed quickly.

Something to think about - what if this works?

We wouldn't be supporting this proposal if we didn't think it had a chance of working. Looking at the users involved so far, and some of the better example questions, we think if any community has a shot at making this work it's this community. However ...

What if it works? What if you finally get these types of questions down to a science? If Stack Overflow decides to relax just a little when it comes to these types of questions, well - you get the drift, and need to keep it in mind. The ultimate form of success for this community might ultimately be rejoining larger, established sites.

It's way too early to say, and I've probably portrayed this as much more complicated than it actually needs to be. Just focus on the quality, favor depth over breadth during your private beta and get active asap on your meta site once it becomes available. We will be watching, we will be helping to the best of our ability with advice and guidance, and we're going to make sure you get a fair shot.

I'll now open the floor for anything additional anyone would like to add (just write an answer) - and we'll see you when the beta is ready :)

  • 7
    Pleased to hear that the site will get a shot at survival. A good set of ground rules here. I suspect there will be a lot of questions - at least in the early days - migrated from sites like SuperUser, Android Enthusiasts etc. I think we should impose the same criteria as you have stated upon those migrated posts. There may be some rewording and/or refining of some such questions. I'm thinking of the "Is there an app for..." type. – Chenmunka Jan 21 '14 at 20:39
  • 2
    Great post! Can this be migrated to the meta site as soon as the sites launches? – juergen d Jan 22 '14 at 0:21
  • @juergend Yes-ish, there's no migration paths from Discuss.Area51. A more refined / condensed version of this might just go in some custom help center entries on the new site - then we can open discussions for asking / answering individually as we need to. – Tim Post Jan 22 '14 at 5:12
  • 6
    @Chenmunka I wouldn't expect too many migrations, at least early on. If this site works, it's going to be because this community got it right - we definitely don't want to go 'dumpster diving' on other sites, if the questions are still relevant just ask them again, in the manner that this community came up with. – Tim Post Jan 22 '14 at 7:19
  • @Tim Post, I agree that this community should set its own standards. My thought was that people who currently flag posts as off-topic elsewhere may, when they know we exist, flag for migration instead. I could be wrong. And yes, we wish to refine such posts to fit this community. – Chenmunka Jan 22 '14 at 9:44
  • Regarding "What's the best IDE for Python," are we going to have a custom close reason for these kinds of questions? I think the community could benefit from having specific guidance, rather than relying on the generic advice of "Too Broad." – Robert Harvey Jan 29 '14 at 18:35
  • How are you planning to deal with the landscape changing over time? Eg Someone asks a question in keeping with your guidelines, a good answer is posted and over the next couple of years gets 50 upvotes. Along comes a new IDE which meets the OPs needs more closely. It's going to be years before the newer answer bubbles to the top organically. – Basic Mar 26 '14 at 12:23
  • @Basic Just leave a comment that this answer is now obsolete. Don't forget that just because a better IDE exists, it will be another year or two before people used to coding in one IDE would switch to the other. So if someone is searching for the best IDE, they will read the comments; otherwise, the votes will reflect how often is this IDE used, rather than how good it is -- I think this will be a given. – Alex Nov 8 '18 at 15:30
  • @TimPost I recently posted a thread for an idea similar to this site; would you mind leaving your feedback, having moderated this site for a while, and seen the issues that come up? Thank you very much in advance. area51.meta.stackexchange.com/questions/28325/… – Alex Nov 8 '18 at 15:31
15

This has probably already been stated, but I would like to make it clear that this site should not be questions in the form of:

What do you think about software package XYZ?

Folks that want that can simply go to product review sites.

To me, this site is about people who know the features or concepts they are looking for but don't know the names of existing products or don't have the technology-specific vocabulary to get there. It helps ease someone into a new technology genre, not provide analysis of a software package they have already chosen.

Note that this does not preclude questions in the form of:

I am looking at software package ABC to do items:

  1. some task
  2. some other task
  3. some other task again

Does software package ABC perform these tasks?

Just my thoughts. Comments?

  • 2
    Exactly. Describe precisely what you want out of the software in order of importance, describe how you envision it working or using it, and you basically create a negative impression of something that someone else just needs to name. These should turn out to be rather good questions for the most part, I'm excited to see how it goes. – Tim Post Jan 22 '14 at 5:07
  • @kmort I agree with your first example, but not with the second one. If you are asking whether a specific software package can perform these tasks, then it is not appropriate for this site, in my opinion. The answer to this question would be either "Yes" or "No" which is not a software recommendation. Make it "I am looking for a software package to do items" and "Which software package performs these tasks?". You might get ABC as an answer (or add "(I looked at ABC for example, but I am not sure if it is suitable)") – Bernhard Jan 27 '14 at 11:58
  • @Bernhard Yes, your phrasing is more appropriate. – kmort Jan 27 '14 at 13:13
7

They tend to overrun a site.

I beg to differ.

Until a few months ago, Stack Overflow did not have a specific policy against recommendation questions. (There is now a specific close reason which declares them systematically “off-topic” while pointing to a meta thread that explains how to write a good recommendation question.) Yet was not overwhelmed by recommendation questions.

Super User has long had a policy against “shopping questions”. The policy was in fact applied to most recommendation questions. Super User does get a lot of recommendation questions, but it was never overrun by them.

On Unix & Linux, recommendation questions have never been a big issue. There are a couple of meta threads on the topic, but no specific policy.

Gaming has a blanket ban on recommendations. I didn't follow all the discussions, but as far as I know “they overrun the site” wasn't the primary argument against them — the argument was a lack of quality.

When Science Fiction & Fantasy started, there were 20% of list questions. Some discussions on SF&F conflated list questions and recommendation questions, but they are in fact two very different categories. Recommendation questions were banned because most of them were bad.

The now defunct Literature site tried out a book recommendation policy (live copy here). Having observed it at work, I would say that:

  • Neither the fact that recommendations were allowed, nor the policy in itself, had a major impact on the site's viability. It died because there were too few participants.
  • When the policy worked, it generated good, interesting questions with good answers.
  • What made the good threads work was specific requirements — not “what shall I read”, but “here's a problem I want to solve” — and detailed answers — all the good answers recommended multiple books.
  • A majority of recommendation questions were bad, but closing them was poorly enforced.

There are two keys to making recommendations work.

The first key, which you mention, is to solve a problem. A question that asks “what's the best X” should be closed. A good recommendation question asks “how do I do X” — and if the answers are “use product Y”, that's fine.

The other key has to do with answers. It's important not to let the answers degenerate into a poll. Stack Exchange sucks at polls. The voting on polls shows:

  1. who posted first;
  2. how popular the item is;
  3. how well the item answers the question.

For the answers to be useful, #1 must as much as possible not be a factor. #2 is a side concern, #3 is the really important one. So it is vital that an answer is not “use this product” but “use this method” (which happens to use a product). If the method uses no specific product, or multiple products, or if an answer recommends different products for different scenarios, that's good.

I find it a bad sign that, even before the site starts, a proposal to require answers to contain a single suggestion is rather popular. Such a policy would be highly toxic to the site. On the contrary, for the site to be viable, answers with multiple suggestions should be encouraged. Answers that just say “use this product” should be deleted on sight.


And yes, it's weird to single out recommendation questions and have them on a separate site. Why not have them on the site where they're on-topic?

The only reason I can see is that recommendation questions require moderation, which sites such as SO and SU are poor at enforcing. On a site where an answer like “You should totally use jQuery” scores well instead of being summarily deleted, there is no hope to have good recommendation questions.

  • 1
    I has less to do with being overrun than it does with the community giving disproportionate weight and attention to such questions. That kind of attention distorts the voting system, consumes everyone's time, and distracts members from other, more deserving questions. In no small measure, it turns off the experts, who we are supposed to be attracting, not repelling. – Robert Harvey Feb 5 '14 at 21:25
1

This might be a little bit early for a site that has not even started yet.

We have to take care of old questions. Software changes and so new software might meet the original requirements, a good example for this is syntax highlighting.

Several years ago, syntax highlighting wasn't standard. Sure, a few products might have it, but now it is standard. Almost everything except windows notepad supports that feature now.

So I expect that a growing number of software meets the requirements of old questions. How should we deal with that? Close them as obsolete?

  • We're pretty much anticipating that supporting these types of questions might require some additions, or possibly changes (albeit minor) to the system. This could be in the form of additional post notices, or a new close reason like 'duplicate' but called 'obsolete' that lets you point people in a better direction (hey, some people do live in caves and didn't hear about syntax highlighting). This is also going to be a bit of learning on the curating and moderation front, what about top scored answers that now point nowhere? This is what we'll explore in the beta. – Tim Post Jan 21 '14 at 16:51
  • Also (darn comment lengths) Just to be clear, all of us (including those of us on the community team) plan to learn quite a bit from doing this - I just wanted to lay down some initial ground rules just so folks know what to expect going into the private beta. Normally, we don't try to address problems that we don't yet have, but this is a bit of a special case. – Tim Post Jan 21 '14 at 16:53
  • Better prevent broken windows. – Johannes Kuhn Jan 21 '14 at 17:10
  • 10
    There are embedded systems that are running operating systems that are much older than modern desktop systems would even consider. Let's not categorically dismiss them as obsolete simply because desktops have moved on. Yes, for most people, they are obsolete, but there are real uses for being able to ask about old software. There should be a way to clearly indicate any known endpoints of the lifetime of the software being discussed however. – kmort Jan 21 '14 at 17:15
  • That is an other solution. Change questions that were about desktop software to questions about old hardware/software (lock the OS version or similar). – Johannes Kuhn Jan 21 '14 at 17:18
  • 2
    There might be the need of an "old answer" review queue... – juergen d Jan 22 '14 at 0:26
  • 2
    @juergend Might be - and this is definitely the motivation we'd need to finally get the broken link queue production ready if it looks like this is going to last in public beta. – Tim Post Jan 22 '14 at 7:21
  • Sounds good. But let's imagine that for SO. You can't really check all posts with a link. How would you do that? Let users flag the answer? What about anonymous response? – Johannes Kuhn Jan 22 '14 at 12:47
  • @JohannesKuhn There's an automated process that checks all the links on SO and notes which posts have links that have remained broken for a while. It then puts those posts in a queue where users can see if they can either find a better link, or figure out what was linked and move it into the post itself. As wayback now has an API, automatically fixing some links could be possible (we're looking at it). Anyway, broken links on accepted / highly up-voted posts is a good criteria for reviewing some potentially stale answers, that's for sure. – Tim Post Jan 22 '14 at 14:29
  • 1
    See this discussion: discuss.area51.stackexchange.com/questions/12881/… – Bernhard Jan 27 '14 at 12:00
1

As a moderator on Cooking, I have seen quite a few recommendation questions, and have had to reason about whether they are good or bad questions for a SE-network site. We didn't have a big discussion on "what should I buy" type of questions, but on the similar "where should I buy" type of question, which is generally a recommendation of stores.

The rule I have noticed is:

Askers who don't know if what they need exists are asking good questions (=seeking to expand an empty set of good solutions). Answers who are trying to choose between multiple products they know are asking bad questions (=seeking to make a choice from a set of multiple good solutions).

Technically, these two categories of askers tend to sort themselves into the categories mentioned in the "question" post. The ones who don't know if a product out there satisfies their needs already know their specific needs and list them in a question. The ones who are overwhelmed by a choice are the ones who ask "which is better, A or B" (Insert ATI vs Nvidia, Emacs vs Vi, etc.)

The reason I am posting this here is that it is a good lithmus test which can help a moderator make a decision about which questions to keep and which to close. I find it a more precise criterion than "specific enough", because specificity is a continuum, and it is harder to judge what is "enough". So my suggestion to the mods is to try applying this criterion and see if it fits with their gut feeling of which questions are good and which are bad. If it provides a good fit, you might want to codify it in the site policy.

A corollary is also that there is not much sense in urging askers of bad questions to make their questions more specific. The reason is that we are trying to recommend something which fits a set of needs, and the person who seeks "the best" doesn't know his own needs. You should of course gather data on this one too before basing policies on it, but you may not have noticed it (I am a requirements engineer, so a bit more sensitive to need fit etc.), so I thought that it is worth mentioning.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .