22

It's been a while since project work has required cutting into my sfse time. Now that I've got some more free time, I was stoked to get in and make some great answers. But it seems things have changed, and it's really hard to find something that's sufficient to provide a decent answer. Instead, I'm often relegated to the close vote review queue in the hopes of pruning out sub par questions.

Anyone else think we're starting to have a question quality problem? And if so, how can we improve things without ostracizing new contributors (or maybe we should be a little tough). Long story short, would love to start focusing on answer and improving questions and answers instead of just filtering through all the chaff of "write my test class" and "it's not working, but don't expect me to provide details" and "my requirement is to do this, please help"

  • 5
    I would love to contribute towards a Community Wiki answer for basic unit test questions if that's something enough people are interested in to make it effective as a generic duplicate. – David Reed Nov 7 '18 at 14:51
  • 1
    @DavidReed, I'd be up for that. – Shane Steinfeld Nov 9 '18 at 22:11
  • @ShaneSteinfeld and anyone else who thinks this might be an interesting project (tangentially connected to Ralph's question!) please drop me a line (contact in my profile). I'm starting a first draft. – David Reed Nov 10 '18 at 22:50
  • 4
    @DavidReed There is a canonical-qa tag you can use for such questions (currently only one q I created, would be good to see more). – sfdcfox Nov 12 '18 at 19:03
  • I am fairly new contributor on this platform but I would echo this. I have felt that sometimes people don't even attempt to make a search on the web. I have seen quite a few questions which with a simple search can be found in the documentation and answers the question directly. – Jayant Das Nov 14 '18 at 16:42
  • @JayantDas - agree with you that many questions often show no research effort, but that has been the case all the time I have been using this site and I remember the same problem back when I was an Oracle developer on their forums back in the early 2000s. – Dave Humm Nov 16 '18 at 11:28
28

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Q & A with variation (lower is better)

Questions and answers by month


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)


Q&A Ratio

Ratio of Questions and Scores to Answers


Unanswered Questions by Month (%)

Unanswered questions by month as %

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month.


More information (and substantial edits).

We're still closing at the same rate we have for a few years now, but our unanswered ratio has gone up considerably in the past year, and voting is down. Answers still get upvoted pretty consistently, but the questions do not (we're currently at about 1 question upvote per 2 answer upvotes, shown in the Q:A ratio graphic).

Questions are being abandoned more frequently, with up to 30% of questions from any given month going unanswered; this trend started almost exactly a year ago, while almost all historical months are under 20%. As cropredy suggests, this might be related to new subject matter for which we have few experts. It may be useful for us to learn or find people who know about these subjects and help us out.

Also, we need to find a way to deal with the more graphical questions. For example, people want help with a Flow or Process Builder, but we have no way to help them without a long dialogue just to figure out what they're even trying to do, specific settings, etc. We might need some Q&A-style canonical questions that we can use to at least link to new contributors so we can skip the dialogue and get meaningful results.

In addition, I suggest that we start more aggressively cleaning up tags on questions. Let's make sure that new contributors are tagging their questions correctly so we can get experts looking at those questions. Incorrectly tagged questions might not get seen by experts with the new watch/ignore tag feature.

And, when applicable, we should remind people, especially new contributors, that it's okay to accept a best answer, it's okay to upvote, etc. And I think more people need to be encouraged to vote overall, or, if they choose not to vote, we should encourage them to suggest edits, add comments, and otherwise generally help improve question quality.

We need at least a few people to look at the questions at a deeper level, start cataloging commonly unanswered questions, and break it down into a few different types: duplicates we can't close, difficult subject matters, commonly mistagged questions, etc. The more of an effort we put in to this, the better we can make our community.

I think at this point we're going to have to go through a bit of a rough patch to clean things up, but I also feel it's worth it. With some better resources, maybe some "quick training" videos, like has been previously suggested, a library of great canonical questions, perhaps a set of common comment templates we can all share (I know Adrian's got a bunch of 'em), we can improve our network.

At this point, I think we have enough evidence that the network needs help, and we need to find specific ways we can help the network and implement changes to provide higher quality questions and more consistent solutions for people that need them. We can organize details in further meta questions, so please don't hesitate to post if you have any ideas.

  • 7
    is it possible that people are just not voting as much as they use to? seems like both question and answer scores have fallen off lately. There are a lot of new contributors that are asking questions but possibly not voting, as they are new. – Travis Naughton Nov 12 '18 at 4:56
  • 1
    @TravisNaughton We've "always known" that new contributors are less likely to vote and/or accept answers, and that those Q&A are the long tail of our network. Our close numbers have increased since beta, but are still only around say 8-10% of questions, and is down from a few years ago. The higher variation from the first chart show that lots of people are still voting on answers, but not so much questions. This suggests a question quality problem; good enough to avoid closure, but bad enough not to get votes. – sfdcfox Nov 12 '18 at 5:16
  • @TravisNaughton Mind you, I'm not a data expert, or a statistics expert. All I know is that I see a lot more answer votes than question votes, and those numbers seem to be down from 2016 across the board anyways, even though question traffic is about the same over the course of a year. If there's any other charts you think I could run, please feel free to let me know. I feel like I'm still missing part of the picture. – sfdcfox Nov 12 '18 at 5:20
  • 1
    I guess my hypothesis is that people are voting on questions and answers less than they use to, or that there are the same number of voters but more questions. The first graph would indicate that questions have had a bigger drop off than answers. I think if we could see "average score per answer as a percentage" we could see if the decreased trend is the same as questions. The second chart shows a steady increase in questions opened. Feels like questions opened and average score is correlated. An increase in questions opened is spread thinner over the same amount of users that are voting. – Travis Naughton Nov 12 '18 at 23:07
  • 4
    Two meta hypotheses: (1) More developers have switched to Lightning Flow + PB and hence the number of questions that folks on SFSE tend to answer has gone down. (2) SFDC as a platform has become more feature rich and SFSE answerers just aren't as familiar with the new feats as in the the 'classic' days of just apex+VF – cropredy Nov 15 '18 at 0:59
  • 2
    @cropredy (1) seems to be part of the problem. (2) also seems to be a part of the problem (e.g. DX, MC, Heroku, etc). We also do have a quality problem; our downvotes are about the same as they were 3 years ago, but our upvotes are down significantly. I added an edit with some new charts. I now know that we do have a problem by the numbers, but we need to spend time researching the cause and proposing solutions. – sfdcfox Nov 15 '18 at 5:34
  • @cropredy sfdxfox everyone ... OK then chaps... What is the plan to get this place back on track? – Robs Nov 20 '18 at 21:58
  • 1
    @Robs We need to have a meeting of sorts, a chat about our problem, suggest ideas, organize them into a suitable Q&A for the site, then start to implement our plans. I've heard a lot of great ideas so far in the comments, but we need to formalize the process. The question is how to get people involved. Do we invite our top 20 or so? Post to Twitter? What's the best way to bring people's attention to this? – sfdcfox Nov 20 '18 at 22:19
  • Agreed. What about Salesforce? Surely they realize this is a higher quality place to find friends and support than their home grown forums? – Robs Nov 20 '18 at 22:27
  • @Robs I don't know of a Salesforce Liaison, but you do, they're welcome to join. The main people we need include our admins, our most active users, and of course, anyone else that's interested in Making SFSE Great Again. – sfdcfox Nov 20 '18 at 22:38
  • What about Wade? – Robs Nov 20 '18 at 22:41
  • 2
    @Robs The community has been grown organically from people passionate about Salesforce, but not necessarily from Salesforce itself (unlike the Developer Forums, etc). Wade might be interested, or might not; it doesn't hurt to see if he'd be interested. However, he doesn't have a lot of interaction with SFSE ( <700 points, <10 answers), so he might not know enough to be helpful. – sfdcfox Nov 20 '18 at 22:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .