Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now
4 added 1629 characters in body
source | link

Q&A Ratio

Ratio of Questions and Scores to Answers


Unanswered Questions by Month (%)

Unanswered questions by month as %

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow


More information (and substantial edits).

We're still closing at the same rate we have for a few years now, I thinkbut our unanswered ratio has gone up considerably in the past year, because I haven't distinguished between duplicates and other reasonsvoting is down. Answers still get upvoted pretty consistently, but we are closingthe questions do not (we're currently at about 5-10%1 question upvote per 2 answer upvotes, shown in the Q:A ratio graphic).

Questions are being abandoned more frequently, with up to 30% of our questions monthlyfrom any given month going unanswered; this trend started almost exactly a year ago, while almost all historical months are under 20%. As cropredy suggests, this might be related to new subject matter for which is either far too high ifwe have few experts. It may be useful for us to learn or find people who know about these subjects and help us out.

Also, we need to find a way to deal with the more graphical questions. For example, people want help with a Flow or Process Builder, but we have no way to help them without a long dialogue just to figure out what they're even trying to do, specific settings, etc. We might need some Q&A-style canonical questions that we can use to at least link to new contributors so we can skip the dialogue and get meaningful results.

In addition, I suggest that we start more aggressively cleaning up tags on questions. Let's make sure that new contributors are tagging their questions correctly so we can get experts looking at those questions. Incorrectly tagged questions might not duplicatesget seen by experts with the new watch/ignore tag feature.

And, when applicable, we should remind people, especially new contributors, that it's okay to accept a best answer, it's okay to upvote, etc. And I think more people need to be encouraged to vote overall, or far too low, if they arechoose not to vote, we should encourage them to suggest edits, add comments, and otherwise generally help improve question quality.

We need at least a few people to look at the questions at a deeper level, start cataloging commonly unanswered questions, and break it down into a few different types: duplicates we can't close, difficult subject matters, commonly mistagged questions, etc. The more of an effort we put in to this, the better we can make our community.

I think at this point we're going to have to go through a bit of a rough patch to clean things up, but I also feel it's worth it. With some better resources, maybe some "quick training" videos, like has been previously suggested, a library of great canonical questions, perhaps a set of common comment templates we can all share (I know Adrian's got a bunch of 'em), we can improve our network.

At this point, I think we have enough evidence that the network needs help, and we need to find specific ways we can help the network and implement changes to provide higher quality questions and more consistent solutions for people that need them. We can organize details in further meta questions, so please don't hesitate to post if you have any ideas.

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.


Q&A Ratio

Ratio of Questions and Scores to Answers


Unanswered Questions by Month (%)

Unanswered questions by month as %

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month.


More information (and substantial edits).

We're still closing at the same rate we have for a few years now, but our unanswered ratio has gone up considerably in the past year, and voting is down. Answers still get upvoted pretty consistently, but the questions do not (we're currently at about 1 question upvote per 2 answer upvotes, shown in the Q:A ratio graphic).

Questions are being abandoned more frequently, with up to 30% of questions from any given month going unanswered; this trend started almost exactly a year ago, while almost all historical months are under 20%. As cropredy suggests, this might be related to new subject matter for which we have few experts. It may be useful for us to learn or find people who know about these subjects and help us out.

Also, we need to find a way to deal with the more graphical questions. For example, people want help with a Flow or Process Builder, but we have no way to help them without a long dialogue just to figure out what they're even trying to do, specific settings, etc. We might need some Q&A-style canonical questions that we can use to at least link to new contributors so we can skip the dialogue and get meaningful results.

In addition, I suggest that we start more aggressively cleaning up tags on questions. Let's make sure that new contributors are tagging their questions correctly so we can get experts looking at those questions. Incorrectly tagged questions might not get seen by experts with the new watch/ignore tag feature.

And, when applicable, we should remind people, especially new contributors, that it's okay to accept a best answer, it's okay to upvote, etc. And I think more people need to be encouraged to vote overall, or, if they choose not to vote, we should encourage them to suggest edits, add comments, and otherwise generally help improve question quality.

We need at least a few people to look at the questions at a deeper level, start cataloging commonly unanswered questions, and break it down into a few different types: duplicates we can't close, difficult subject matters, commonly mistagged questions, etc. The more of an effort we put in to this, the better we can make our community.

I think at this point we're going to have to go through a bit of a rough patch to clean things up, but I also feel it's worth it. With some better resources, maybe some "quick training" videos, like has been previously suggested, a library of great canonical questions, perhaps a set of common comment templates we can all share (I know Adrian's got a bunch of 'em), we can improve our network.

At this point, I think we have enough evidence that the network needs help, and we need to find specific ways we can help the network and implement changes to provide higher quality questions and more consistent solutions for people that need them. We can organize details in further meta questions, so please don't hesitate to post if you have any ideas.

3 deleted 105 characters in body
source | link

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Sum of question scores

Question scores since inception


Sum of answer scoresQ & A with variation (lower is better)

Answer scores since inceptionQuestions and answers by month


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Sum of question scores

Question scores since inception


Sum of answer scores

Answer scores since inception


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Q & A with variation (lower is better)

Questions and answers by month


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.

2 added 1791 characters in body
source | link

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Sum of question scores

Question scores since inception


Sum of answer scores

Answer scores since inception


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Question scores since inception


Answer scores since inception

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

I've felt a same general idea that our network was having issues, but I wanted more concrete data, so I ran some numbers through the data explorer, and created some pretty charts.

Sum of question scores

Question scores since inception


Sum of answer scores

Answer scores since inception


Questions opened and closed per month

All questions vs closed per creation month


Average score per question as a percentage

Ratio of questions to total score (higher is better)

Please feel free to zoom in to get a better idea of what's going on. Basically, each data point shows the number of net points that were accrued for questions and answers since 2012.

As you can see, we had our last strong peak was January 2017, followed by decreasing bounces back. It seems at this point we've flatlined around 1,000 points a month this year, down from closer to 2,000 for the past four or five years.

I don't have an immediate answer to the problem, but it's clear that we need to investigate further. There is something wrong with the network, a systemic problem that needs to be looked in to. It may be in part, because we're closing rather than encouraging edits, or maybe closing too many duplicates already.

I know we recently got the "new contributor" marker to help us identify new users, but we need to do more. We need to encourage everyone to do better questions. The answer quality hasn't fallen from what I can tell, it's just that there's less questions to go around, so fewer answer scores to go around.

I will do some more research this weekend, so expect an update to this answer, but for now, just know that it's not your imagination, there's definitely a problem we need to look in to and address.

If we consider the number of questions asked versus their average score, which I called % Score in the last chart, we'll see that the overall quality of questions appears to be suffering as this number goes down. This number represents a percentage, so a month with 100% means that, on average, every question got one vote. Our current trend is less than 50%, meaning that if we took all the points for the month, we couldn't even up vote half of the questions asked that month.

I understand the ratio was crazy back in pre-beta because it was a core group of users, but even as late as January 2017, we had over 150%, meaning the average score for questions was 1.5; we're now below 0.5. Note that this value lumps all votes for a question into it's asking month, so naturally older questions will typically have higher scores.

However, the overall trend still reflects that older questions are apparently higher quality than newer ones. We should try to find a way to get better numbers. We're still answering a decent number of questions (the question to answer ratio is good), but we need to focus on helping better questions.

The open-to-close question ratio is interesting. I thought that might show a lot more closures, but really, we've been holding steady at about 120 closes a month. I'll add some more numbers tomorrow, I think, because I haven't distinguished between duplicates and other reasons, but we are closing about 5-10% of our questions monthly, which is either far too high if they're not duplicates, or far too low if they are.

1
source | link