David Gibson

Hello, my name is David Gibson and I'm a senior data scientist at Stack Overflow and I'm super excited to be talking to you all today about measuring efficiency and collaboration with knowledge reuse.

If there's any questions at all, feel free to ask them in the Stack Overflow for Teams Slack channel.

So Stack Overflow.

I'm sure many of you are familiar with our popular public website, but we might be less familiar with how big we actually are.

So every month we get a hundred million visitors to our Stack Overflow and our tech stack exchange sites.

This puts us in about the top 200 most visited websites in the world.

And as far as how many questions we have we get a new question every 14 seconds on Stack Overflow.

And today we have about 52 million questions and answers all over the site.

As far as how frequently developers are visiting our site we see that 81% visit us Weekly and 53% visit us daily.

So it's definitely safe to say that.

At least a good chunk of your developer population is just using Stack Overflow on a regular basis.

We also have Stack Overflow for Teams, which you can think of it as a private version of Stack Overflow and through talking to customers.

We've identified Three core challenges that many tech teams face first is onboarding and learning new skills.

How can you effectively onboard new hires and provide learning opportunities for existing ones? The second one is solving big tech challenges as companies go through big Transformations.

They need to work together and to solve problems and document their Journey along the way lastly is working in a distributed team environment not a surprise to anyone but many challenges come up with working in a hybrid remote or mix of the two environments.

And as if you talk to developers not many of them are looking forward to going back into the office full time.

So it's definitely here to stay in distributed doesn't just necessarily mean remote.

It can just be in different offices different time zones different schedules.

So how do you collect all that knowledge and not make it synchronous? so every organization is driving towards scale with an eye on operational efficiency.

You need scale you need an asynchronous collaboration to enable efficiency throughout the organization and no matter what your budget or your scale whether you're a Microsoft or a new startup you always thinking about how you can do more with less.

So At Stack Overflow and especially myself.

We love surveys every year.

We've run one of the largest developer surveys out there and we run a series of other more topic specific surveys and one of the things we see is that we see that 68% of developers encountering knowledge silo at least once a week.

So a knowledge silo is when someone in the organization they have information and it's not shared across the org.

And then another crazy number is we see that about three-fourths of developers need help from someone outside your immediate tech team.

So not only are developers getting stuck and hitting these silos often, but they're also are working collaboratively across the organization.

So one of my favorite jokes about being a programmer or developer is that to be a good programmer? You have to be good at Googling things.

And we actually see this is there's some Merit to this joke with 39% of respondents saying they often are always have to try multiple searches when solving problems.

So this is changing a key.

We are keyword adding to to the error message that you have for moving from there and message you have or combinations of a few so luckily.

This is a skill that's actually developed over time because we see that it can take up to five years five or more years to figure out how to ask the right question only half of developers with zero to five years of experience say it's easy for them to identify helpful resources.

But then if you're looking at the more senior developers that are six to 14 years of experience this jumps all the way up to 67% So put this differently about an entire 1/3 of your skilled Workforce struggling to find the answers to their problems.

And another half of your Junior Workforce is also struggling.

But this does open up a lot of opportunities for us.

So even if it if some say it's easy to find information 62% of developers spend more than 30 minutes each day searching for answers and solutions and 25% spend over an hour.

So these numbers are huge if you add this up by week that's 30 minutes a day for five days that adds up quick.

So not only is there a time aspect but there's definitely some monetary aspect this as well.

So it costs money and time.

With all that time searching developers are often using many resources.

To glue this this the answer together.

So 77% of developers say they often are always used many resources when solving problems not a few not a couple but many and this makes sense when you run into a problem you might start with a Google's if it's a problem of open technology.

You might start with the Google search.

Look at some API documentation definitely kind of Stack Overflow, of course, but if it's an Institutional knowledge, then you might search in Slack you might search in your Gmail your your email you might have another Wiki or knowledge sharing base.

There's not really a unified search interface here and you don't know where the information really should be.

And what this shows us is that it is often a challenge for technologists to get the knowledge that they want even if they do know where it is.

So this chart is or this not the stat is so important that I get it doesn't need a chart at all.

So over half a developers report waiting on answers to questions often causes interruptions and disrupts their workflow.

And this makes sense, you're working on Project a you get stuck.

You have to reach out to a team.

So then you you content switch to say another project and while you're deep into that in other project you finally get your answer and then you have to context switch back to that first project.

So there's definitely a cost to that not only for waiting but also it takes time to refocus and then switch gears essentially.

So speaking on of waiting on answers for every question asked someone actually needs to answer that question and the people answering these questions are equally impacted we see that 46% of developers spend more than 30 minutes a day answering questions.

Add this up, this is about two and a half hours a week of answering questions.

Now personally, I love to see people at answering questions because that's your your collaborating your mentoring maybe more Junior people or not technical people, but two and a half hours a week is definitely a lot.

So maybe not removing this this barrier, but definitely reducing in some cases.

And there's definitely opportunity for that because over about half a developers say that they find themself answering the same question over and over again.

And this is where I think we should definitely focus on removing you get asked the question ones.

Then you get asked questions again.

And then the third time you find it write it down.

It's kind of like automating things.

It's like how many times do you need to do a task to automate it and then by the time you have it automated you, you know, I have to do it.

That's usually you'll have to be at least.

So with all this context switching waiting for answers all the unnecessary frictions.

There's definitely some pain points that can be reduced and more specifically there's time to be gained.

So this is a problem that we want to solve both with Stack Overflow but more specifically with Stack Overflow for Teams. We want to be able to measure the efficiency of the knowledge life cycle.

So how can you measure the efficiency of the knowledge in your organization throughout the lifecycle of both learning and product development? Well, that's where knowledge reuse comes in knowledge reuse is the recycling process.

It's where an individual or group reuses gained knowledge from someone else examples of knowledge reuse include finding the answer to a question where someone already asked it looking up something in a document that already exists or asking a colleague or a colleague showing you something that they've worked on at work.

All these are great examples of knowledge reuse.

So the power of knowledge is really measured in how much times is useful and with Stack Overflow.

We're able to find where these really powerful pieces of knowledge are pieces of knowledge are So one of my favorite questions on Stack Overflow is how do I undo the most recent local commits and get so this question is you was asked 13 years ago.

So one of the more the older questions on the site, but it's been edited 90 times over the years.

So not only has it stayed relevant.

But also it's state up to date because people are editing it.

It's also been upvote in nearly 24,000 times and it's been viewed 10 million times since it was asked.

So even if at this question was asked 13 years ago, it is still relevant and people want to keep this information up to date and this just proves if you have a question, you're likely not the only one but I would say this is even more so the case in in a private setting or inside of organization and likely there's someone in your organization who has the answer on Stack Overflow.

It could be someone across the world in your organization.

It could be someone in the same team of you, but you just haven't asked that question.

So the challenge is in making this scalable.

So I bet if you viewed this question or if you a few questions, that's that however, you'll probably view it again.

And that's exactly what we see.

So 62% of Stack Overflow visitors view the same question in a three-month period so this is proves that knowledge or use isn't a one and done thing your reference the same knowledge multiple times.

Dozens of times until you know or sometimes you might just not need to to Really retain that information with undoing the git commit.

Yeah, it's it's helpful.

But if you are on doing a lot of commits, you might know it if you do it like once every few months, you might not read it remember but remember that information.

So knowledge reuse is reusing what others have already learned created and proved now introduce is important for leaders but also for contributors because it encourages everyone to save time but also money so it does this by a few different ways one is it reduces Risk by having the right answer or like the source of Truth documented for everyone.

You don't have the risk of if someone leaves your organization, they take that knowledge with them or you don't understand how a source works.

So you have to have that document and you have to have the right answer.

It also increases Effectiveness.

So less time searching less time waiting for answers less time answering people give people gives everyone more time to focus on building things and collaborating and going through those big big tech Transformations that I mentioned in the beginning of the presentation.

It also scales the more information available the more it is used and the more it is shared.

But also the more people want to contribute if you're the only one asking questions you it might be a little daunting.

But if everyone in your organization has a culture of asking and answering questions, then it would definitely be fun to participate in that.

So with all that background, let's talk about how we actually measure knowledge reuse on Stack Overflow for Teams first.

We look at knowledge View events without a question of DNS.

That means if you're looking at one piece of content and you don't ask a question, that means you've successfully reused one piece of knowledge.

Say you view five pieces of the content and you don't ask a question.

That means your knowledge reuse value is 5.

But if you view three pieces of content, and then you end up asking a question, then your knowledge reuse would be zero because you had to ask a question but this isn't a necessarily a bad thing and I definitely don't see this is a bad thing because you're building knowledge because now you started the knowledge creation process by asking that question and then someone down the line can use that question in the future.

Another great thing we do is we actually normalize it by the number of users in the instance.

This allows us to get an average user level for knowledge reuse, but more importantly what we found when customers they really want to know benchmarks at how they're doing against other cohorts of similar companies.

So we have these gross numbers of how many pieces of content were actually used and we also have it normalized by users so you can see how you're on average how your community is tracking For more real-world example here is a major software company and we can see that they view 185,000 pieces of content during the month of August.

So this is huge imagine.

185,000 questions not being asked because the content already exists.

So this is definitely at a massive scale, but you can see how it's actually scaling over time as well.

So as more and more people are using the system and is more and more people are learning how to look up questions.

They're reusing more and more knowledge.

So it's definitely a scalable approach.

And to make this personally to make this personal how many questions do you think other teams? Ask each other for this for this example, these were all the questions that weren't asked but how many questions were asked is something to ask yourself.

Here's the same graph but actually normalized at the per uses per user basis.

So we see on average.

There's a score of about 1.5 which means that for everyone one in person in the organizations.

They didn't ask one and a half questions this month.

So another way is they didn't have to ask their teammates.

How do I do X they don't have to wait around for answers and no one had to take time out of their day to answer the questions that they had because it already existed.

It's equally more time learning.

It's more time learning and more time building things.

So time not we wasted on asking questions or waiting for answers is always time that could be spent working our projects or collaborating.

So I bet knowledge reuse has already happening in your organization.

But with Stack Overflow for Teams, we actually encourage it and let it scale exponentially because of this network effect.

I'll be walking through a more detail each of these steps, but at a high level the first one is uncovering and knowing your subject matter experts you need to know who they are identify them and let them be able to do what they do best.

The next is make knowledge easy to capture and easy to reuse.

So not only should it be a low barrier to entry to create but it should be a low barrier to entry to actually access this information.

Speaking of that you need to make sure that knowledge is discoverable and is the implemented with it existing workflows.

If a lot of your teams use ChatOps tools, we have Integrations with ChatOps tools, but always make sure that it is accessible and it should be like the first place that people look and lastly is ensures its resilient.

We all know about tech debt and even knowledge needs to be maintained over time.

So the first one is empowering your SMEs or your subject matter experts.

So you need to find and Empower them.

They're the ones who hold the answers when you're when your teams have questions and they're the experts across your organization and I guarantee you that you have many subject matter experts.

And oftentimes other teams might not even know they exist or know who to go to.

They might know someone is a developer on a certain system, but maybe there's a different part of it, but they don't know who to go to so they have to go to the only person they know and that can cause a chain effect.

But if they know you're some subject matter experts that reduce the barrier.

Initially to create asynchronous paths for your teams to find answers without impacting them if everybody's messaging them in Slack or if everybody's emailing them that's taking time away from them to be building and using their knowledge to build the next generation of the technology.

And two is really important not to conflate documentation with knowledge.

We all know that long-form documentation can feel like a burden for writers.

But also maintainers we've all pushed it off from writing the technical documentation for a project.

And then also when we're writing we're always looking at it from a lens of you already know the whole system you can cut a few details out and it still makes sense to you but for a new user or a junior developer, this isn't isn't the case at all.

Only use long-form documentation when it's absolutely necessary like customer-facing products or platform level Technologies or API documentation.

On the other hand questions and answers are much more scalable with smaller Time Investments both for the writers and also the readers it's very targeted.

There's it's one question one answer but there might be a few answers, but ultimately there's one right way to do it and it's it's a lot easier to digest that one question than it is to find that question in some long form documentation and extract that extract that out.

Three is you don't have to know everything.

You just have to know where to find it.

We've all heard the phrase knowledge is power.

Well, honestly finding knowledge can be even more powerful.

So you have to centralize this knowledge make it easy to find but also make it the first place people look developers are already very familiar with Stack Overflow our public platform.

So with that we're for teams is a very comfortable environment for them to be in they know the question answering dynamic.

They know how it works.

So adoption is actually we found the adoption is much greater with developers because it's something they're already very familiar with and as mentioned you need to integrate it with existing workflows our chatops Tools in our ChatOps integrations have shown to be extremely powerful and effective because of the because they meet developers where they are.

The last one is automated resilience.

We all know that information can get outdated over time very quickly and with Stack Overflow for Teams. We have some pretty cool things in place to to make sure that it's really resilient and up to date.

And what this essentially means is making sure that the content that you create today stays up to date one two three years from now.

Just think about how many outdated wikis are in your organization or updated documents or or dashboards anything any any content that gets created has to be maintained and kept up to date unless there's an explicit timeframe on it.

So the great thing that about Stack Overflow for Teams is we address this exact problem with our Content Health feature content health is based on the frequency of views and the last edit date to flag at risk content to be reviewed.

If something's flagged it goes through subject matter experts and it only takes a few seconds for them to look at the content and they can flag it as looks good needs the edit or mark it as obsolete.

So this is extremely quick to go through dozens of pieces of content if they are all up to date it just check it and it goes back into the queue for for later date.

But if it's obsolete, it's super easy to just deprecate this information.

It's still available, but it's just marks this Obsolete and then editing if it's a quick edit the time commitment is very low.

So obviously this sounds super efficient.

So post-launch.

We actually saw that editing activity increased by 600% for three months after launching this feature.

So I joke as a data scientist their sometimes that we don't even need statistics to know that that was a successful product launch.

So definitely people are very excited.

But it proved that people want to keep knowledge up to date and it goes back to the thing of lowering the barrier to entry sometimes you can create knowledge, but also people are excited to keep it up to date if they are the new subject matter expert maybe they don't need to answer the question.

But now they kind of own that piece of content by making sure it's up to date throughout the time that they're at the company.

So we believe that knowledge reuse is the important metric for collaborative and efficient teams.

We believe that all learning starts with a question and that with Stack Overflow and Stack Overflow for Teams. We want to connect to you and your developers with the right answer whether that is how to undo a git commit or a institutional or organizational tool that you have a question about Thank you so much.

And if you have questions, please make sure to ask them in the Stack Overflow for Teams Channel and I will be answering them and I believe a few of my colleagues are going to be available to answer them as well.

Thank you so much.