Knowledge management in an age of AI
Summary
AI has forever changed the way we find knowledge and create content. Nagarjun Kandukuru and I believe that this shift in behaviour has deep implications for enterprise knowledge management. To recognise this shift, we’ve written an AI-first KM manifesto, which we’d like to introduce to you.
A few weeks back, I told you about some knowledge management (KM) experiments we’re conducting at Thoughtworks. Glean, an AI-powered search platform, was at the heart of our pilot. My colleague, Nag, who’s also a contributor to this website, is running a few more experiments, with Gemini and custom tools such as RohanRFP.
Nag and I have been collaborators for the longest time, so while our day jobs are quite different, we exchange notes all the time. In recent months, we’ve been developing an opinion about what KM should look like in this age of generative AI.
Effective knowledge management often solves two kinds of problems.
Knowledge retrieval; i.e. find the right information.
Knowledge synthesis; i.e. use that information to create a new artefact.
AI has revolutionised the way we solve each problem.
AI powered search platforms such as Glean help us surface company knowledge not just as a list of search results, but in a variety of other formats depending on the user’s preferences and the use case.
Retrieval augmented generation or (RAG) can then either synthesise these results to a tailored insight, or even a new artefact.
Nag and I see the search-verse and the RAG-verse as a perfect couple for knowledge management in an age of AI.
Making sense of decentralised knowledge through AI-search and RAG
To illustrate this point, let me quote a colleague from sales.
“Glean as a productivity tool for RFPs, has reduced the time needed to find case studies in half. It has also reduced the time to qualify an opportunity because it can summarise our solutions allowing a new salesperson to align a solution to a client problem.”
The battle between tradition and modernity
The promise of the AI-coupled search-verse and RAG-verse aside, it’s never enough to deploy modern tools and expect transformative change to follow. To get the most of modern tools, you’ll need a modern mindset. Traditional KM, however follows a rather centralised, command-and-control approach.
We find it surprising that despite all the consumer patterns of knowledge sharing that we see on the internet, many knowledge managers still hang on to a rather outdated notion of enterprise knowledge sharing. Let me list a few such practices that you may find familiar.
Gate checks and approvals for what can and cannot list on the company knowledge platform.
Treating user-generated content with suspicion. “On brand” and “official” content rules!
Excessive focus on a predetermined information architecture.
Add your favourite KM peeve to that list.
The battle between traditional and modern KM, is a bit like the cathedral and the bazaar that Eric Raymond referred to, in his 1999 essay about open-source software. Interestingly, that text inspired people like me to think differently about software, when we started our careers in the tech industry. I find it amusing that 25 years hence; I have to invoke Raymond to make my point. Here’s how the cathedral style of KM differs from the bazaar.
| The cathedral style | The bazaar style |
|---|---|
| “Experts” tightly control knowledge, and it moves through formal, top-down channels. | A diverse group contributes and shares knowledge in an open, decentralised way. |
Look around at all the disruption we’ve seen in the tech industry over the last five years. Experts can’t get around this pace of change, fast enough. Nag and I believe that a bazaar style of KM can respond to such change in organisations and industries much better than the cathedral style.
We needed to express our sentiments about this bazaar style model of KM. So; we went about writing a manifesto. Why not, right? The rest of this article presents our first draft of this manifesto and the principles that accompany it.
The AI-first KM manifesto
AI is changing the way the world creates, consumes and finds knowledge. Through our experience of the AI revolution, we’re coming to value:
| Freeform knowledge creation | over | specific templates and formats |
|---|---|---|
| All knowledge open by default | over | access only on a need-to-know basis |
| Speed and relevance of knowledge retrieval | over | volume and elegance of documentation |
| Frictionless interactions | over | permissions and processes |
| Emergent structure | over | predetermined information architectures |
While there’s value in the items on the right, we value the items on the left more.
Principles for bazaar-style (AI-powered) KM
People create knowledge all the time, often in the flow of their work. They are doing so right now!
Modern tools allow users to create, curate and signal the usefulness of content, allowing the most helpful content to bubble to the top.
The structure of a knowledge base should change with the business. AI can help curate this evolutionary knowledge architecture.
AI can automate several administrative and librarian-like functions of cathedral-style knowledge management.
An enterprise’s corpus of knowledge should combine publicly available information with the firm’s proprietary knowledge.
People enjoy sophisticated user experiences on the consumer web. To promote adoption, a company’s knowledge infrastructure should be as low-friction as possible.
People expect to find content more through chatbots, search, and social networks than through structured navigation.
In technology firms, locating expertise is just as crucial as finding helpful content.
One size doesn’t fit all. People should be able to personalise their interactions with the company’s knowledge stack to suit their work styles.
Knowledge managers must be community evangelists who enhance the experience of interacting with the company’s knowledge stack.
Citizen developers can address niche knowledge management use cases through AI and low-code tools.
AI has limitations. Garbage in, garbage out. Long-form content that SMEs curate can help prevent AI hallucinations.
Like the async-first manifesto, our KM manifesto could also use some feedback and tweaks. So, if you’d like to contribute and improve this document, send me a collaboration request for this Google doc. I’d love to hear how you think AI-powered tooling can bring us closer to the bazaar style of KM.