The case for project archaeologies
Summary
We often overestimate the value of grand documents over the unglamourous acts of daily documentation on projects. With the advent of AI-powered KM platforms, it’s time to revisit the fundamentals of team handbooks and documentation systems, if we are to generate relevant and timely knowledge insights for people.
I’ve spent my entire career in the tech services industry. We’re the aristocrats of gig work, though that could change very soon. For tinkerers and lifelong learners, the services industry affords an interesting value proposition:
Build experience solving a variety of problems, leveraging all sorts of technology across many domains. All in a relatively short time!
Indeed, this is why many startup founders and CTOs may have started their careers in IT services.
For the tech consultancy, though, it’s a business of highly optimised knowledge work. Clients believe their problem is novel, and they may be right, but the consultancy benefits from a repeatable approach to solving each class of problems. You can’t reinvent the wheel each time you implement, say, a data mesh solution for a banking client. Or, for that matter, when you run an assessment to scope a transformation program. If such activities are too custom, the consultancy runs into many problems.
Solutions take time to implement.
The engagement needs several senior professionals.
The team’s leverage model and profitability take a hit.
Projects cost far more than the client’s spending appetite.
Knowledge management (KM) is vital in professional services firms, to meet this need of repeatability. KM’s job is to “codify” the firm's knowledge and practices so employees can service their clients almost on autopilot.
Of course, knowledge doesn’t appear out of thin air. A consultancy firm builds differentiated services through experience. One project leads to another; if organisational memory is strong, these experiences culminate in repeatable practices and assets. A company, however, is an abstract entity. Since people join and leave the company, there must be a way to baton-pass knowledge from one group to another.
Now, professional services firms employ many tools and tactics to orchestrate this baton pass, but in today’s post, I want to address a popular theory of knowledge transfer. I call it the dazzling doc theory.
Dazzling docs to the rescue
Let’s zoom into the experience of a consultant on a professional services gig. At a simplistic level, they go through a few steps for each new gig they go on.
They search and evaluate gigs to zero in on their next assignment.
Their new team onboards them to the project.
The consultant works on the team for a few months or years.
And finally, they roll off the gig.
Here’s the theory. What if every consultant could write about their experiences and learning on the project when they roll off? Wouldn’t the rest of the company benefit from it? Surely, other consultants joining a similar project could consume this document and learn from the author’s experience! Moreover, if everyone wrote such a roll-off review, imagine how much knowledge a consultancy could build up about their accounts, their work streams and, by aggregation, all the company's work!
If only a consultant would write that dazzling doc, all would be good with the world!
But the docs don’t dazzle
The dazzling doc theory is alluring, no doubt. Every services company has toyed with it at some point. During my stint with one of the big four consultancies, I remember consultants writing post-mortems and experience reports at the end of studies and engagements, which would go into a centralised repository called the “Knowledge Network”. My employers and other tech consultancies have also tried variations of this practice.
That said, the documents haven’t dazzled. Their utility is limited, especially in tech.
Biases and limits of memory: A professional services gig in tech can last anywhere from six to 24 months. When people write a report about their experience after such a long stint, they often over-index recent events and forget about more impactful events that may have occurred earlier in their tenure.
Inconsistent quality: Technologists aren’t often the most diligent writers. While some people may produce a high-quality writeup, others may perform this activity as a box-tick exercise.
Context specificity and confidentiality: In consulting, context matters. Specifics about the domain, tech, methodology and the client’s problems are essential for a nuanced understanding of a project. But when a consultant writes up a report for a generic audience while respecting client confidentiality, they end up writing generic bullshit. Here’s an example of such writing.
“We managed the transition from project to product-centric teams and redefined how they prioritise and manage investments. After an assessment, we created tools and processes as well as a product management playbook to support newly formed teams. The playbook provides guidelines for program management, funding, and delivery, resulting in $11 million in incremental value in an inventory quality initiative which initially predicted a $7 million revenue increase.“
That text may look impressive on a marketing brochure, but it’s useless to a fellow consultant.
Time constraints: Getting into the details isn’t easy either. Most people write roll-off reviews between gigs. It takes time to compile rich reviews, and often, there isn’t enough time for people to devote to such thoughtful writing. Take my example. I spent 16 years at my employers before my first stint on the beach. I’ve never had the dedicated time to write such reviews.
The pace of technological change: Have you noticed how fast technology has changed in the last year or so? If someone spends a year on a project, some of their learnings are already dated by the time they’re ready to write a roll-off review. What’s even the point of writing up these dated insights, then?
As you can imagine, the ceremony of the roll-off review may not live up to its promise. In my years as a consultant, I’ve rarely searched for someone else’s experience report. I’ve asked people pointed questions, but I haven’t sat myself down to read a dazzling doc. If you work in the services industry, you’ll probably agree that your experience is similar to mine. But then, what's the solution if dazzling docs don’t live up to their promise?
Daily discipline will beat dazzling docs
Thankfully, KM tech has improved in leaps and bounds over the last few years. AI-powered search and retrieval augmented generation is making it easier than ever before to make sense of company and team knowledge. In 2025 and beyond, agents are the new show in town. Using platforms like Glean, employees can create no-code AI agents that perform complex tasks and automate business workflows while using the company’s knowledge bases.
These new KM capabilities should encourage us to improve our rigour of effective team documentation. One of the first chapters of my book is about creating team handbooks - simple microsites that organise the team’s knowledge in a navigable form. You can spin up these handbooks in a matter of minutes using tools like Notion or Confluence.
Once you have a team handbook, the team can create artefacts in their workflow. The DEEP acronym helps you remember the triggers for documentation on most teams.
D for Decisions. Teams make many decisions over their lifetime. Each time a team decides something, they can document it so that someone who wasn’t involved can understand the rationale for the decision.
E for Events. Meetings, retrospectives, post-mortems, workshops, and town halls are all events. By documenting such events, teams preserve the outcomes of the interactions, learnings, and knowledge.
E for Explanations. Every project has a body of knowledge. Instead of sharing such knowledge verbally, teams can write things down. Or record them. Create once, share many times.
P for Proposals. During a project's lifetime, the team and its stakeholders will share many ideas and plans. Whether a team implements these ideas or not, documenting the thought process and details of such proposals helps foster decision hygiene and builds the team's collective memory.
None of these documents are complicated to create. Tools like Confluence provide ready-made templates to simplify such daily documentation. And what’s the KM benefit? Well, the daily documentation discipline builds a clear project archaeology for AI-powered search, chatbots and agents to leverage. Not only are these artefacts essential to a successful project, but they also build up a real-time, relevant knowledge base for the rest of the company.
When teams build knowledge in the flow of their work, AI can generate relevant and current insights for everyone
The time of big, heavy project documentation is behind us. Instead, we need relevant, just-in-time documentation that modern, AI-powered systems can synthesise and make sense of. The humble, unglamorous daily discipline of creating team artefacts has far more value than the mythically awesome roll-off review. If you’re responsible for knowledge management at a professional services firm, encourage your colleagues to build up their team handbooks. Any AI-powered KM solution will benefit more from this project archaeology than it will from a dazzling doc approach.