Understand That You Need Documentation
Get better quality information into people's hands and they will make better decisions.
– DisciplinedAgile
From the analysis perspective, having concise, up-to-date, and easily understandable documentation of the current state has an enormous impact on analysis effectiveness. This is because finding the best solution to business needs always requires knowledge of how things work at the moment. This involves understanding not only the IT systems but also other important parts of the organization, such as processes, rules, or organizational structure, for example.
Even though we usually implicitly think of documentation as a snapshot of the current state, another interpretation is a development specification or generally a to-be state. During solution analysis and implementation, teams produce a lot of information that is worth keeping. The amount, form, and timing of the documentation cannot be prescribed and depends on many factors, such as the size of the project and team, relationships with the stakeholders, or the selected software development approach.
Reasons for Documenting
1. People Are Not Always Available
Keeping information in people's heads, be it the project knowledge or knowledge of the enterprise, is risky. If the information is not recorded, its availability depends on the availability of the people; therefore, the information may be temporarily unavailable if the right person is out of the office, or permanently unavailable if the person leaves the company.
On the other hand, it is not possible to document all the information available. There will always be facts that remain undocumented, shared only by word of mouth, or stored just in the information system source code. This mainly applies to details that are used daily by ordinary employees and are therefore easy to obtain. General information, on the other hand, spans multiple departments, processes, or information systems and is usually known only to people in management positions. These facts are much harder to obtain, which is why they are worth documenting.
This applies not just to as-is documentation but also to documentation produced during projects. Even if the project team is composed of t-shaped stars who can collaborate and share information effectively, nobody can guarantee that the team will remain unchanged until the end of the project. Especially in long-running projects, one must be prepared for situations in which half of the business stakeholders and two-thirds of the development team leave the project or even the company. The high effectiveness of the team gained through not burning any time on documenting things is suddenly worthless if all crucial project knowledge is lost.
2. Project Finishes, Product Continues Deploying the solution, such as a system or a modified process, is the primary goal. But no matter how much documentation was needed to build the solution, the secondary goal is to operate the solution and extend it over time. These activities are very likely to be done by people who were not part of the original implementation team, and they will need relevant information to fulfill their tasks.
Software is "done" in the same way that the dishes are "done". Software, like clean dishes, is only done if you never plan to use it again.
– Dave Cheney
Additionally, the new solution will most likely not exist on its own and will probably be part of a bigger ecosystem. So it is not enough to document just the solution itself, but it is also crucial to describe how it fits into that ecosystem and how it influences it.
3. Record It When You Know It Let's assume that the project team is a dream team, the cooperation with stakeholders is brilliant, the team is delivering what is required, and they are delivering it on time. This setup should be supported as much as possible, and the team should not be bothered with documenting things because it would only decrease effectiveness, right? Not really. The team now has exceptional conditions: the members have the knowledge of the solution and they have access to the important stakeholders. Never again will anybody be in this position, so documenting the solution will only become harder and harder.
The cost of recording knowledge is small compared to the cost of acquiring that knowledge or regenerating it at some point in the future.
– Karl Wiegers
4. People Forget
Each project takes some information as input and produces some information as output. If the source of the input information is only people and the output information is shared only in the team members' heads, it is risky. People tend to twist facts, both intentionally and unintentionally, and they forget, both of which influence projects negatively. If the information is recorded, it cannot be malformed that easily.
Agile Traps
Agile has truly changed the way teams deliver solutions. The process is more flexible, teams focus on important things and do not waste time creating complex specifications upfront because they know there is a chance they will be out of date by the time development begins. It is therefore natural that teams working in an agile fashion produce less documentation, as they do not document every single requirement, and instead, they rely more on communication and collaboration. However, teams also started to misinterpret agile, especially one of the core agile principles:
"Working software over comprehensive documentation" (see Agile Manifesto)
Many teams have started to interpret this principle as "Documentation is a waste of time; we just need to be delivering solutions". This is very short-sighted, serving only as a justification for lazy teams not to document their solutions. As we already mentioned, the solution life cycle does not end with deploying the solution and giving it to users. Each solution is going to be operated in production, maintained, and extended. If this is not easily achievable, the solution has failed.
Another common misinterpretation is "We don't have documentation because it is all in the source code". Even though the source code is really the only documentation which always accurately interprets how things work, new team members cannot learn the basics directly by examining the source code, as it is too low-level:
"We don't have documentation, look at the source code"
Source code can very accurately tell what the application does, but it cannot explain why - it cannot interpret the context and what the role of the system is in the whole IT ecosystem. Source code is also useless for teams that are responsible for the maintenance of the solution.
Agile principles should not be taken literally. Instead, they should be interpreted as something to prefer. Agile teams prefer to present working software rather than going through comprehensive documentation. They also prefer t-shaped people who can look at the source code instead of asking five different people for help. But the world is not black and white, and true agilists understand that not everything preferred is always feasible.