Effective Analysis and Documentation

Effective Documentation

Producing quality documentation that provides value and is easy to read and maintain presents a nontrivial challenge. This chapter summarizes principles and practices that support these attributes by applying simple, lightweight, and iterative techniques and approaches.

1. Document General Concepts

Every type of documentation, be it for analysis or post-release, is always to some extent inaccurate, since it is merely a simplified model of reality. Is this a problem? Documentation reflecting reality 1:1 would be ideal, but it would also be unmanageable, as updating it every time a tiny detail changes would not be realistic. General concepts, on the other hand, do not change that often. They serve as an information hub that redirects the reader to more detailed resources if needed, which also makes them excellent for introducing new team members to the project.

Overviews are very useful and easier to manage; details are only sometimes useful and difficult to manage.

More is not always better. Maintaining very detailed documentation represents a significant overhead that must be justified, regardless of whether we are talking about pre-release or post-release documentation. It is safer to document only general aspects and include details only if it is justifiable and the information is stable.

2. Document Stable Things

During analysis and development, information changes frequently as more facts are discovered and clarified. With the increasing adoption of agile approaches, information turbulence during development is even more common, and what is more, it is welcome. However, this also implies a need to change the way we analyze and document solutions. It is ineffective to document details, create perfect models, or write formal specifications if they are likely to change soon. Instead, it is reasonable to wait and document only what is stable enough so that unnecessary rework is avoided.

3. Iterate

Complex problems cannot be analyzed in one go and should instead be broken down into smaller chunks and analyzed iteratively, from high-level to detailed. The same approach applies to documentation. Documentation is never perfect on the first attempt and is usually developed over time. It is continuously revisited and iteratively improved until it reaches a decent quality.

4. Document Rationale

Documentation is primarily focused on capturing What and How—what organizational components we have, what their capabilities are, and how they internally work. But in most cases, this is not enough. Unless the documentation includes the reasons why things are the way they are, the information is not trustworthy for readers. They will ask for justification, such as who approved the design or where this information comes from. As a result, it is equally important to document the Why. We elaborate on this topic in a separate practice here.

5. Document With a Purpose

Creating documentation comes with a commitment. Each time you document something, no matter if it is a process, a system, or a low-level algorithm, you publish an artifact that you guarantee to be reasonable, correct, and of decent quality. Fulfilling this commitment takes time. That is why you should always ask first what the purpose of the documentation is, whether it will provide value, and whether it is really needed. Don't create a sequence diagram for each use case just because your team leader instructed you to do so if you are sure the diagram will not bring any benefit. Documenting must always have a clear purpose and provide value that exceeds the cost of creating and maintaining it.

Below we list situations in which creating documentation is justifiable:

  1. To analyze something: Analysts, software architects, developers, and others all need to analyze or design aspects of the solution before moving to implementation. For instance, it is hard to just sit down and implement a complex process without discussing it with stakeholders and designing it with developers. Creating a process model is a convenient way of verifying the design, and it will also serve as an input for developers.
  2. To document organizational capabilities: It is a hard task to discover how the organization must change to meet business needs without knowledge of what the organization is currently capable of. It is not possible to find the best solution without understanding what the company currently has and what it can currently do. Each organization should have accurate and up-to-date documentation of its processes, systems, guidelines, rules, and people.
  3. To define contracts: Communication between two entities is usually done through a formal interface. Whether waiting for an external sub-process or calling an external system, a fixed contract is needed so that both sides understand their roles within the relationship and the method of communication. Since the contract is binding, it is convenient to document it and share the specification with all parties involved.
  4. To conform to regulations: The organization may also be required to create specific documentation to prove it has been following specified rules. This may be to prove to customers that the solution meets agreed requirements or to confirm that the company conforms to regulations issued by the government or other third-party entities. Since the documentation is obligatory, the organization has no choice other than to create it, but the team should always verify first whether the regulation is still valid to avoid unnecessary work.
  5. To communicate: Communication through exchanging documents should never be preferred over personal communication, as personal communication will always be much more effective. Requirements should be reviewed and verified in a meeting, not via commenting and exchanging specifications. Nevertheless, face-to-face communication is not always possible. Business stakeholders may not be available at a given time, communication across time zones is complicated, and sometimes formal confirmation is required. As a result, documents are still a valid way of communication in specific cases, yet it is neither preferred nor recommended.

While it is justifiable to create "a document" in the cases presented above, in the following situations, you should seriously consider whether documentation is really the best approach:

  1. Documentation is ordered: If the team is ordered to create some type of documentation (there is a guideline, or a manager said so), it should ask if the guidelines are still valid, who exactly is going to use the document, and whether creating it makes sense at all.
  2. Documentation is a custom: Many teams create documentation just because they were told to do so in the past, so they continue to do it even though there is currently no legitimate reason for it.
  3. To prove productivity: Some teams produce a lot of unnecessary documentation just to prove to managers or customers that they have been working hard. Progress should be demonstrated by a working solution, not by models and documents.
  4. To freeze requirements: Teams working in a waterfall fashion create a "requirements document," make the customer sign it, and then try hard to deliver exactly what has been approved. However, fixing the scope by freezing requirements has proven not to work. Good relationships with stakeholders, collaboration, and following a common goal through iterative analysis and development are far more effective approaches. Freezing requirements is very likely to turn against the team, as the customer will change the requirements anyway. The team will then have trouble delivering the solution since the documentation will inevitably lack information, which will cause delays.
  5. To communicate: See point 5 in the previous list.

6. Keep Documentation Simple

Because repetition is the mother of retention, let's remind ourselves again: documentation is not the primary goal; its purpose is just to support teams in finding and implementing solutions to business needs. For this reason, documentation should be as simple and straightforward as possible, while still providing value and meeting the required quality attributes.

  1. Take an iterative approach: create the most essential documentation, review it with your audience, and add details only if required.
  2. Use lightweight techniques: simple statements, bulleted lists, sketches, basic use cases, etc.
  3. Don't be afraid of simple analysis and development documentation. Projects usually do not fail because of simple documentation, but we have seen many projects fail due to wasting time on comprehensive specifications instead of focusing on delivering the solution.
  4. Avoid duplications.

7. Review and Keep Up-to-date

If the goal is to maintain a high-quality standard for documentation, it is inevitable to put a review process in place. It is a tough task to write unambiguous documentation that will be clear to all readers and to synchronize the look and feel of documentation across the organization. Letting somebody review the produced documentation is beneficial for identifying ambiguities and evaluating overall documentation quality. It is especially important for as-is documentation whose lifespan will be a couple of years, so the effort put into the review will pay off.

8. Well-Arranged

All the effort put into creating quality documentation may be wasted if it is poorly organized. It is necessary to structure the documentation coherently and divide it into logical parts so that it is easy to navigate. The concrete structure depends on the needs of the particular project or organization; we can only provide you with examples of analysis documentation and post-release documentation.

It is also crucial to have a clear "starting point" for each logical component. Its purpose is to provide an overview serving as an entry point from where the reader is referred to more detailed parts, so that information is "layered" and the reader is not overwhelmed with details right from the beginning.

9. Structured and Interconnected

Documentation is not a monolith. It consists of several layers which allow viewing the subject from different perspectives. The layers could be horizontal and vertical.

To maintain good readability and navigability within the documentation, links between the layers must be created and maintained. It should be easy to navigate from one aspect to another and from high-level descriptions to more detailed views.

10. Unified

Team members come and go, documentation is handed over from one person to another, and it is shared among various parties. Besides, it is usually necessary to read multiple documents to understand the big picture. This is why documentation should be as standardized and unified as possible.

11. Public and Easy to Edit

"A common mistake that I've seen architecture groups make is to put their diagrams on the walls within their work areas but not the work areas of the application developers."

– Scott Ambler

The information included in the documentation is only beneficial when it is easily accessible to everyone interested. Nobody will review diagrams if they are stored in a complex modeling tool that requires training just to open the diagram. Business stakeholders will not help identify bugs in the documentation unless it is presented to them in a form they are familiar with. Besides, documentation of all components must be accessible from a single place, so there is no need to search for specific documentation, ask for credentials, etc. An example of such a solution is presented in Part III.