Four goals of agile documentation

Documentation is a vast area. It can be done in lots of different mediums, describing many different aspects of software for various stakeholders. Here, we focus on how developers can use high-level documentation to communicate inside a team, and with direct stakeholders. All of these are things that can complement the documentation already provided by code, scripts, and tests.

1. Create a common understanding

I often catch myself working under the assumption that everybody on the team has the same understanding of what we are doing. “Surely the view of the architecture in my head is clear to everybody?” That would imply we don’t have to write these seemingly obvious things down or have a sketch on the wall. This is a fallacy that, especially more tenured developers on the team are prone to.

Architecture show-and-tell

To verify how accurate our assumptions about common understanding really are, you need to get each team member to lay out their understanding. You can do an exercise in your team where you break into groups, have everybody put together their view of the architecture and then present back. This can help surface misunderstandings and complement each other’s knowledge. You also come out of it with an overview picture to evolve and use in future discussions and on boardings.

The wall of common understanding I always like to reserve some space for a “wall of common understanding” with overview pictures of core concepts. The contents of this wall depend on the software at hand. There’s usually some form of a systems overview with external dependencies, frequently a component and technology stack overview. If the environment setup is elaborate, you might put that up; if it’s really simple, you might decide not to waste space on it.

Regardless of what ends up on this wall, it’s important to set a space constraint. For example, if the team is co-located, this could be a piece of wall space that holds six or eight sheets of paper. Then you fill that space with everything that you believe everybody on the team should know about the state of the system you’re building. The space constraint is useful because we can usually only keep a limited amount of things at the forefront of our minds anyway. The goal here is to identify the essence of what everybody should know so that you can work towards your goals efficiently.

Doubling the space just because you’re working on a very large system will dilute that purpose.

Identifying candidates for the wall

● What are you always explaining to new team members within their first week?

● What do you often need to explain to stakeholders to give them a high-level overview of the system?

● What does not change frequently?

2. Surface and understand complexity

As developers, we strive for simplicity in our software, and “self-documenting,” well readable code. In spite of this, we’ll always end up with parts that aren’t trivial, yet critical. Creating documentation for these parts, in particular, is important. That makes it not only easier to explain them, but also helps to evaluate if the implementation makes sense, and to uncover accidental complexity.

Infographics

The format of “infographics” has helped me with this type of documentation. 
Take an example of a team building an offline-first web application. In this architecture, we had modules for data synchronisation, data model migrations, and conflict, all of which are inherently complex concepts. We built them in the beginning, and then rarely touched them as the development continued. Using infographics to describe these concepts gave us a concise overview without simplifying things too much and hiding the complexity. They’re a mix of graphics and text, with a relatively high level of detail. The reader is supposed to take some time to read and absorb them. This is in contrast to the more high-level overview diagrams mentioned before, where you usually aim for a quick overview.

Identifying candidates for infographics

● Complicated things you don’t touch frequently, but every time you do touch them, you have a hard time remembering how it works. The more critical a component like this is to your application, the more important it is to document it.

● You just designed a new part of the system, and you want to check if the design’s simple enough? If you can’t come up with an understandable

infographic, that might be a design smell. The thought process of creating the documentation can help make the design even better.

What type of information you put on the wall or turn into infographics can also depend on factors like the seniority of the team, the frequency of change in team composition, the type of application, phase of the project (e.g., how old is the code), or your business domain.

Paper widget kits

Another aid I’ve used to facilitate the frequent explanation of a non-trivial part of the architecture is a set of paper widgets. You can use them to build up a picture step by step, as part of a conversation. Our “explain our data model” kit for example contained screen sketches, data entities on index cards, and a set of cards representing our data import jobs. We used it to not only explain the model, but also talk about where the data came from, and what it was used for.

Identifying candidates for widget kits

● Parts of the software that has a lot of history, and lots of things “between the lines” that are hard to put in writing.

● Things that I prefer having a conversation about, instead of giving somebody a wiki entry to read.

● Things with a large scope, where a step-by-step explanation helps understanding.

3. Create empathy

Documentation creates understanding - not only an individual’s understanding of the code or the architecture but also understanding in the sense of empathy between the people involved in building it.

The empathy between tech decision makers and developers

I finished off ​ Part One ​ of the article with an example of how I used a paper widget kit to efficiently explain a data model. That particular kit had a purpose beyond the basic communication of information. The database technology had been quite an important strategic decision in this company, but it was causing a lot of effort for our team. It wasn’t the best fit for the part of the system we were building. The kit and a step-by-step approach gave us an effective and repeatable way to bring across our requirements and create empathy and understanding for our challenges quickly. It helped to keep emotional factors out of the repeating discussions and keep them efficient and fact-based.

"Empathy among developers

Working on software without guidance, without documentation,

is anxiety-producing."

In her ​ article about "Empathy Driven Development"​ , Duretti Hirpa describes the notion of an “empathetic codebase”. Creating a codebase like that means taking advantage of “any tool that will provide context”, including documentation. “Docs or it didn’t happen”.

By creating documentation, we show empathy for our fellow teammates and create an environment where people feel guided and safe when changing the software, especially the newer members of the team. This goal applies particularly to the category of documentation for onboarding and troubleshooting, like “readme” files, or checklists for things that are not automated yet.

The empathy between developers and non-developers

It’s not uncommon for developers to complain about their product managers, saying they “don’t really understand how the software actually works”. Documentation can help developers bridge that gap.

Here is an example: When building a PDF document generator, the product owner stopped by our desks almost every day to report yet another case where they didn’t like how a particular “page break” was placed by the layouting algorithm. Sometimes a fix we did today would bring back a flaw we tackled the day before. To facilitate those discussions, we created a poster outlining the logic of the “page break” algorithm. It helped us show how introducing new rules could invalidate existing rules, and we could then discuss priorities on equal footing. Making the logic transparent helped the users empathize with us because we showed them the complexity in a more understandable way, and they also understood the scope of the implementation better. It made us feel like we were creating this feature as a team, not as two opposing parties of “feature requesters” and “feature implementers”. Slack’s overview of how the software determines the need to send a notification​ is another nice example of this.

Identifying candidates for this type of documentation

● When a lot of bugs are reported, that turns out to be caused by the misunderstanding of a particular feature.

● Features that are going through a phase of change and consist of an elaborate system of rules and logic. Everybody involved needs a good understanding of that logic to work on the change together.

4. Help our future selves make informed decisions

“If you look at another engineer's work and think, ‘That's dumb. Why don't you just...’ Take a breath. Find out why the problem is hard.” ​ Adrienne Porter Felt Creating an overview that demonstrates the complexity of your solution, as just described, also helps create empathy with other technologists. It helps them understand why the problem you’re solving is hard, which is a good foundation for a constructive discussion.

But what if you can’t understand why the problem is hard, because nobody knows what the problem was in the first place?

Architecture decision records

In his 2011 ​ blog post about architecture decision records​ , Michael Nygard writes:

“One of the hardest things to track during the life of a project is the motivation behind certain decisions.” But, he adds, unless you understand that rationale, you can only either accept the decision blindly or reject it blindly. Writing down the reasoning, context, and consequences of a decision right after you take it can hugely improve future decision-making.

You can record decisions with many tools, from a Word document to a Wiki, to simple text files. I’ve successfully used ​ this little tool​ to create markdown files that were then version controlled along with the code. However, the tool isn’t so much the challenge here. The difficulty is to get into a routine of creating this really valuable form of documentation. We have to overcome the urge to just get on with the execution once we’ve come to a decision, and pause for a bit of documentation while

the details are fresh on our minds.

Identifying candidates for decision records

● Decisions that needed a lot of discussions to get to

● Decisions that are harder to change

● Imagine somebody joining your team in the future: Do you think they would challenge this decision without having more context?

● “Whack-a-mole” decisions: Every two or three months a team member brings them up and wants to change them, only to find out after a day of revisiting that there were actually good reasons for the decision. Having a clear record of the original decision and requirements makes these regular challenges more efficient. It then usually boils down to checking the documented list of requirements in the record and asking “Has the problem or the context changed in any way?”

Once you get better at identifying these moments, remember not only to describe the solution you went for, but also the problem and the context. For example, if you write down that you chose technology X because it’s scalable and supports Docker, then you’re actually listing characteristics of your solution, but not really why you need them. Why do you need scalability, why do you need Docker support? That reasoning will help your future self decide if circumstances have changed in a way that allows for a less scalable solution, or for one without Docker. The Nygard-recommended

structure of “​ Context – Decision – Consequences​ ” helps to focus on a problem description.

How to keep it up to date?

One of the most frequently asked questions with regards to documentation is: How do I keep it up to date? The short answer is: You probably don’t - at least not 100%. In the end, anything that is not necessary to keep the software running will ultimately be out-of-date to some degree, so executable documentation forms like readable code, tests and scripts are important foundations. The techniques described here are all just complementary to that.

The following are a few principles I use to try and keep the more high-level, non-executable documentation reasonably up to date.

Create as little as possible

Having as little documentation as possible is the only long-term protection for outdatedness. And the more details you add, the higher the probability that you will have to update them soon. Again, think about the value a piece of documentation or detail brings, and remember that you will have to maintain it, just like every line of code you’re writing.

Also, don’t be afraid to throw things away! Some of the examples described earlier are actually only useful temporarily, it’s okay to throw them out when they aren’t helpful anymore.

Include documentation grooming in team rituals

Find ways to include the grooming of your documentation in your team rituals. I find a weekly “developer huddle” useful for this. This is a meeting in which the team discusses technical topics and questions that have come up during the week. You should also make good use of team member rotations to check the current usefulness and understandability of your documentation.

Make it visible

If a piece of documentation is buried in some corner of your wiki that you rarely go to, then it will most certainly wither away. If it’s up on the wall on the other hand, for everybody to see, or in a “readme” file that every new team member comes across, outdatedness will be noticed and corrected more frequently.

Create ownership through collaboration

Make sure the creation of documentation is a collaborative activity, to create collective ownership. This will increase the probability that all team members have an eye on how up to date the docs are, and correct them when necessary. If only one person on the team is creating documentation, they won’t keep up in the long term.

Agile + Documentation = <3 font="">

Ultimately, the code is the only truth describing the current state of your systems. High-level documentation serves as maps to find entry points and your way around. And you need to put special emphasis on documenting the things that code cannot tell you: History and context. The self-documenting code does not help you challenge past decisions, or understand why things were done a certain way. Therefore, at the minimum, consider writing decision records, and take good care of your commit messages.

I hope I could show with these examples that the process of creating and grooming documentation can be a great catalyst for some of the other ​ agile principles​ , like “Simplicity”, “Business people and developers work together daily”, “Attention to technical excellence and good design”, or “The team regularly reflects on how to become more effective”.

If you’re somebody who tends to create a lot of documentation, and you want to reduce some waste: A focus on the value documentation will help you prioritise. If you’re somebody who creates very little documentation, reflect on if you’re missing some of the values mentioned, and what types of documentation would improve your team’s effectiveness.