Agile story grooming checklist

Stories come in all shapes and sizes but it’s easy to miss hidden complexity in all of them.

Anyone who has worked on an agile software development team will likely be familiar with the process of Backlog grooming. Whilst the specifics of how the backlog grooming sessions are carried out may vary from team to team, usually one of the key aims is to break down features into deliverable chunks (i.e stories) of valuable working software and assign an estimate representing the effort involved in delivering each chunk. It is usually the estimation portion of these sessions which prove to be deceptively problematic for teams, regardless of which estimation technique is chosen.

The purpose of this article is not to offer an opinion on estimation techniques themselves, rather it is to highlight something I’ve witnessed across different teams (and fallen victim to myself) when it comes to estimating stories, and which I believe can often be the cause of skewed estimates:

“We tend to base our estimation of effort on how much code will be required, but omit other other key factors required to get the feature into the hands of our customers.”

What do I mean by that? Well, usually backlog grooming sessions are run by the product owner, who will present features / stories to the engineers in terms of what problems they would like to see the product solve for our clients. For example, the PO may declare that they would like “our clients to be able to capture their customers marketing preferences in a manner compliant with GDPR regulations”. Usually these stories are received in one of 2 ways by the engineers when they are first presented:

1. They identify this to be a really straightforward story whose criteria can be met with a minimal amount of coding. Happy to have solved the problem so quickly, they can be lured into immediately declaring it to be a 1 pointer, taking additional positive reinforcement from the PO’s reaction to this low estimation. The PO then moves on to discussing the next story with everyone hoping it too will be just as ‘easy’.
2. There’s a collective, sharp intake of break, realising that this story poses a complex technical challenge to be overcome. Perhaps it requires a lot of code to be written and so they start breaking it down into smaller stories. Or perhaps this will require changing a key algorithm in our code and therefore have to think about all the business invariants that could be affected and how they will be preserved or adapted. In this situation it’s not uncommon, and understandably so, to get a bit overwhelmed with all the moving parts of the problem. So the team’s focus then often tends to narrow in on solving for the happy path scenario. When a seemingly viable solution is finally carved out, usually a large amount of mental fatigue has been accumulated in the process, for both the engineers and the other attendees. As a result, the other aspects of the story can often be overlooked and the team proceeds to estimation based on happy path alone.

Unfortunately, in both scenarios we have focused only on a thin slice of the story and forgotten to consider all the ‘other’ factors that will go into delivering it. I would liken this to falling for the trap Uncle Bob calls ‘Grabbing for The Gold’ . Uncle Bob used the term to describe falling into the trap of jumping into immediately writing code to solve the core problem. In doing so, you’ve ignored the edge cases and it’s these edge cases that will ultimately thwart your ability to deliver a complete, working solution and will require you to change your approach to the overall problem. I think a parallel can be drawn here, numerous layers of abstraction higher, at the backlog grooming level too.

Why does this happen?

In each of the scenarios described above, some key factors of the story delivery process were overlooked during the story estimation process. In my opinion, this can be explained by one or more of the following reasons:

• Favouring simplicity — In the case of simple stories, I think the problem is our brains tend to favour simplicity and therefore internally we take pleasure in solving the problem so quickly and in a way that is clean and without complications or mess. Perhaps we then sub-consciously avoid digging further, for fear of uncovering that hidden complexity which will spoil our clean, simple solution. Another reason simplicity can be favoured is when it’s mistaken it for ‘easy’.
• Mental fatigue — In the case of complex stories, it comes down to the fact they are simply more mentally fatiguing. This fatigue can arise because the problem, by its nature of being complex, can have multiple possible solutions, each with their own tradeoffs. In any good team of engineers, this will result in a healthy debate around which solution is most preferable. Whilst this discussion is good, it takes time and requires us to concentrate on this complexity for longer. The other contributing factor to the fatigue is that problems which are complex in nature, require us to keep so many variables at the forefront of our mind at one time, which is something humans are not great at doing.
• The audience — Another reason I’ve found we can sometimes omit to consider key factors of story delivery when arriving at an estimate, is the audience. Perhaps the engineers feel that getting into the lower level technical aspects will be less relevant for some attendees of the backlog meeting, such as POs and designers, and therefore don’t want to waste their time by forcing them to sit through these conversations.

Avoiding the trap

A while back the engineering teams at Phorest found ourselves falling into this trap every now and again and wanted to avoid it. The engineers in our teams are responsible for the entire lifecycle of a feature, from estimation, implementation and automated testing, to infrastructure creation, deployment, monitoring and maintenance. Whilst it’s great to have such autonomy in our work, it’s also a lot of complexity to manage and it’s therefore even more pertinent that we reduce cognitive load wherever possible.

We decided to create a backlog grooming checklist. It allows us to assess each of our stories against it, to ensure we’ve considered (to the best of our knowledge) all the factors required to deliver that feature into production. This is something we’ve been trialling as we’ve expanded our traditionally single engineering team into smaller sub-teams. The diagram below shows the story grooming factors we decided to include in the initial version of our checklist. For details of the sorts of questions we typically ask ourselves when addressing each factor, see the end of this article.

The story grooming factors that have formed our own checklist.

A couple of caveats I would add regarding the above checklist is that:

1. The factors listed are not exhaustive. Some factors may not be as relevant to some teams, and other teams may need to consider factors that are not listed.

2. Not every story or feature will require something to be done for each factor, but having them listed there, even if for nothing else other than to rule them out, takes away the need to keep them at the forefront of our mind at all times.

Benefits

This checklist could be an artifact that your team produces collectively, a collaboration of members with different technical backgrounds and levels of experience. The end result is that any engineer on the team has at their disposal the collective experience of many other engineers when estimating a story, rather than just their own individual experience. Also, for those larger engineering teams that are split into many smaller sub-teams, it can also act as a way to share knowledge and experience across those sub-teams too. Just as it shouldn’t be treated as an exhaustive list, it also shouldn’t be treated as static. It can evolve, with updates being made to it as the experience of the team evolves.

Where it fits in

I have found this checklist to be useful when used in conjunction with a technical backlog grooming session. This is a session attended only be the technical members of the scrum team, and is a chance to look at the story in a lower level of technical detail, in order to consider if/how each of the factors in the checklist would influence the approach to delivering a given feature. In this session we usually discuss stories which have already been introduced to us by the PO in a recent backlog grooming session. We can then provide our PO with a more considered delivery estimate at a subsequent backlog grooming session.

Conclusion

• Planning a story requires thinking about a many different factors and it can be easy to overlook some of those when caught up in group discussions during the backlog grooming session
• It can be overwhelming to try and keep all these factors at the forefront of your mind during grooming sessions, especially when faced with challenging / complex stories.
• Having a collective story grooming checklist provides an opportunity for team members with different levels and areas of experience to collaborate and produce a more robust checklist, which will hopefully produce more consistent results when delivering stories

I’d love to hear people’s thoughts on this approach and if other teams are already taking a similar approach, I would be interested to learn more about how your checklists look.