Agile planning is hard for enterprise and cloud-native development. Enterprise and cloud-native products share the common characteristics of distributed systems with multiple runtimes and networked traffic. For example, a simple 3-tier reference architecture on Google Cloud consists of two Cloud Run services, one Cloud SQL instance, and one Cloud Memorystore service. For the build process, it consists of Cloud Build and Secret Manager. There are more services to consider for networking and security. A Team or individual may be responsible for each of these services. A new feature or enhancement may affect multiple services. How do you plan and track the progress of features?

Problem

Each Team or individual creates stories or tasks for their systems and tracks their progress. This siloed tracking makes it difficult to track the overall status because the feature status has to be deduced manually from many different tickets and is subject to interpretation.

The second issue is with measuring metrics such as lead time. How long does a change take to complete? We know the average task or story completion time, but often adding them up does not give an accurate picture because different Teams may have worked on the change at various times, which means, in reality, the lead time was much longer. The lead time could also be shorter than the sum because some tasks were completed in parallel.

The issue with being unable to measure lead time is that it hinders resource planning. Management may think we need more developers because of slow lead time, but the delays may be because there was not enough breakdown of tickets, large tickets were in progress for a long time, or there was too much work in progress.

Solution

Epics are ideal for tracking the completion of a feature. Epics are just big stories, but nothing prevents them from being as small as a story sometimes. They have traditionally been used to track a group of stories, so they align with both Scrum and Kanban methodologies. Unlike stories that ideally need to finish within a sprint, there is no pressure to appropriately size epics.

The Epic captures the behavior of different systems in tasks or stories. For example, adding a new customer may mean adding them to the authentication, feeds, and metadata management systems. Adding a new customer is done when all three are done.

Much of the consideration for the story is taken up at the Epic level. For example, the acceptance criteria, system tests, and the definition of done are applied to the Epic.

Recommended Workflow

While there are many ways to implement an Epic-based workflow, a simple Kanban workflow that may work is as follows.

• A Feature is only documented, not ticketed because a feature has a different lifecycle than a ticket. Features are typically upgraded over time due to business or compliance needs unless retired or obsoleted.
• A Feature is implemented as a series of Epics. Each Epic has a start and target end date.
• An Epic is scheduled for development before work begins. At this stage, the Epic is broken down into stories or tasks.
• Stories and Tasks move from backlog to in-progress to complete.
• When a story or task is in progress, the parent Epic is also in progress. The Epic is also done when all the stories and tasks are done.

Other Considerations

• An Epic is a good candidate for evolving the system's architecture since the Epic potentially affects multiple components.
• When multiple people work on an Epic's stories or tasks, who owns the Epic? Each Team may solve it differently - a team lead, a product manager, or someone else. I recommend the most affected component's Lead programmer to be the Epic owner, but if all components are affected equally, pick the person responsible for all of the components.
• Epics can be demonstrated to show progress to stakeholders and get end-to-end feedback.