"When is it going to be done?”
Peter the Product Manager looks up from his desk. He gets this question a lot. You see, Peter is responsible for building and launching an internal product for Big Company, Inc. This product promises to revolutionize the way Big Company interacts with customers and will create efficiency throughout the organization. Naturally, every department wants to know when it will be done and what they are going to get.
Peter has done his homework. He has talked to users, researched the competing solutions and alternatives, created personas, and run workshops. He also created a backlog of features that he felt were the right ones for an MVP—small but powerful, high value for a manageable cost.
But as development entered its fourth month, with the team deploying production as often as they dared, Peter kept getting these same questions every sprint.
So, as a well-trained practitioner of scrum, Peter sends his burn-up chart, along with a few other metrics, by email to the rest of the company.
Figure 1
Five minutes after hitting ‘send,’ Dave the Developer stops by. "Hey, Peter. You know that our velocity has picked up the past couple of sprints, right? I don't think it's going to take us that long to finish what's in the backlog.” Peter takes another look, "Hm... you're right. I'll update the metrics and resend. Maybe this will make people even more excited!”
Figure 2
Ten minutes after the new metrics were sent, Laurie the Lawyer comes over. "Pete! Saw your emails with the metrics from the project. They're looking good! It got me thinking about this big new client we’re trying to land. If we could roll the product out and give them a look, I think it would put us head and shoulders above the competition. Any chance we can pull that release date in another month? We don't need to have all the marketing and stuff ready to go, but we could really use the features already in production to land this whale."
Pete promises to see what he can do. They are forecasting to be under budget anyway. Maybe if he talked to Chris the CTO and Pam the VP of Product, they can spend a little more money and free up some folks from other projects to accelerate the development effort.
The next day Peter brings an updated set of metrics to his meeting with Chris and Pam. He is confident these metrics will easily make the case for an accelerated timeline.
Figure 3
Chris likes what he sees and agrees that everything appears to be on track. His only question is on scope. The metrics show increasing scope for most of the first four months of the project, followed by a big dip.
It isn’t the fact the scope had increased that worries Chris. That is expected when everyone is trying to build the right product rather than just stick to the plan. What he questions is whether or not the scope is really as stable as the metrics indicate. “I've worked with our team,” Chris said. “They’re creative folks who will keep coming up with new ideas or find more things they want to tinker with. Plus, we've got a bunch of holidays coming up. Are you sure we can get it all done if we move up the release date?"
Pete hadn’t considered those things. Intangibles like creativity don’t really show up in metrics. He shrugged, “Now I'm not so sure."
Context is key
Why tell this fake (but realistic) story? As product managers, our responsibility is to create a vision for a product and then work with our teams to execute on that vision. A big part of that job description includes being able to report on the progress of whatever product we have promised to deliver. Putting aside tactical questions about the definition of done or specific mechanics like ‘sprints’ and ‘DevOps,’ the basic question we need to answer is, “What are we getting and when are we getting it?”
At Devbridge we strongly believe in transparency. We’re so serious about it, we created the PowerUp app that gives our clients more detail and metrics on the development of their product than just about anyone else. But as exhaustive as our metrics may be, they aren’t the whole story.
Consider these two sets of metrics. Both were built using the same underlying data as Peter’s metrics from the story we told.
Figure 4
Figure 5
Is this project green or red? Are we on time and on budget? Are we delivering meaningful software? The answers lie not in the metrics themselves, but in the context in which the metrics are being generated, and with the product people who use these metrics to guide their decision-making processes.
We're not saying that metrics aren't important. We're not saying that you should manipulate your metrics just to make you look good. We're saying the opposite. Metrics are important, but they aren't meant to be taken out of context.
Look at Figure 5. It shows development is 30 percent over budget. Is that good or bad? What if the story here is that we’ve found a new opportunity to expand the product’s use into markets we originally thought were closed? A situation like that requires a bunch of additional research and for new features to be added over time. With no fixed scope, we’ll need more budget to achieve these new goals. Are we green, or are we red? The metrics are there as a starting point for a conversation about our objectives, and how we’re progressing towards meeting them. They aren’t the end of the story.
At Devbridge, embracing transparency will always be one of our most important values. We open our books and use the PowerUp app to show our clients everything. But more importantly, we teach our product managers to use context when putting these metrics together. If we're forecasting to go over budget, we share those metrics with clients and then have a conversation about why and what we think they should do about it, based on business objectives and story. It’s how we work to deliver the right products, the right way, at the right time.