Data-informed product delivery
Analytics, metrics, big data, machine learning… We are in an era of more available information than ever in digital product delivery.
How do you find the right data, track it back to a meaningful outcome, and use that data to inform priority for product development and investment?
We’ll look at effective ways software products have leveraged data, the best time to incorporate it in your process, and how to focus on meaningful statistics to make the best decisions informed by your experience and institutional expertise.
Watch the talk:
My name's Tim Kearney. I'm the Director of Product Management here at Devbridge. And this morning, we're going to talk about story map refactoring and what does that even mean. I imagine some of you are familiar with the concept of story mapping already. Maybe some of you are not. It's fairly embedded in product orgs at this point. I think this book, User Story Mapping, show your hands who has this book. Anybody? I tried to find one in the office, I couldn't find the prop but this is what it looks like. I have a Kindle version so I can't hold up my Kindle version. This guy, his name is Jeff Patton, and he sort of created this method that probably most of us are using these days, around how do we figure out what we're going to build in a way that is all encompassing to all the stakeholders involved.
Here's the quote he has at the preface of this book. "Story mapping keeps us focused on users and their experience, and the result is a better conversation, and ultimately a better product." I think most people agree with that because people are doing this, including us at Devbridge. We make a point to take a lot of photos of this process when we do it with our clients, because it's really important for how we go about ultimately winning business. It's part of how we scope a project, but it's also part of how we demonstrate expertise to clients who are interested in working with us, as we try to very quickly understand what their business problem is, who their end users are, who the people in the room are and what their intentions and potentially biases are, and work through problems together sometimes in this office, sometimes in client offices, to ultimately end up with something that looks like this.
What do we want to get out of that process? The largest obstacle to coming up with something that we want to build is getting people aligned. Especially in this consulting environment, we have potential customers come in to talk to us about problems and frequently have stakeholders in the room that are not normally even in the room together. In a large organization, they know they have problems to fix and they pull a sneeze from different parts of the organization and throw them in a room and say, "Figure out how we're going to fix our problems," and everyone shows up in the room with their own idea of what they think the solution looks like. They have a vision in their heads but they have no idea kind of going into the process, how aligned they are or not.
This process forces those stakeholders to put their ideas up on the board, and then we can see that, and they can see each other's thoughts and understand where they agree and where they disagree through dialogue and asking the right questions. The point is to get everyone kind of thinking the same sort of thing so that they walk out of the room. And when we ultimately propose work, whether it's a new work or the next phase of work, we all have a shared sense of what we're going to be working on together. We start with something like this. We call this our box of fun. Everything you need to run a story mapping workshop. And what does it actually look like for those who haven't gone through this? We ask stakeholders to write down the tasks that they think their end users ultimately have to do to capture the full set of requirements, and they can be kind of different levels of granularity.
There's three examples of probably what we all did this morning I assume, certainly myself. At a high level, I could say I got cleaned up. Maybe a level below that, I definitely took a shower. And as part of taking a shower, there're specific acts within there. In working with people who maybe aren't even used to this process, you're going to end up with a wall of ideas that are going to be all over the place in terms of something very granular, something very big. And we need to make sense of it. A more sort of technical example might be something like this.
We use search engines in every product pretty much these days. Manage search results is something that we want users to be able to do, but that's very high level. Okay, we want them to filter search results. Okay, now we want them to filter search results by category. Again, as we scope work for a project, we might get all different levels of granularity or fidelity on this stuff. We get it all on the board and we try to make some sense of it. We organize these stories into groups. Affinity mapping. Get a sense of these kinds of things, and these piles are kind of the same. And then these piles are all kind of the same to ultimately drive towards something that looks like this. This is what a story map looks like with high-level sort of themes of work on top, epics and stories below it.
Again, coming out of a workshop, a lean requirements workshop that we run, these yellow sticky notes are going to be kind of all over the place in terms of how clear they are, how big they are, how small they are, and we need to work on them. What we can do in the room is get further alignment with the stakeholders on relative priority of these things. We organize them to some degree. What do we need to think about first? What do we need to think about second? What should we not even think about?
What is this process good at? We just talked about alignment. Stakeholders come in, they're not sure if they're aligned, we try to force some alignment on them. As we walk out of the room and agree what it is, we're going to go ahead and pitch them, or start working on the next day. It gives us a broad picture. We're not writing full user stories. We're not writing acceptance criteria here. It's just a wide kind of lens about everything we agree we need to accomplish at some point throughout this project. We just talked about prioritization. What's most important, what's least important, at least from the business standpoint? And then, this sense of shared ownership. So especially in a consulting model, we're doing the work, but we're all in it together. And so it doesn't matter... If we deliver functional software that ultimately is not what they wanted, we failed. If we deliver software that is what they wanted but doesn't work right, we also fail.
This process in talking through all these things from a functional requirements perspective but also non-functional dependencies, things like that, in the room helps us walk out of the room with shared ownership. But what is this really not capture in those conversations? Dependencies is one. We get a sense of prioritization from the business from the business stakeholders themselves, but what's hard to tell which things need to happen first, especially at a technical level. We don't see the guts and the wiring of a prospective product. We just know we need these features first and these features later. We don't really see relationships. There're no lines between these things. It's a matrix of rows, of a journey across the product, and columns about specific features to build over time, but the relationship between these things is kind of tenuous.
Again, it's very high-level. It's not like you take this and you just start working tomorrow. You have to then dive into the weeds and remember a lot of the context that was talked about in the room when those stories went up on the board. And it definitely doesn't really just how we're going to do this. It drives at what and sort of when, but how is left for later. In our world, if we are proposing work to a client, we can't punked on how. How is going to be a big factor in how long something's going to take, understanding what some of the risks to the project are. Maybe there's an integration that we need to make with a client's ecosystem that isn't ready yet. We can't just promise our side will be ready to go if we're waiting for them. That doesn't get captured in this process very well.
This is another sort of sub-chapter in User Story Mapping. "Software isn't the point." And that's true through a specific lens of building a product that meets its user's needs. But for us, this is not quite true. We don't own the products at the end of the day. If we're successful, we will build something that our clients love and we're going to hand it over to them and they have to own it. They're not just buying some glossy end user product. They're buying software they have to maintain forever, unless they want to work with us forever. That's typically not the way that these contracts are going to run. We have to give them something that makes sense under the hood.
Everyone walks out of a successful workshop thinking about this. We figured it out, we talked through our problems, we're building a Ferrari. I can't wait to get back to the office and tell everyone that we're going to do this. And that's great and everyone should have this up on their vision board, but the engineers who are going to ultimately have to build this thing kind of need to be thinking more like this. We need to get the client thinking along these lines too because, like I said, they're going to own this. If it were just our kind of secret projects, we might make something that looks like this but it's really styrofoam and then we go and try to drive it around and get in an accident and it's a mess. Validating this stuff is important too. That doesn't really happen in the story mapping process because you're so focused on paint colors and where the leather is sourced from in your Ferrari.
What else can we do to try to align what it is we're going to build under the hood? We move to this concept of a data model. Many of you are probably familiar with it. What we're really talking about here are relationships between things. If you're totally unfamiliar with data modeling, there are three real types of relationships in one of these sort of data model systems that you would draw. A one-to-one relationship, hopefully this is true of everybody here. And that each person has one head. If I find a random one of you and I check to see if you have a head, you do have one head. If I find a head in here, I hope I find a person too. We're not sharing either of these things.
Each person has many eyelashes. This is a one-to-many relationship. The inverse is not true. Everyone's going to go home, we're going to clean the office. Maybe we find an eyelash on a table, on the floor, that belonged to somebody. It did not belong to two of you, I hope. And then, a many-to-many relationship. A person, and ethnicities. I'm 75% Irish and 25% Italian, although there's probably other percentages that I'm not aware of, small slices. We're all probably mutts on some level. But I have multiple ethnicities. And if I choose an ethnicity, Irish, it's going to affect many people in this room. That's an example of a many-to-many relationship.
What's an example of using this in a product? This is an example story map we use all the time internally for onboarding purposes. You don't have to get into the weeds here. It's an E-commerce platform for buying industrial parts. We see themes or epics and features and stories just as we expect. And you could walk out of the room with this as a scope with a client and feel pretty good about what it is that you need to build. But there are things here that we could think harder on. Viewing a list of products and viewing a list of apparel and viewing a digital library, these are all things that you can buy off this E-commerce site. Maybe this is really the same story represented three different ways in three different epics. Similarly, searching for stuff, searching and browsing and filtering. The different words, they sound like different verbs, but when it comes down to building things behind the scenes, we're talking about modifications to a sequel statement. We're not really talking about vastly different things. This is kind of one function potentially in the system we build.
Add to wishlist. Somebody in the room throws that up on the board. It makes perfect sense, yeah, we should do that. There's no wishlist feature or epic anywhere in here. It's just thrown up on the board. When it comes to working on that, there's a lot more stuff we need to figure out. What does that mean? What is a wishlist? It's kind of a throwaway story in here. To help kind of flush that out, we try to think about it in this data model concept. This is the same hypothetical E-commerce site represented a different way, with these relationships. You could potentially draw this where all those parts and pieces of apparel and user manuals, they're all just generic things in our system that we want to be able to search on. That we want people to be able to add to a wishlist. We want people to be able to find items and put them into some sort of cart. And that cart ultimately hopefully gets turned into an order and just doesn't sit stagnant in that cart forever.
It makes sense to try to look at these things together. In our lean requirements workshops, while product managers and product designers are very focused on the story mapping process and the user journey process, engineers should be thinking about these kinds of things. What are the relationships that I hear you talking about? What are we going to have to build behind the scenes? And then when all is said and done and everyone leaves feeling good, work together to figure out is there anything in my drawing that's not represented in yours, and is there anything in yours that's not represented in mine? And are there questions that arise in that discussion about maybe there was a lack of understanding about a very specific topic? It gives an opportunity for us to validate some of the stuff that goes on under the hood with a client in a visual way and make sure that we ask more questions if necessary before we commit to doing things.
This is that same original story map that we were looking at before, but reconfigured in a way that aligns more with what that data model looked like. If we put them sort of side by side, scope wise, largely the same kind of concepts, it's just a different way of organizing it. Why? Why do that? Well, we talked about some of this stuff already. There's the potential for a consolidation of some of the efforts. If we had those three sort of list stories that we looked at, those could all end up in a backlog in different epics and get story pointed and get worked on and end up as three different pieces of code in the backend. And they all work. It's not that it's going to fail, but in doing something like that, there's additional effort for our clients. There're additional costs. And with that goes additional technical debt.
If we have three different gallery kind of concepts going on and the client says, "Well, we want to add a new feature or some new functionality to all of them," we're doing three times as much work as we were before. And then we say, "Oh, you know what we should've done? We should've consolidated this stuff in the beginning and now we're maintaining it all over the place." That's debt that ends up in your backlog, now you write stories for consolidating this stuff later when maybe a little bit of effort up front, thinking of things in a different way could have saved us the trouble.
Another quote from the User Story Mapping bible. "The best estimates come from developers who really understand what they are estimating." That's kind of like, "Sure." But that's a real reason to do this, because it does force engineers to think through some of the underlying things in the room while the client is there, and work with product managers to figure out if everyone's on the same page, think through some of the dependencies in the system that aren't captured in the story map, think through relationships. Sometimes, the difference between thinking something is a one-to-many relationship and it turns out to be a many-to-many relationship is a huge wrench thrown into there, into the process. You have to rethink a lot of different stuff. Maybe you had database tables that you thought were ready to go, and now you have to kind of re-imagine your whole setup.
As far as why, validating what we heard with the client in a different way and better estimation in considering the project or the product in two different lenses. The third potential benefit here is if this work is big... And this is a very small looking story map. If we're talking about a digital transformation and rebuilding an entire enterprise platform, there's a lot more than this, both wide and deep. That work might need to be broken up into teams. Maybe some of the stuff that we're talking about ends up as separate microservices on the backend. And we have a specific microservice for everything about users and a specific microservice for everything around the cart and ordering parts. Having the work organized in a way that it can live in its proper service at the end and have separate engineering teams think about those things is a way to create less friction in the system.
If we had, say, three different teams working in our code base and we didn't organize the work the right way, and we had sort of stories spread across the system in ways that they weren't built in the backend, you could have three different teams all having to write, work on stories in the sprint in the same code base. And that happens all the time. It's not that that's the end of the world, but there's additional sort of work and friction in the system in making sure code is reviewed and pull requests are done on time, branching potentially gets more complicated. Whereas if you had organized your backlog and planned your work with a team in a way that was more aligned with what the code base actually looks like, then maybe you have some savings down the road with respect to this stuff. Organization of a backlog is a potential win here.
Last, what happens if there's some conversation in a room around saving a search? In our story map, we had the concept of people searching for parts. There was nothing in there about wanting to save a search. As we start thinking about what do we do next, what's the stuff that we're not talking about in this version of work but we know it's probably coming, we can start to think about those things in this diagram and how does that change relationships that we already kind of assume we need to build. If we're going to build some sort of function around saving searches, that becomes sort of a new object in this model. We could then think about the relationship between searches and users.
What is saving a search mean? Is it just every time somebody types in keywords and hits save, we just save the keywords and associate with that user? Is that a one-to-many relationship where any individual user can just have many searches but searches only belong to one person? Or, you can brainstorm around what happens to the system. If we say that's actually a many-to-many relationship so any given person can have 10 searches saved, but if we save searches as independent things, maybe that same search can belong to a thousand users? And what insights can we learn from that? There are two potential wins there. There's data storage, potentially, if we're talking about millions and millions of searches or billions of searches. And 60% of those rows are duplicative because we're just saving everything with a user every time, that becomes a performance problem.
Similarly, if 60% of the searches are the same, we can more quickly develop some insights and some dashboards around, "Hey, are you aware, e-commerce company, that 60% of the people are searching for the same thing? Here's a graph of everything going on in your system with the saved search, and you see that everyone's looking for this very specific item. Maybe that needs to be highlighted in a way that you're not thinking about. Maybe you're not even aware of that." You can catch that stuff in analytics and drop stuff to some third party package and try to find these things out. But if you do capture something that way, you're going to have to go back and account for it in the backend anyway. There's not a lot of costs in doing this upfront and there's the potential for a big upside.
Five why's. Excuse me. Consolidation of effort in what engineers are going to have to build. Validation with stakeholders on what's going to be under the hood. What does that chassis look like for their Ferrari? Not just shiny paint. Better chance to estimate the work better for engineering. Optimization of the backlog. If this thing grows, it needs to be split into 10 microservices with 10 engineering teams working on it. And hopefully, less debt in the backlog because of forethought and more opportunity for those bells and whistles and wins with a client later when you think about how you can add new features. Thank you for your time. Thanks for stopping by. We hope to see you again.
About the event:
Full Stack Friday is a monthly, meet up hosted by Devbridge at our Chicago headquarters. We get together to indulge in a delicious pancake breakfast and talk shop. We bring in guest speakers to present talks focused on issues relevant to product people and full stack teams. For more information about the next event, contact us directly.