Process metrics: Determine product & technical maturity
The last set of metrics measures the maturity of practices in a product organization. Fortune enterprises don’t transform into product-centric organizations overnight. Even new greenfield product ventures make compromises to ship to market faster. As an organization converts the portfolio, leverage product and engineering best practices, which include tracking quantitative data on a per-product basis and documenting progress over time. Understanding where an application and corresponding teams sit on the maturity scale helps product leaders identify elements needing attention and products carrying the highest risk.
Large enterprise programs and small greenfield builds benefit from a maturity score. Technical and product debt accumulates even when taking precautions.
The maturity score measures percentages across two domains:
Product management and design
Engineering
The specific weighting assigned to a condition is outside the scope of this paper. These attributes tend to be unique to a given organization and its products.
Product management & design maturity conditions
Product managers and designers are two key roles in product teams. Subsequently, evaluate the efficacy of various product management and design activities. Measure adoption and execution best practices (e.g., product canvas, user research, workshop participation, stakeholder commitment to rituals, and healthy team allocation). Track team allocation, activities during the workshop, kickoff of the project, and delivery rituals.
ALLOCATION OF RESOURCES
CONDITION
WHY IT MATTERS
Allocate a full-time product manager (or several)
The team manages scope and feature decisions without external dependencies.
Allocate a full-time designer (or several)
Make sure design activities are part of the sprint; design needs to be consulted in the requirement definition and the final UI QA.
Allocate enough engineering and testing members
Understaffed teams are bound to fail; ensure ample engineering resources to accomplish predefined goals.
WORKSHOP EFFECTIVENESS
CONDITION
WHY IT MATTERS
Administer at least one product workshop within the last six months
Workshops guarantee alignment and clear outcomes for the next six months of releases. Going longer than six months will lead to churn.
Ensure the cross-functional team members attend the workshop
Knowledge transfer is highly effective for cross-functional team members. Product managers, engineers, and designers all need to share information with another when developing products.
KICKOFF EFFECTIVENESS
CONDITION
WHY IT MATTERS
Share the Definition of Ready and Definition of Done with team
The DoR and DoD are critical for effective cooperation inside a cross-functional team. Lock in expectations between a product owner, the engineers, and designers.
Make sure the backlog is fully estimated
Story point estimates help the team establish initial milestones and assign risk to stories with high complexity.
Get access to end user design research
Software should never be built in a vacuum. User research, user testing, and demos need to have a real audience.
DELIVERY EFFECTIVENESS
CONDITION
WHY IT MATTERS
Groom two sprints of the backlog in advance
Grooming a couple of sprints ahead reduces noise for the team, provides the ability to quickly swap in groomed stories when another piece of work is blocked or finished early. Avoid idling of engineering team members at all cost.
Follow sprint rituals (e.g., stand ups and retros)
Observing rituals ensures ongoing team communication and supports continuous improvement through collaboration.
Require stakeholders involved in decision-making to attend sprint demos
Demos give stakeholders an opportunity to accept work as well as make the software real. The build-measure-learn loop doesn’t work without them.
Provide normal sprint reports (e.g., spend and burndown)
Use sprint reports, many of which are covered in this paper, actively as a tool.
Document change requests in Jira (or the internal tool of your choice) to communicate the impact of adjustments to the business
While change requests are not part of the agile process, tracking pivots in the product strategy helps product leaders explain decisions made to sponsors, some of which work outside of the product organization.
Engineering maturity conditions
Like product managers and designers, engineers play a key role in building products. There are standard engineering competencies and best practices that each software project should observe to some degree to inform behaviors—good or bad—for the future of the product and organization. Measure the adoption of technical best practices like CI/CD, code quality, security, performance, monitoring, and technical debt). Track DevOps, testing, technical maturity, coding standards, security, accessibility, and monitoring.
DEVOPS
CONDITION
WHY IT MATTERS
Feature continuous integration (CI)
CI allows automated integration of multiple workstreams into a single main branch, multiple times a day. It reduces overhead and risk, increases confidence, and improves communication.
Automate deployments
Building and deploying code in the dev, staging, UAT, and production environments happens with a push of a button, eliminating room for user error.
Use containerization
Containers prime the application for scale with each having lateral scalability without unnecessary overhead.
Include continuous deployment (CD)
While challenging to attain, CD pushes code from development into production multiple times a day (assuming all tests run successfully) and represents the target state for mature product companies.
TESTING
CONDITION
WHY IT MATTERS
Run unit/integration tests and coverage metrics
An advanced testing strategy automates the mundane and drastically reduces the long-term costs of quality assurance. Furthermore, less technical debt is accrued and software is easier to maintain.
Administer performance testing
Performance goals and tests help teams avoid production challenges by anticipating future needs.
Add end-to-end testing
End-to-end tests guarantee that the software and integrations work correctly.
TECH MATURITY
CONDITION
WHY IT MATTERS
Track technical debt
Tracking technical debt mitigates the risk of blindsiding a product team with productivity dropping over time as the debt mounts.
Evaluate and review the architecture
As an application matures, the underlying architecture needs to be evaluated and areas for refactoring need to be identified to avoid reaching a legacy state.
CODING STANDARDS
CONDITION
WHY IT MATTERS
Provide code style check automation
When the code style is checked automatically and conforms to the rules defined by a project team, maintainability improves in the long run.
Use static code analysis
Tools automatically scan code for vulnerabilities and security best practices, helping automate and avoid security issues.
Offer feature branches
Use pull requests and feature branching as part of the engineering strategy. The code from the master branch should always be ready to be released to production.
Make sure an artifact repository is used
Add an artifactory repository (e.g., Sonatype Nexus) to store intermediate and final results of the software engineering process (e.g., npm packages, OSS project forks, docker images, release packages), which simplifies management and propagation of reusable components.
SECURITY
CONDITION
WHY IT MATTERS
Complete a vulnerability and security assessment
Establish and observe security standards (e.g., OWASP 10) for the application.
Provide a secret management option
Provide a central secret management platform such as HashiCorp Vault to avoid secrets being stored openly in code.
ACCESSIBILITY
CONDITION
WHY IT MATTERS
Follow compliance standards
Establish and observe a level of application accessibility.
MONITORING
CONDITION
WHY IT MATTERS
Monitor logging and reporting
Monitoring solutions provide insights and alerts for irregular patterns in production environment.
