Go beyond velocity with advanced product metrics 

Configuring a comprehensive framework of metrics that drive results

Download white paper

Process metrics: Determine product & technical maturity

The last set of metrics measures the maturity of practices in a product organization. Fortune enterprises don’t transform into product-centric organizations overnight. Even new greenfield product ventures make compromises to ship to market faster. As an organization converts the portfolio, leverage product and engineering best practices, which include tracking quantitative data on a per-product basis and documenting progress over time. Understanding where an application and corresponding teams sit on the maturity scale helps product leaders identify elements needing attention and products carrying the highest risk.

Large enterprise programs and small greenfield builds benefit from a maturity score. Technical and product debt accumulates even when taking precautions.

The maturity score measures percentages across two domains:

  1. Product management and design

  2. Engineering

technical maturity chart Velocity

The specific weighting assigned to a condition is outside the scope of this paper. These attributes tend to be unique to a given organization and its products.

Product management & design maturity conditions

Product managers and designers are two key roles in product teams. Subsequently, evaluate the efficacy of various product management and design activities. Measure adoption and execution best practices (e.g., product canvas, user research, workshop participation, stakeholder commitment to rituals, and healthy team allocation). Track team allocation, activities during the workshop, kickoff of the project, and delivery rituals.

ALLOCATION OF RESOURCES

CONDITIONWHY IT MATTERS
Allocate a full-time product manager (or several)The team manages scope and feature decisions without external dependencies.
Allocate a full-time designer (or several)Make sure design activities are part of the sprint; design needs to be consulted in the requirement definition and the final UI QA.
Allocate enough engineering and testing membersUnderstaffed teams are bound to fail; ensure ample engineering resources to accomplish predefined goals.

WORKSHOP EFFECTIVENESS

CONDITIONWHY IT MATTERS
Administer at least one product workshop within the last six monthsWorkshops guarantee alignment and clear outcomes for the next six months of releases. Going longer than six months will lead to churn.
Ensure the cross-functional team members attend the workshopKnowledge transfer is highly effective for cross-functional team members. Product managers, engineers, and designers all need to share information with another when developing products.

KICKOFF EFFECTIVENESS

CONDITIONWHY IT MATTERS
Share the Definition of Ready and Definition of Done with teamThe DoR and DoD are critical for effective cooperation inside a cross-functional team. Lock in expectations between a product owner, the engineers, and designers.
Make sure the backlog is fully estimatedStory point estimates help the team establish initial milestones and assign risk to stories with high complexity.
Get access to end user design researchSoftware should never be built in a vacuum. User research, user testing, and demos need to have a real audience.

DELIVERY EFFECTIVENESS

CONDITIONWHY IT MATTERS
Groom two sprints of the backlog in advanceGrooming a couple of sprints ahead reduces noise for the team, provides the ability to quickly swap in groomed stories when another piece of work is blocked or finished early. Avoid idling of engineering team members at all cost.
Follow sprint rituals (e.g., stand ups and retros)Observing rituals ensures ongoing team communication and supports continuous improvement through collaboration.
Require stakeholders involved in decision-making to attend sprint demosDemos give stakeholders an opportunity to accept work as well as make the software real. The build-measure-learn loop doesn’t work without them.
Provide normal sprint reports (e.g., spend and burndown)Use sprint reports, many of which are covered in this paper, actively as a tool.
Document change requests in Jira (or the internal tool of your choice) to communicate the impact of adjustments to the businessWhile change requests are not part of the agile process, tracking pivots in the product strategy helps product leaders explain decisions made to sponsors, some of which work outside of the product organization.

Engineering maturity conditions

Like product managers and designers, engineers play a key role in building products. There are standard engineering competencies and best practices that each software project should observe to some degree to inform behaviors—good or bad—for the future of the product and organization. Measure the adoption of technical best practices like CI/CD, code quality, security, performance, monitoring, and technical debt). Track DevOps, testing, technical maturity, coding standards, security, accessibility, and monitoring.

enginering maturity conditions Velocity

DEVOPS

CONDITIONWHY IT MATTERS
Feature continuous integration (CI)CI allows automated integration of multiple workstreams into a single main branch, multiple times a day. It reduces overhead and risk, increases confidence, and improves communication.
Automate deploymentsBuilding and deploying code in the dev, staging, UAT, and production environments happens with a push of a button, eliminating room for user error.
Use containerizationContainers prime the application for scale with each having lateral scalability without unnecessary overhead.
Include continuous deployment (CD)While challenging to attain, CD pushes code from development into production multiple times a day (assuming all tests run successfully) and represents the target state for mature product companies.

TESTING

CONDITIONWHY IT MATTERS
Run unit/integration tests and coverage metricsAn advanced testing strategy automates the mundane and drastically reduces the long-term costs of quality assurance. Furthermore, less technical debt is accrued and software is easier to maintain.
Administer performance testingPerformance goals and tests help teams avoid production challenges by anticipating future needs.
Add end-to-end testingEnd-to-end tests guarantee that the software and integrations work correctly.

TECH MATURITY

CONDITIONWHY IT MATTERS
Track technical debtTracking technical debt mitigates the risk of blindsiding a product team with productivity dropping over time as the debt mounts.
Evaluate and review the architectureAs an application matures, the underlying architecture needs to be evaluated and areas for refactoring need to be identified to avoid reaching a legacy state.

CODING STANDARDS

CONDITIONWHY IT MATTERS
Provide code style check automationWhen the code style is checked automatically and conforms to the rules defined by a project team, maintainability improves in the long run.
Use static code analysisTools automatically scan code for vulnerabilities and security best practices, helping automate and avoid security issues.
Offer feature branchesUse pull requests and feature branching as part of the engineering strategy. The code from the master branch should always be ready to be released to production.
Make sure an artifact repository is usedAdd an artifactory repository (e.g., Sonatype Nexus) to store intermediate and final results of the software engineering process (e.g., npm packages, OSS project forks, docker images, release packages), which simplifies management and propagation of reusable components.

SECURITY

CONDITIONWHY IT MATTERS
Complete a vulnerability and security assessmentEstablish and observe security standards (e.g., OWASP 10) for the application.
Provide a secret management optionProvide a central secret management platform such as HashiCorp Vault to avoid secrets being stored openly in code.

ACCESSIBILITY

CONDITIONWHY IT MATTERS
Follow compliance standardsEstablish and observe a level of application accessibility.

MONITORING

CONDITIONWHY IT MATTERS
Monitor logging and reportingMonitoring solutions provide insights and alerts for irregular patterns in production environment.

Continue to:Build powerful products backed by metrics