Set measurable best practices

An important step is to establish new protocols and instill new habits in the development team to culturally support the move to continuous deployment. The contents of this section outlines recommended best practices. Take sufficient time to map out and explain practices for the team to follow. Embracing clear, shared objectives helps ensure a smooth and safe transition from continuous delivery to deployment.

Capture real-time metadata

All teams need a data contingency plan. When running database or schema updates, teams cannot use production downtime to execute scripts. Instead, administer all data management while the application is in use.

Enable and disable toggle features

Feature toggles isolate prototype features for pre-acceptance by some or all users. These toggles can be disabled if they become problematic. When development reaches a sufficient level of maturity, the toggle is enabled so that the next release includes the feature and is accessible to at least one user. If a problem arises concerning that feature, it should be easy to avoid a production rollback by disabling the feature with an administrative function. The development team can anticipate some of the problems and preemptively offer remedies that allow production to retain the latest release.

Prioritize site reliability engineering

With the advent of DevOps, a new role emerged for site reliability engineers creating scalable, efficient, and highly reliable software systems. These specialists focus on smooth production performance, disaster recovery, availability, latency, change management, capacity planning, and monitoring. All of these are necessary when pursuing continuous deployment. For larger product groups that aspire to full release automation, it becomes key to dedicate a skillful DevOps team member to tackle it all.

Verify functionality before and after release

Decouple the staging delivery from the subsequent deployment into production while automating each. If there are no pre-production checkpoints, regardless of automation, delivery and deployment occur nearly simultaneously. The tendency to make quick go/no-go decisions poses risks and leaves minimal contingencies. Decoupling is the best practice, together with the insertion of checkpoint alerts and configuring for full automation all along the pipeline.

If verification reveals critical defects, automated deployment continues while automatic alerts are sent to the gatekeepers who make an immediate rollback/proceed decision. Enabling alerts is vital to avoid corrupting the production environment or disrupting operations. A deployment-complete alert goes out, at which point all functional changes are automatically verified in production.

Use the following practices to drive automatic verification:

• Test automation throughout the pipeline. Ensure the ability to test repeatedly via automation.

• Test data management. Maintain complete sets of test data in a version control database to ensure consistency and relevancy with an expanding feature set.

• Include an automated suite of non-functional tests. Test system attributes, such as security, reliability, performance, maintainability, scalability, and usability, thoroughly before release.

• Provide production verification testing. Automatically test all changes in production, preferably while changes remain hidden during verification.

Monitor the production environment

Verifying that deployed features don't break on the way into production is a must-have pre-release quality check. However, teams also need to measure a feature's performance and value over the lifespan. The insights behind the feedback loop come primarily from robust monitoring capabilities, which must be in place before release. In the pursuit of automatic continuous deployment, go beyond the scrutiny of constituents of the release vehicle, and continuously monitor the production/release environment.

Any investment put into monitoring the environment must be effective and automatic, which requires full-stack instrumentation. By doing so, the deployment team gains full confidence in real-time measures of system performance, end user actions, incidents, failures, and business value. By comparing actual measurements with predefined metrics, environment problems separate from application problems.

Continuous deployment means speed through automation. Quick problem identification and resolution become possible only with instrumentation in place. Configure real-time monitoring in the environment to achieve such speed.

While some business-value metrics are not collectible until release, teams need to know how to measure against predefined metrics once the decision to release occurs. Three practices to support monitoring metrics include:

1. Full-stack instrumentation: Monitor for metric deviations across the entire production environment technology stack.

2. Visual displays: Include admin dashboards with real-time, automatic measurement updates.

3. Federated monitoring: Provide a top-level dashboard that unifies critical measures across technologies and applications, offering a holistic view of problems and relative performance levels.