How does your DevOps measure up?

With the sheer amount of software being created today, software development techniques are rapidly and continuously evolving. Even industries that previously had a very fixed cycle, such as car manufacturers, are moving towards continuous delivery. As more companies adopt an Agile development process to support continuous delivery, testing (or quality assurance) and delivery need to keep pace.

Hence, the rise of DevOps, which manages the entire product lifecycle. But, there are varying levels of DevOps. In this post, we define those levels and share with you our approach. How does your enterprise measure up?

DevOps level chartt

Level 0

At Level 0, there’s no automation whatsoever. Developers in this category build applications manually, then package and store them in a shared location. Deployment is a manual process through production machines. In extreme cases, developers send deployments directly into production machines, sometimes simply overwriting files in a production environment. Rollback, in this case, is manual and often not even possible. Rolling deployments are difficult—nearly impossible. Disaster recovery is also a manual and lengthy process. Self-healing is impossible, as infrastructure is handled manually.

Typical company profile

Typically, non-software development companies would live here. Projects are small and usually delivered by lone contractors. Level 0 is suitable when there is intermittent deployment with minimal need for updates. For all other enterprises, Level 0 is not a great place to be.


  • Minimal infrastructure and automation requirements


  • Difficult to trace what version is currently deployed

  • Although it is possible to have processes around manual builds and deployments to have trackability, it’s extremely error-prone

  • Procedures for deployment are usually outdated and often exist only in the heads of application support or developers

  • As the number of applications to maintain and deploy grow, this method becomes increasingly difficult to manage

  • Teams spend most of their time doing manual work. As a result, applications are not properly maintained and releases are infrequent

  • Scaling is also difficult. More resources are required with each new machine, and each scaling increases the chance of errors

Level 1

Level 1 improves on Level 0. Building and packaging is done automatically by the build server. The artifact produced is easily traceable from where it originated. The build runs unit tests to verify consistency. Artifacts are placed into a shared repository for application support to pick up. Deployments are still carried out manually by application support, following repeatable procedures. However, as with Level 0, rollback is manual and often not possible. Rolling deployments are difficult—nearly impossible. Disaster recovery is also a manual and lengthy process. Self-healing also is impossible, as infrastructure is handled manually.

Typical company profile

Most modern software development is in this category. Level 1 allows developers to quickly build packages and pass them on for application support to deploy. This is suitable for companies that have a low number of products with minimal changes.


  • Application builds are repeatable and traceable to their sources

  • Direct intervention isn’t required. Developers just drop the correct build into a shared location

  • Consistency checks with unit testing (the build isn’t successful and doesn’t produce artifacts if unit tests are failing)

  • Allows quick repeatable builds


  • Increased infrastructure requirements (requires CI environment)

  • Requires automation knowledge and automation of builds

  • Still requires deployment procedures, which may be outdated

  • Even though builds are repeatable and quick, deployments could still be long and not prone to errors

  • Scaling complexity is same as Level 0

Level 2

Here, the building, packaging and deploying for all environments is automated. Following deployment, integration tests can verify the build. Code changes automatically trigger builds, unit tests and deployments to the integration environment with the ability to promote builds to successive environments. Infrastructure is not yet automated, however. Machines are prepared manually before builds are set.

In Level 2, rollback is possible by deploying the old version of a build. Rolling updates are difficult to set up and are application-specific. Disaster recovery is manual. It’s also time consuming, as infrastructure needs to be manually rebuilt to a specific state. Self-healing at this stage also is impossible, as infrastructure is handled manually.

Typical company profile

This is where most companies with multiple products and teams are trying to get. Level 2 allows teams to quickly deploy new versions of a product for clients to use. Deployment automation reduces the potential for error. With manual tasks automated, the workload of application support is reduced—they’re no longer a bottleneck. This category is ideal for companies that have multiple teams working on products with frequent, rolling changes.


  • Builds and deployments are fully automated. They are repeatable and traceable to the source

  • No manual intervention from application support or developers is needed, resulting in reduced error rates

  • Application consistency can be verified by automatically running integration tests with each new build

  • Quick and repeatable deployments are possible


  • Even more infrastructure requirements

  • All tools must support automation, requiring automation specialists to be highly skilled

  • Build and deployment maintenance is required

Level 3

Level 3 features full infrastructure automation, builds and deployments. When a deployment is complete, the new required infrastructure is created and a new build is deployed. Tests verify the consistency of a deployment, allowing for green/blue deployments. The rollback is simply switching back to an old infrastructure or rebuilding it from scratch in an automated way. Disaster recovery easily becomes a new deployment that rebuilds infrastructure and redeploys the application. At this level, self-healing is possible as infrastructure is scripted and can be rolled out automatically as soon as any erratic behavior is detected.

Typical company profile

This is where SaaS companies should be. The full automation at Level 3 allows for uninterrupted service and continuous feature improvements.


  • Not only are builds and deployments automated, infrastructure setup is as well

  • No human intervention is required, resulting in fewer mistakes

  • Together with proper software architecture, Level 3 opens capabilities for fast releases—plus the removal of transitional environments, such as quality assurance, user acceptance testing—after all


  • Setup is complex

  • Highly skilled engineers are required on development, DevOps and app support level

  • There’s no solution to buy, so the "bed-in" time is substantial until full automation is reached (all errors are polished away)

  • Maintenance becomes an ongoing, complex task

DevOps: A way to get started

If your organization is looking to take its DevOps to the next level, our approach combines discipline training with collaborative execution. We look at DevOps as a continuous delivery process of getting product to market sooner, at a lower cost, higher quality, etc. We’ve learned through experience that theory alone is not enough. Our approach, at a high level, is:

  • Select a pilot program that can leverage DevOps (something in the range of $1-5M, so that results can be demonstrated quickly)

  • Build a beachhead team from two organizations. Essentially a cross-functional, blended team with people from both sides that can work together. Internal teams pick up skills from an external resource while operating within context of a real project versus just theory

  • Use delivered product as a foundation of DevOps as well as the cultural shift needed to be successful. Advertise engagement across the organization as a metric of success. This helps spread the change management process

This approach works well when there’s support from an SVP and training is facilitated in parallel both top down and bottom up.


The needed level of DevOps prowess differs from organization to organization. With an increasing demand for software that evolves with its users, embracing DevOps can give enterprises a leg up on the competition.