Evidence indicates that agile software delivery results in a higher success rate for software projects as compared to waterfall delivery (view our infographic: 4 Advantages of Agile Over Waterfall Software Delivery). The product (I’m going to substitute product instead of project going forward—the agile way) is of higher quality, quality assurance costs go down, and more scope can be shipped to market in a shorter period of time. In fact, we’ve seen a tenfold decrease in costs for delivery of identical software products for tier 1 banks when an agile approach and toolset were used. Knowing all this, why do some companies still insist on the slower waterfall process?
Blame the industry. Highly regulated industries such as financial services, healthcare, and electric power all struggle with adoption due to the burden of regulatory compliance. Processes established years ago still govern, even with the software delivery landscape changing dramatically. What we’ve come to discover, however, is that these organizations can get eighty percent of the value from Agile while changing only a fraction of their processes and practices, without, in any way, compromising on regulatory requirements.
In this blog, and a more in-depth white paper, I outline five steps to help financial services institutions (i.e., banks, credit unions, and insurance companies) create stellar, value-added products quicker.
Step 1: Reduce project (product) size.
All large transformations start with small success stories. Select an initiative from the product portfolio and distill it to the smallest functional modules, allowing for the team to iterate on sub-components of the overall product. This gives the opportunity for the cross-functional team to familiarize themselves with the workflow, experience the rhythm of agile rituals, and build confidence in each other.
The team is encouraged to fail. Failing on a small piece of delivery, a sprint, endorses collaboration, learning from mistakes, as well as limits the financial impact of failure since the scope is not significant. Long term the team grows to understand that failure is a natural part of software delivery and that the real indication of success is shipping the right product for the users. In fact, embracing failure also aids organizations in adopting a more innovative mindset - not all ideas are meant to succeed, but you’re more likely to hit a jackpot if you allow your teams to experiment.
Step 2: Use Lean Requirements to jump-start delivery
The objective of Lean is to maximize customer value while minimizing waste. Lean requirements help ship working software over comprehensive documentation, which is one of the cornerstones of the agile manifesto. Start by conducting a lean workshop as a group. The primary objective is to arrive at a shared understanding of the deliverable as a team, while including stakeholders and customer needs in the exercise. During the workshop—that can be as short as four hours or as long as three days—we establish personas, perform a story mapping exercise, draw user flow diagrams, and go through basic prioritization exercises for the captured scope.
Lean requirements are accessible, create a shared understanding, and allow all members of the cross-functional team to contribute their ideas for the product (a team of ten is better than doing it solo). It’s also low-cost, takes only a couple of days, and provides just enough information for the team to start moving and shipping functional software. A side effect of involving senior executives in workshops is that alignment and buy-in happens through participation. You no longer have a swoop-and-poop project derailment half-way through the project because your sponsor has committed to the scope and schedule in the workshop.
Step 3: Upgrade your tools to coordinate communication
If your current process is wrought with a number of disconnected systems, administrative teams, and lacks transparency, take a look at Jira. Jira is a tool created by folks at Atlassian, and is the gold standard in the industry of software product delivery. It helps all members of the team stay connected and current. There are other tools in the industry, but none have the maturity and breadth of functionality offered by the Atlassian family of products (HipChat, Confluence, etc).
You can get started by licensing a cloud instance of Jira if you don’t want to get your infrastructure folks involved. That’s a great way to move lean. Once you’re more comfortable and need a more granular, configurable instance you can opt for their enterprise package and set up a dedicated application within your own data center.
Regardless of the tools you use, the objective is to promote individual communication and interaction—another cornerstone of the agile manifesto. Jira works because it gets out of its own way and allows teams to have meaningful conversations unburdened by busy work, documentation, and disconnected sources.
If your organization's Enterprise Delivery Framework team states that Excel documents are necessary to stay compliant, know that Jira can be customized to accept all of the custom fields that your EDF teams wants you to track.
Suffice to say, we've only skimmed the surface of what a shared toolset offers.
Step 4: Invest in Continuous Integration and Delivery
More often than not, we see very limited automation in how environments are set up, how source code is built, and how deployments are handled from environment to environment.
A sample scenario would look something like this: Engineers are working on local machines and checking code into a source repository (TFS, Git, etc.). Each developer tests their code, builds the solution locally to make sure it runs as expected, ad infinitum until the project is finished and ready for testing. SIT/UAT phase kicks off and all hell breaks loose: integrations don’t work, defects come crushing down in endless Excel documents exported from the enterprise quality assurance system, and timelines slip. Worst case, all projections of cost/timeline get thrown out the door because these problems were discovered once delivery was already finished. In other words, the end of the project when most of the budget has been spent.
Continuous Integration to the rescue. Let’s start with a short definition: Continuous Integration is a development practice and toolset that enables engineers to integrate their code into a shared repository on a daily basis; CI build servers run automated tests and build the product, allowing individual contributors to detect defects early and often (read: as soon as they check in source code).
Specific benefits include:
You avoid a lot of the conflicts when merging code because everyone is using a single repository, committing, and building every day (and often several times a day).
Deployment is automated. Single click. In tools, such as Team City, you go to the control panel, initiate deployment and watch as your production environment is updated to a copy of the application in your QA/UAT environment. Or isn’t if there are build errors, but likely there aren’t because you have been building every day, multiple times a day, across multiple machines, and running automated tests.
Your delivery becomes predictable. There are no last minute hot-fixes. No calls at night or on the weekends, unless you’re gathering up the team to have a beer and celebrate another successful product delivery.
Your infrastructure, support, and quality assurance costs go down because you get more done quicker, with less.
More detail can be found in our article on Continuous Integration, Continuous Deployment, and Continuous Delivery.
Step 5: Integrate your QA and delivery teams
More often than not, a project plan will have a User Acceptance Testing and System Integration Testing phases somewhere at the end of delivery. During UAT, testers run the product through specific scenarios that mimic production use. A user needs to check their account balance by tapping on “Account Balance,” a user makes a transfer and receives an SMS confirmation, and so on. SIT deals with the technical integrations and looks at the product as a whole—not just the application being launched to market, but also the dependencies, the integrations, authentication handlers, underlying API’s, etc. The problem with this approach is that a project manager planning this waterfall exercise in pursuit of better quality has absolutely no control over the outcome.
There are several tiers of integration you can bring to your delivery strategy. Pick one that is easiest to pull off within your organization.
Additionally, integrate UAT into your delivery sprints. Have the QA team integrate into the sprint based delivery schedule and have them execute tests on fully functional software. While this may seem like more work, it actually significantly reduces churn since issues are resolved quickly and are included within acceptance of the sprint. Using the same toolset (such as Jira) allows the team to quickly map defects to user stories and validate the acceptance criteria used, avoiding the disconnected practice of tracking defects in an enterprise quality assurance system that is in no way visible to the delivery team.
Use development and staging environments with access to all dependencies and introduce Unit Tests and inclusive code coverage. A unit test is a test that validates the smallest testable part of an application. It independently scrutinizes small functional blocks of the product for proper operation, reducing the quantity of manual testing long term. An automated testing strategy is another area that will introduce up-front costs into the project, however severely decreases manual quality assurance involvement throughout delivery. Unit tests are written to satisfy or fail based on the acceptance criteria within the user story in your backlog. Each time code is checked into a centralized repository, the build server (e.g. TeamCity) performs the build and executes the tests. If tests fail, the engineer is not allowed to check in the changes, automatically preventing defects.
Agile transformation: Next Steps
An agile transformation for a large, heavily regulated institution with an established enterprise delivery framework will take time and grit to execute. You’re turning an aircraft carrier around, after all.
