Continuous Integration with TeamCity Server
“Automations” - a term we the developers are quite familiar with. But still, sometimes we have to rely on manual deployments, builds, migrations and other repetitive development tasks. The problem is that the requirements for those tasks change and configuring our automations to match those requirements becomes a harder task than just running a script or tool with specific parameters once or twice.
We know that writing clean and modular code allows for a better reusability; we want our CI tasks to be reusable in other projects as well. Having a background of Jenkins, CruiseControl and GO CD usage, we decided to stick with the JetBrains TeamCity server, that in my opinion is one of the best CI servers. It comes with a bundle of useful features out of the box.
I will introduce some of our favorite TeamCity (TC) features and how we use them in our everyday work.
User Access Management
We use the Active Directory domain controller to manage our employee accounts, the groups they belong to and permissions they have. We needed a way for all of our developers to have access to our TC server by using their Active Directory credentials. Luckily, TC supports LDAP integration, so TC server access was easy for us. TC synchronized all the required accounts and even groups.
Next, we added some additional groups and possibilities to differentiate between our own "internal users" and "outside users". The outside users are our clients, contributing developers from other companies and other third-parties that might need access to specific projects.
In TC you can easily add permission for users and groups per project, which is useful when we have users that need specific permissions for running some of our deployment and build tasks.
Project Structure and Permissions
At Devbridge, we offer different platform and technologies for our projects, such as PHP, .NET, iOS, Android, node.js, etcetera. So, it’s best to separate projects that are on PHP from .NET.
TeamCity allows the creation of hierarchical project trees that allow you to group projects by platform, environment, name, etcetera. So first we create the grouping by platform (PHP, .NET, iOS and so on), and inside the platform group we create additional “environment” groups (Development, Staging, Production). Only then do we add the “project” for the required environment.
So, why didn’t we group environments per project? That’s because we use those environment groups to handle permissions for project configurations and running builds. We have a group of users that are the only ones that are allowed editing and running the builds for production evironments; the same goes for the staging environment.
The final project tree looks like this:
As you add new projects, you will soon notice that the build configurations you create become repetitive. Here’s where TeamCity templates come in. Instead of copying the build configuration and changing some of it’s parameters, you can create a template with a set of configurable parameters. Lets say you’re sequence looks like this:
4.Deploy to remote server
5.Tag a revision in the repository
There are only a few parameters that might change when you create a new project with the same sequence, for example: source repository, repository for dependencies, testsuite, remote server information and a prefix for the tag.
You create the template the same way that you create a project build configuration, and then you define parameters that might change as variables. Whenever you need to recreate the same build steps with a different repository and a tag prefix, you just create build configuration based on the template and you fill in the variables in a generated form.
Done! While creating a project based on a template you might see something like this:
As you can see, I’ve added three parameters that I use in my template, and I can specify their initial values right before I create the configuration.
Another nice thing about the hierarchical project structure is that the templates, VCS roots, Meta-runners and SSH Keys are saved on the project level and are only visible to child projects, so if we create a template on the PHP platform level, we can reach the template in all PHP environments and projects. But, it won’t be visible in the .NET platform.
Build Runners and Meta-runners
When you create a project, there are two things you can do next. Either create a subproject or start adding build configurations for that project.
Build configurations consist of build steps, and a build step is usually a configured build runner. TeamCity comes with alot of usefull build runners for JAVA and .NET, and you can install additional runners or meta-runners.
For example, a common build runner is the “Ant runner”. When you add it to the build configuration, you will have options to configure it, such as path to the build.xml, targets to run, JDK versions, additional command line parameters and so on. The GUI for managing your build runner configurations is clean, easy to understand, and has a lot of information explaining what’s where.
Sometimes you use tools that are not included in the default installation of the TC Agent, such as the “Heroku tool”, that allow you to deploy to the cloud. You then have three options:
1.Configure a Command Line Runner to execute heroku commands
2.Download an existing Heroku meta-runner
3.Create you own meta-runner with your defined configuration fields.
A meta-runner works exactly like a normal runner. You create it similarly to the build configuration template, but it actually allows you to extract the build steps and the parameters that are required to allow the use of extracted steps as a single runner with the ability to configure those parameters. This is useful when you have some build steps that are used in other build configurations. The meta-runner itself is extracted as an XML so you can share it with others on the web.
Here’s an example of how I convert common build steps to a single runner.
First, I assign parameters to the dynamic parts of my build steps. Here I add %first_parameter% and %second_parameter% to my command line runner, and the second build step just runs a simple cleanup script.
Secondly, I export the build configuration as a “meta-runner”. You just give it a name, in my case “Example”.
We can then use the newly exported meta-runner in other build configurations. Instead of seeing two additional build steps, we only see one with the parameters we defined.
This simplifies parts of your configurations and allows for better reusability.
Distributing Builds over Agents
TeamCity allows you to distribute builds over different build servers, called Agents. Every time you run a build, it searches for an available agent from the “Agent pool”. If the build requirements for that agent match, then the build is executed on it.
Additionally, each agent might have different parameters, such as:
●Software installed on that agent
You can use these parameters to configure your build requirements. For example, you might want to run the build on those agents that only have PHP installed, or when you have two separate agent’s with different types of Windows and you want to see if the build is successful on both of them.
The installation and configuration of those agents is pretty simple and straightforward (especially on Windows). Installation includes the software required for all of the included runners like NuGet, Ant, MSBuild, etcetera.
We’ve been using TeamCity for about six months now. Since then we’ve created numerous project builds and already have a lot of reusable material for future builds. The learning curve of how to create and edit builds is pretty low, so developers new to TC can jump in right away and start creating builds by follow a few of our guidelines.