Azure DevOps + CI / CD + docker

Hey Guys!

In my previous article, I described how to easily create infrastructure for host Docker containers. Today I want to do present, how to connect it with CI / CD process.

Requirements.

  • The project with defined Dockerfile file,
  • The account in Azure Portal | Azure DevOps | Github (or another repository),
  • Server with infrastructure,
  • 1 – 3 hours of work.

Azure portal – configuration.

Firstly, You must create subscription at Your azure portal account (of course if you don’t have it for now and don’t worry, is free operation ;-).

Then, the next step is to create a special azure container registry, for a dedicated application. To do this You must click Create a resource and choose Azure Container Registry.

Probably this step can be replaced by dockerhub service, but in my case I prefer keeps the a lot of things in one place ;-). My current costs for 3 containers (backend, frontend & cdn) are equal to 4.99 euro monthly.

Don’t forget about enable login option for Admin user in Access Keys section!

Azure DevOps – configuration.

If You don’t have any project created yet, please create a new one by click + New project.

Now, You have to make one of the most important decisions in your life… ( sorry, I had to 😀 ). Where You will keep Your repositories? I’m a big fan of GitHub, so in this post, I will focus on that way of persistence, but You can also choose Azure DevOps (btw. in one project, You can have X Repos <3). When You pushed Your repo with Dockerfile, please go into Your project, and choose Pipelines section.

Azure DevOps – Pipelines.

Pipelines in azure can build & publish code. In our case, pipeline will be responsible for creating dockers images and pushing them to created previously – Azure Container. To achieve this, You must click the blue, magic button – New Pipeline.

Now You can see a very powerfully & user-friendly Pipeline Wizard. In the First step you should decide, Where is Your code. If You have, like me a big love relationship with GitHub it isn’t a problem ;-)!

After selection, You must specify what exactly repository should be handled by this pipeline.

Then You must choose type of pipeline in our case it will be Docker – Build and push an image to Azure Container Registry. After clicked on that You can choose, what subscription will be used here & where image will be stored. Don’t miss about location of dockerfile ;-)!.

At Finish, You have generated a YAML file with content related to building a docker image. The very important remind here, please check what system was used for image build. It should be this same, like Yours server in my case it will be Linux system, but if You have Windows Server, change it to this same like destination.

Very good habit is also set docker images tags related to type of image. In my case, there will be always 3 types:

  • production,
  • stage,
  • release.

It can be done by add variable like tag, You can see this sample in above snippet in line 17. To manipulate this value by special branch I use:

stages:
- stage: Build
  displayName: Build and push stage
  jobs:  
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)
    steps:
    - bash: |
          if [ $(Build.SourceBranch) = "refs/heads/release" ] ; then
            echo "##vso[task.setvariable variable=tag]production"
          elif [ $(Build.SourceBranch) = "refs/heads/master" ] ; then 
            echo "##vso[task.setvariable variable=tag]staging"
          else
            echo "##vso[task.setvariable variable=tag]test"
          fi
    - task: Docker@2
      displayName: Build and push an image to container registry
      inputs:
        command: buildAndPush
        repository: $(imageRepository)
        dockerfile: $(dockerfilePath)
        containerRegistry: $(dockerRegistryServiceConnection)
        tags: |
          $(tag)

And for lazy bastards, powershell:

- pwsh: |
    If ("$(Build.SourceBranch)" -eq "refs/heads/release") {
      Write-Host "##vso[task.setvariable variable=tag;]production"
    }
    If ("$(Build.SourceBranch)" -eq "refs/heads/master") {
      Write-Host "##vso[task.setvariable variable=tag;]staging"
    }
    If ("$(Build.SourceBranch)" -eq "refs/heads/develop") {
      Write-Host "##vso[task.setvariable variable=tag;]test"
    }

Azure DevOps – release.

Okay, now we have built and pushed our docker image. Time is to create a deploy process. Firstly You must register Your host in Deployment pools. In this step, you have to take decision – what is scope of this machine in Your organization. If it will be used only by one project, You should register it on the project level, else choose the organization level (I always prefer the second option – organization level). To do this please go to Organization Settings -> Deployment pools, + New button. Here You can also set permissions to projects.

Now, You must invoke script (bash | PowerShell) in Your host machine. After that, You will see pretty, green status next to Your machine name ;-).

Use a personal access token in the script for authentication – this option is very helpful, can handle all things related to access | permissions. I usually use it, in another way token should be passed while the script will be invoking.

Well, the last thing to configure is release. The release section in azure is responsible for delivering products to host machines. To do this step, please go to Releases and click + New.

You should see again, very user-friendly & powerfully wizard. Please select a template Empty job.

Artifacts – the result of pipeline works. It not will be used in our case directly, (because our artifact is hosed on container registry) but it can be very useful for creating automatic triggers that can start deploying process after succeeding pipeline work.

If You want to enable trigger, please click on thunder icon and select Enabled option. You can also set special filters (ex. for master branch, deploy it to production).

Stage – work on dedicated machine. After clicked on that, default task is Agent Job. It should be removed, for this partial case all job will be invoked on Deployment group job, so please create it now.

But for now, before the configuration of job definitions, we will define release variables. It’s very helpful because these values will be passed into multiple places + you can create some generic (tfu..) staff ;-). Also, If Your application is occurred in .net core environment, here You can define some appsettings.json values like connection strings | address to CDN. I also used this mechanism for Angular, here You can read more about it.

Great! So for now, we have only one step left – configure of job on deployment pool. Before start it, You must be sure, Task should be in type Run on deployment group. Now, You must select the Deployment group (Your destination host) and set name ;-).

Ok, let’s define first task on our stage. It will be docker login. This is very important step, without it our destination host, can’t fetched any custom docker images, hosted on azure container repository.

To do it, click on +, next to Deploy, and select Add.

By default, it’s type buildAndPush, but we already did it by our pipeline process. For define, this type should be changed to login.

Remember also about selection of Container registry!.

Great! For now, You must create the rest of docker operations. All things will occur in bash scripts, so here You must create another task but for bash type. Firstly, it will be docker pull (always, must be sure to have the freshest image). Azure will replace variable $(image.production) to value define while ago in variables task.

docker pull $(image.production)

The second operation should be a docker stop. But if this is first deployed, You don’t have an image with that name, so an exception will be throw ;-). To handle it, You should check checkbox Continue on error.

docker stop $(docker.name)

Well 3rd task is removing of actual image, but still with option Continue on error.

docker rm $(docker.name)

And finally the last task, docker run ;-).

docker run --detach \
  --restart=always \
  --name $(docker.name) \
  -e VIRTUAL_HOST=$(host) \
  -e LETSENCRYPT_HOST=$(host)  \
  -e sql:connectionString=$(settings.cs) \
  -e cdn:address=$(settings.cdn.address) \
  --expose 80 \
  --expose 443 \
$(image.production) 

You see in my script I inject some values strongly related to my project sale architecture b2c. This project can have multiple instances, for different organizations, so settings will also have a lot of variants. The best option to modify it is docker environment variables. In my case, .Net Core keeps all information in appsettings.json file, it can be changed by additional arguments passed to docker run with parameter -e.

sql:connectionString

{
  "sql": {
    "connectionString": "***",
    "inMemory": false,
    "schema": "public"
  },
}

VIRTUAL_HOST | LETSENCRYPT_HOST are described in my previous article 😉