Introduction
Bitbucket Pipelines is an excellent CI/CD tool that allows users of Bitbucket to automate the Continuous Integration and Continuous Delivery of their software, including:
- Building code
- Running automated tests
- Auditing code for vulnerabilities
- Performing static code analysis
- Packaging code artifacts
- Publishing code
- Deploying code
Challenge
One of the challenges developers may face after adopting Bitbucket Pipelines for multiple repositories is the need to add steps to their pipelines that install software not included in their chosen base Docker image. Examples include installing:
Over time, developers may end up with the same steps copied & pasted between pipelines and repositories, which can become burdensome to manage, and add to the runtime of pipelines.
Solution
One way to address this challenge is by creating a custom Docker image to use for pipelines. These images can be hosted on Docker Hub or, in the case of this example, Amazon Elastic Container Service.
In this post, we will demonstrate:
- Creating a custom Docker image for building .NET Core code
- Using Bitbucket Pipelines to tag and push images to Amazon ECS
- Using the custom Docker image in Bitbucket Pipelines
Creating the Docker Image
The .NET 6.0 Docker Image provided by Microsoft does not including common packages such as Node.js and NPM, or utilities for compression. These dependencies are useful for publishing web applications. In addition to these dependencies, we also use the following tools in many of our builds at IDMWORKS:
- AWS CLI
- Library Manager CLI
- Sonar Scanner for .NET
- Java JRE (required by Sonar Scanner)
Let’s take a look at what a Dockerfile would look like that includes the above tools.
FROM mcr.microsoft.com/dotnet/sdk:6.0
USER root
# Install zip, unzip, JRE, and Node.js
RUN curl -sL “https://deb.nodesource.com/setup_16.x” | bash –
RUN apt-get update \
&& apt-get install –yes –no-install-recommends \
zip \
unzip \
openjdk-11-jre \
nodejs
# Install the Amazon AWS CLI
RUN curl “https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip” -o “awscliv2.zip”
RUN unzip awscliv2.zip \
&& ./aws/install \
&& rm -r ./aws/
# Install the Library Manager CLI and Sonar Scanner
RUN dotnet tool install -g Microsoft.Web.LibraryManager.Cli \
&& dotnet tool install –global dotnet-sonarscanner
# Clean up temp files
RUN rm -r /tmp/*
# Remove unneeded packages
RUN apt-get -qy autoremove \
&& rm -rf /var/lib/apt/lists/*
We won’t go into too much detail here as it’s beyond the scope of this post; this is a fairly straight-forward Docker file that extends the image mcr.microsoft.com/dotnet/sdk:6.0 by installing our common build tools.
Pushing the Docker Image
To begin using the new Docker image, the first thing we’ll want to do is a create a new repository (either private or public) in Amazon Elastic Container Registry. In this example, we’ll create a repository for the fictional company Hooli named hooli/dotnet-6-sdk.
Once we’ve created the repository, we can click the View push commands button in the Amazon AWS Management console to see the commands we can use to push images from the command line, e.g.:
Next, we’ll introduce a bitbucket-pipelines.yml file so that we can use conventional Git workflows and CI/CD to automate tagging and pushing the image to Amazon ECS.
As before, we’ll need to provide values for the following pipeline repository variables:
- AWS_HOOLI_ACCESS_KEY_ID: access key ID for a service account in the Hooli AWS account
- AWS_HOOLI_SECRET_ACCESS_KEY (secured): secret access key for a service account in the Hooli AWS account
Conclusion
With these pieces put together, we now have a custom Docker image, hosted in Amazon ECS, that can be used to build .NET Core code without having to install additional dependencies in each pipeline. In addition, the Docker image itself has a set of pipelines that automate the Continuous Integration and Continuous Delivery of the image, automatically tagging and pushing the image using conventional Git workflows.
In future posts, we’ll look at related topics surrounding the automation of CI/CD with Bitbucket Pipelines and .NET Core code, such as:
- Publishing and consuming custom NuGet packages w/ AWS CodeArtifact
- Deploying applications to AWS Elastic Beanstalk
- Scanning for vulnerable NuGet packages with the .NET CLI
- Performing static code analysis using SonarCloud
Author: Nathanial Woolls, IDMWORKS, IAM Architect
Of note, we’re using the amazon/aws-cli image so that the AWS CLI is available, and we’re using the docker Bitbucket Pipelines service so we have access to Docker commands. We’re also using YAML anchors to introduce re-usable pipeline sections.
To make use of this bitbucket-pipelines.yml file, we’ll need to enable Bitbucket Pipelines for the repostory, and provide values for the following pipeline repository variables:
- AWS_HOOLI_ACCESS_KEY_ID: access key ID for a service account in the Hooli AWS account
- AWS_HOOLI_SECRET_ACCESS_KEY (secured): secret access key for a service account in the Hooli AWS account
With this configuration in place:
- Pushing (or merging) code to the develop branch will build & push a Docker image to AWS ECR with the tag “latest”.
- Pushing (or merging) code to the master branch will build & push a Docker image to AWS ECR with the tag “lts”.
- Pushing a Git tag with a version — such as 1.0.0 — will build & push a Docker image to AWS ECR with a tag based on the Git tag.
Using the Docker Image
Once one of the above Git workflows has been performed in order to push the tagged image to Amazon ECS, new and existing pipelines for building .NET Core code can be updated with the following at the top of their bitbucket-pipelines.yml file:
image:
name: 1234567890.dkr.ecr.us-east-1.amazonaws.com/hooli/dotnet-6-sdk:latest
aws:
access-key: $AWS_HOOLI_ACCESS_KEY_ID
secret-key: $AWS_HOOLI_SECRET_ACCESS_KEY
As before, we’ll need to provide values for the following pipeline repository variables:
- AWS_HOOLI_ACCESS_KEY_ID: access key ID for a service account in the Hooli AWS account
- AWS_HOOLI_SECRET_ACCESS_KEY (secured): secret access key for a service account in the Hooli AWS account
Conclusion
With these pieces put together, we now have a custom Docker image, hosted in Amazon ECS, that can be used to build .NET Core code without having to install additional dependencies in each pipeline. In addition, the Docker image itself has a set of pipelines that automate the Continuous Integration and Continuous Delivery of the image, automatically tagging and pushing the image using conventional Git workflows.
In future posts, we’ll look at related topics surrounding the automation of CI/CD with Bitbucket Pipelines and .NET Core code, such as:
- Publishing and consuming custom NuGet packages w/ AWS CodeArtifact
- Deploying applications to AWS Elastic Beanstalk
- Scanning for vulnerable NuGet packages with the .NET CLI
- Performing static code analysis using SonarCloud
Author: Nathanial Woolls, IDMWORKS, IAM Architect