CircleCI Basics

CircleCI consists of jobs which can be run in parallel or in sequence. Each job consists of a set of steps which can execute Docker commands for us. A config.yml file is created inside a .circleci folder at the root of the project which contains all the instructions on the jobs to run.

# .circleci/config.yml
version: 2.1
jobs:
  build:
    docker:
      - image: docker:17.05.0-ce-git
    steps:
      - run: echo hello-world

jobs --> the key which is used to describe single or multiple jobs

build --> is the name of the job, can be named to anything

docker --> specifies the type of executor like machine, docker, macOS, etc

image --> the primary image used for the primary container

steps --> contains the series of steps which need to be executed in the primary container

run --> one of the many commands type which run under steps (can also use deploy, checkout etc - discussed later)

How does it work

CircleCI spins up a main docker container called - primary environment for each job. All the commands we list under steps for a given job, will be executed in this main container. There is an exception to this when we use docker as our executor.

In that case, for any docker commands relating to building/pushing images and running containers, our primary container requires a separate remote docker environment/container which is used only for those commands.

We can create this by providing a special property called setupremotedocker. We can also provide a version of docker we would like to run (just in case) or else leave it as a string value.

Now we can run any docker related commands like so:

# .circleci/config.yml
version: 2.1
jobs:
  build_and_test:
    docker:
      - image: docker:17.05.0-ce-git
    steps:
      - checkout
      - setup_remote_docker
      - run: docker build -f Dockerfile.dev -t app-testing .
      - run: docker run -e="CI=true" app-testing npm run test

NOTE: The checkout prop is used to indicate to copy over all the code from our VCS into the default directory which is /home/circleci/project by default. This can be changed by using the working_directory prop, if needed. Without the checkout prop, our code won't be copied and we will get error like Docker file not found.

Deploy React App to S3 with CircleCI & Docker

After creatng a new app with create-react-app tool, create a repo with github or bitbucket and push the code over to the master branch.

Steps on what we want to achieve:

• create 2 dockerfiles - one for PROD and one for DEV
• Use docker to run our unit tests on circleCI with dev dockerfile
• Create a final build using docker with prod dockerfile
• Finally, deploy the build to AWS S3 which is pre-configured to run as a static site

NOTE: the last 3 steps will run on circleCI

Dev Dockerfile

# Dockerfile.dev
FROM node:alpine

WORKDIR "/app"

COPY package.json .
RUN npm install

COPY . .

CMD ["npm","run","dev"]

This is a typical configuration for the dev environment. By default, we want to be able to run our server locally and see everything working as expected.

However, when running this file on circleCI, we want to run our tests first before the final deloyment takes place. To achieve this, we will overrride the default CMD of this dockerfile with npm run test during run time.

We have already set this up in the above example YAML file with job named - buildandtest.

Prod Dockerfile

While setting things up for production, we have to mindful of few important things.

Firstly, we could have used the Dockerfile.dev file for PROD as well, all we had to do was override the default command with npm run build. However, for separation of concerns and any future changes, we decided to keep them separate.

Secondly, when we produce our final build, it will live inside a remote docker container and we will need to copy it over to our local primary container.

Thirdly, we will take advantage of orbs which are pre-written steps of functionality and will allow us to keep moving quickly especially with installing things like aws-cli for S3.

# Dockerfile
FROM node:alpine

WORKDIR "/app"

COPY package.json .
RUN npm install

COPY . .

CMD ["npm","run","build"]

Let's add another job to our circleCI config and name it buildanddeploy.

orbs:
  aws-s3: circleci/aws-s3@1.0.15
jobs:
  build_and_deploy:
    docker:
      - image: circleci/node:10.16.3
    steps:
      - checkout
      - setup_remote_docker
      - run: docker build -t react-app-build:1.0.0 .
      - run: docker run --name myapp react-app-build:1.0.0
      - run: docker cp myapp:/app/build .
      - aws-s3/copy:
          arguments: "--recursive --acl public-read"
          from: build
          to: s3://docker-react-nishant/

Let's break down the steps:

Step 1 --> copy over all the source code from VCS into our primary docker container's default location

Step 2 --> setup a remote docker container instance for running all build/run related docker commands

Step 3 --> Build an image using main Dockerfile and tag it with a name react-app-build:1.0.0

Step 4 --> Run the image created in the last step and name the container myapp

Step 5 --> Using the name of the container from previous step, copy over the build folder to our local docker container

Step 6 --> Use circleCI's pre-built orb for deployment to AWS S3

NOTE: when copying over the build folder, we use the syntax :. In our prod dockerfile, we had specified our working directory to be /app. This ensured our build folder to live at /app/build inside the remote docker container.

Now finally, we will use workflows property provided by circleCI to ensure our tests run first and only when they successfully pass, will our build be created and deployed to AWS S3.

workflows:
  my-custom-workflow:
    jobs:
      - build_and_test
      - build_and_deploy:
          requires:
            - build_and_test

Here we have listed our jobs and have also specified that buildanddeploy job requires buildandtest job to finish first.

NOTE: We can also make use of filters prop to even specify the branch for which we should run a specific job. This is specially helpful when we are using staging and prod branches in github. Perhaps, for any changes on staging branch, we want to deploy to a different S3 folder or maybe even deploy to an EC2 or ECS instance, all this can be achieved by specifying filters -> branches -> branch name.

Finally, the full YAML file for circleCI looks like so:

version: 2.1
orbs:
  aws-s3: circleci/aws-s3@1.0.15
jobs:
  build_and_test: # this runs for checking our tests
    docker:
      - image: circleci/node:10.16.3
    steps:
      - checkout
      - setup_remote_docker
      - run: docker build -f Dockerfile.dev -t app-testing .
      - run: docker run -e="CI=true" app-testing npm run test

  build_and_deploy: # only runs when all tests pass and used for deployment
    docker:
      - image: circleci/node:10.16.3
    steps:
      - checkout
      - setup_remote_docker
      - run: docker build -t react-app-build:1.0.0 .
      - run: docker run --name myapp react-app-build:1.0.0
      - run: docker cp myapp:/app/build .
      - aws-s3/copy:
          arguments: "--recursive --acl public-read"
          from: build
          to: s3://docker-react-nishant/

workflows:
  my-custom-workflow:
    jobs:
      - build_and_test
      - build_and_deploy:
          requires:
            - build_and_test

Main CircleCI Configuration Reference

Configuration Reference relating to Docker

Running Docker Commands with circleCI

AWS-S3 circlCI Orb Doc

Using workflows to manage complex Configuration

Executors and Images Description