At the beggining of the project I had to take a decission, what CI Tool should I used. In that moment, I compare TravisCI and CircleCI face 2 face.

Benefits

  • GitHub Native Implementation
  • Pipeline-as-code CI configuration & delivery
  • Easy to configure and start from scratch
  • Pre configured to work with Golang OOB (out of the box)
  • SAAS

Lacks

  • Fixed configurations in some things
  • I had to follow Travis / Circle rules
  • Shared resources
  • Long builds (5 - 10 min vs 1 - 2 min locally)

So, few months ago, I bet for TravisCI, and in the both subprojects worked like a charm, configuration was easy and with a minimum of time inversion I had my CI infrastructure working.

(You can view my travis.yml if desired)

language: go

services:
  - docker

go:
  - tip
  - master

before_install:
# Setup dependency management tool
- curl -L -s https://github.com/golang/dep/releases/download/v0.3.1/dep-linux-amd64 -o $GOPATH/bin/dep
- chmod +x $GOPATH/bin/dep
- docker swarm init

install:
  - dep ensure
  - go get golang.org/x/tools/cmd/cover
  - go get github.com/mattn/goveralls

before_script:
  - go get -t -d -v ./... 

script:
  - go test ./... -race -coverprofile=coverage.txt -covermode=atomic
  - go build -v ./...
  - $GOPATH/bin/goveralls -service=travis-ci

after_success:
  - bash <(curl -s https://codecov.io/bash)

Facing problems

I started to work with Docker Swarm using the Docker Engine SDK/API. Locally everything, code & tests worked OOB. But the (big) problem came to scene. TravisCI and CircleCI doesn't allow to perform the command docker swarm init.

This step is mandatory due to the essence of Automation Test Queue is Docker Swarm.

To not have a Docker Swarm configured in my test environment started to broke my tests because my tests could not be executed.

Migration

I've been working with Jenkins few years ago. In the beggining as an Automation Engineer and in the last year as DevOps.

So I know the usage, configuration & maintainability of Jenkins, this will help me to design and start a new CI pipeline.

For the moment I only run Unit Test, this makes the things much easier.

Steps

  • Find a machine to deploy Jenkins (should be able to access Docker Socket)
    • 24/7 accessible
  • Design a Jenkins Pipeline
  • Set up Git-flow

Amazon Web Services to the rescue

AWS offers a Basic account for students with a free EC2 instance (12-months). For the moment, only one instance should be enough for the project.

We can scale it up whenever, so this provider fits our needs.

I've installed a Minimal Ubuntu Server image. Also we need to install more stuff:

  • Jenkins (TCP port 8080 : Enabled in EC2 console also)
  • Docker

Configuring Jenkins

  • GitHub integrations: WebHooks & repositories
  • Docker socket for Docker-in-Docker
  • Adding jenkins user to docker group

Creating a Docker Image for testing & building

golang official image should be enough to start, but after few tries, I faced with some problems:

  • dep it's not installed by default
  • dep installation takes time
  • Some permission problems

The easiest solution is to generate a new Docker Image for our needs:

FROM golang:latest

RUN apt update -y \
  && apt install musl-dev -y
  
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh

RUN mkdir -p .cache

# Install golang/dep
WORKDIR /root
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
ENV PATH="/root/bin:${PATH}"

Also an Alpine flavoured version:

FROM golang:alpine

RUN apk update \
  && apk add ca-certificates wget unzip git \
  && update-ca-certificates \
  && apk add curl \
  && apk add gcc \
  && apk add musl-dev \
  && apk add bash

RUN mkdir -p .cache

# Install golang/dep
WORKDIR /root
RUN curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
ENV PATH="/root/bin:${PATH}"

Disclaimer: Alpine version it's not compatible with cGo & other components, like running go test with -race flag

https://github.com/mtenrero/go-dep-Docker
tenrero/golang-dep-alpine

Jenkinsfile

pipeline {

    agent {
        docker {
            image 'tenrero/golang-dep-alpine:1.10.2'
            reuseNode true
            args '-it -v /var/run/docker.sock:/var/run/docker.sock -v $WORKSPACE:/go/src/app -w /go/src/app'
        }
    }

    stages {        
        stage('Prepare Environment') {
            steps {
                sh 'echo $GOPATH'
                sh 'rm -fr /go/src/app/vendor && exit 0'
                sh 'go get -u github.com/golang/dep/cmd/dep'
                sh 'go get -u github.com/golang/lint/golint'
                sh 'go get -u github.com/tebeka/go2xunit'
                sh 'go get -u golang.org/x/tools/cmd/cover'
                sh 'go get -u github.com/mattn/goveralls'
            }
        }

        stage('Download Vendor') {
            steps {
                sh 'cd /go/src/app && dep ensure'
            }
        }

        stage('Test') {
            steps {
                sh 'cd /go/src/app && go test ./... -race -coverprofile=coverage.txt -covermode=atomic'
            }
        }

        stage('Build') {
            steps {
                sh 'cd /go/src/app && ./build.sh'
            }
        }
    }
}