Gitlab - templates

2021-06-15 5 min read 3sky

Welcome

Working as a consultant is hard. You have so many tasks to do. The day is a bit too short, but the opportunities to learn are awesome. Today I have time to compile my notes about GitLab, mostly CI part. However, as a code repository for organizations, GitLab is my favorite solution. Fast, easy to use, permission matrix, and deployment keys management. All this stuff is awesome. It’s not a marketing post, but could be :)

GitLabCI

It’s a very popular solution because it’s easy to use and fast to set up, fortunately, it’s free - if you don’t cross the limit (400 CI/CD minutes per month). The main thing is setup here. It’s just one file called .gitlab-ci.yml, placed in the repository root.

Problem

There are no problems in this article. That will be text about templating, and why they are helpful. Let’s assume that we have a simple pipeline for deploying on VM. Also, take a look at code comments.

image: ubuntu:latest

before_script:
    - apt update
    - apt install openssh-client -y
    - eval $(ssh-agent -s)
    # base64 is very usefull in case of id_rsa formating
    - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
    - mkdir -p ~/.ssh
    - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

stages:
    # order of stages is important and is configure here
    - create_env
    - build
    - deploy

# now we need to build .env file for compilation time
# that's tag driven deployment, so we deploy to particular
# env based on tag

build_the_dotfile_dev:
    stage: create_env
    script:
      - echo "$DOTFILE_DEV" > .env
    artifacts:
      paths:
        - .env
    rules:
      - if: '$CI_COMMIT_TAG =~ /^development/'

build_the_dotfile_stg:
    stage: create_env
    script:
      - echo "$DOTFILE_STG" > .env
    artifacts:
      paths:
        - .env
    rules:
      - if: '$CI_COMMIT_TAG =~ /^staging/'

build_the_dotfile_prod:
    stage: create_env
    script:
      - echo "$DOTFILE_PROD" > .env
    artifacts:
      paths:
        - .env
    rules:
      - if: '$CI_COMMIT_TAG =~ /^production/'

# now we build our app
# this step is generic
build_the_app:
    image: node:12-buster
    stage: build
    script:
        - git checkout -b '$CI_COMMIT_TAG'
        - yarn install
        - NODE_ENV=production yarn build
        - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
    artifacts:
      paths:
        - current.tar.gz
      expire_in: 2 hrs
    rules:
      - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'

# logic is simple
# 1. we transfer our artifact to VM
# 2. run the deployment script with some mv/pm2 logic
# 3. environment based on TAG again
deploy_front_to_dev:
    stage: deploy
    script:
        - scp current.tar.gz deployer@$DEV_SERVER_IP:/home/appuser/front
        - ssh deployer@$DEV_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
    rules:
      - if: '$CI_COMMIT_TAG =~ /^development/'

deploy_afront_to_stg:
    stage: deploy
    script:
        - scp current.tar.gz deployer@$STG_SERVER_IP:/home/appuser/front
        - ssh deployer@$STG_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
    rules:
      - if: '$CI_COMMIT_TAG =~ /^staging/'

deploy_front_to_prod:
    stage: deploy
    script:
        - scp current.tar.gz deployer@$PROD_SERVER_IP:/home/appuser/front
        - ssh deployer@$PROD_SERVER_IP "bash /home/appuser/deploy.sh front $CI_COMMIT_TAG"
    rules:
      - if: '$CI_COMMIT_TAG =~ /^production/'

That’s nice, but also too long - 83 lines of code. Likewise, it produces too many typo possibilities…

Solution

Here comes templates. It’s some kind of object, which we can just call in our pipeline. After adding templates code looks like that:

stages:
  - create_env
  - build
  - deploy

# note dot on the begining
# templates must start with dot, otherwise
# will be executed
.env_template: &envs
  stage: create_env
  image: alpine
  script:
    - echo "$DOTFILE" > .env
  artifacts:
      paths:
        - .env

.deploy_template: &deploy_tmpl
  stage: deploy
  image: ubuntu:latest
  before_script:
    - apt update
    - apt install openssh-client -y
    - eval $(ssh-agent -s)
    - ssh-add <(echo "$DEPLOYER_PRIVATE_KEY" | base64 -d)
    - mkdir -p ~/.ssh
    - 'echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  script:
    - scp current.tar.gz deployer@$SERVER_IP:/home/appuser/front
    - ssh deployer@$SERVER_IP "bash /home/appuser/bin/deploy.sh front $CI_COMMIT_TAG"

build_the_dotfile_dev:
  variables:
    DOTFILE: "$DOTFILE_DEV"
  <<: *envs
  rules:
    - if: '$CI_COMMIT_TAG =~ /^development/'

build_the_dotfile_stg:
  variables:
    DOTFILE: "$DOTFILE_STG"
  <<: *envs
  rules:
    - if: '$CI_COMMIT_TAG =~ /^staging/'

build_the_dotfile_prod:
  variables:
    DOTFILE: "$DOTFILE_PROD"
  <<: *envs
  rules:
    - if: '$CI_COMMIT_TAG =~ /^production/'

build_the_app:
  image: node:12.22.1-buster
  stage: build
  before_script:
    - yarn policies set-version 1.22.10
  script:
    - git checkout -b '$CI_COMMIT_TAG'
    - yarn install
    - NODE_ENV=production yarn build
    - tar --warning=no-file-changed --exclude=current.tar.gz --exclude-vcs -zcf current.tar.gz ./ || [[ $? -eq 1 ]]
  artifacts:
    paths:
      - current.tar.gz
  rules:
    - if: '$CI_COMMIT_TAG =~ /^development|^staging|^production/'

deploy_front_to_dev:
  variables:
    SERVER_IP: "$DEV_SERVER_IP"
  <<: *deploy_tmpl
  rules:
    - if: '$CI_COMMIT_TAG =~ /^development/'

deploy_afront_to_stg:
  variables:
    SERVER_IP: "$STG_SERVER_IP"
  <<: *deploy_tmpl
  rules:
    - if: '$CI_COMMIT_TAG =~ /^staging/'

deploy_front_to_prod:
  variables:
    SERVER_IP: "$PROD_SERVER_IP"
  <<: *deploy_tmpl
  rules:
    - if: '$CI_COMMIT_TAG =~ /^production/'

I know it’s still long. But the logic is in one place not in 3, also managing the new environment will be easy and clean. You can add templates as separated files and then just import them.

Testing

Another intresting topic, it’s important to check the syntax before committing to dev. So there is an easy option to test your stages. The only thing you need is to run this command in your local repository.

docker run -d \
  --name gitlab-runner \
  --restart always \
  -v $PWD:$PWD \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest

Then to execute the task type.

docker exec -it -w $PWD gitlab-runner gitlab-runner exec docker <task name>

There is one issue that makes me cry… I can’t find an option to run the whole pipeline. Running stages is fine, but there is no option to test caching, or sharing files between tasks.

Summary

No summary require. It’s clean and easy-to-use templates, GitLab is awesome, but life is hard. I’m 27yo, yesterday I bought my first bicycle helmet. Safety first. Take care.