3sky's notes

Minimal blog about IT

Clojure app on Google Cloud Run

2020-03-18 11 min read 3sky

Welcome

I want to create a small pipeline based on GCP Run and GitHub Action. Also increasing skills while #statathome is the best possible solution for spending time. Let’s make some real stuff.

Tools used in this episode

  • Google Cloud Platform
  • Terraform
  • Docker
  • Clojure
  • Github Action

Google Cloud Platform

Why Google Cloud Platform

I like Web GUI, command-line tools, GKE. I also have some credits to use. Cloud Run is very nice and cheap for a one-docker small app.

Let’s code - Google Cloud Platform

  1. Create a new project

    gcloud projects create [PROJECT_ID] --enable-cloud-apis
    
    # --enable-cloud-apis
    # enable cloudapis.googleapis.com during creation
    # example
    # gcloud projects create my-small-gcp-project --enable-cloud-apis
    
  2. Enable services

    gcloud services list --available | grep -e run -e compute -e container
    gcloud services enable compute.googleapis.com
    gcloud services enable container.googleapis.com
    gcloud services enable run.googleapis.com
    
  3. Set project name

    gcloud config set project my-small-gcp-project
    
  4. Create a service account and add necessary permission

    gcloud iam service-accounts create ci-cd-user \
    --description "Account for interact with GCP Run, CR and GitHub" \
    --display-name "my-github-user"
    
    gcloud projects add-iam-policy-binding my-small-gcp-project \
    --member \
    serviceAccount:[email protected] \
    --role roles/compute.admin
    
    gcloud projects add-iam-policy-binding my-small-gcp-project \
    --member \
    serviceAccount:[email protected] \
    --role roles/run.serviceAgent
    
    gcloud projects add-iam-policy-binding my-small-gcp-project \
    --member \
    serviceAccount:[email protected] \
    --role roles/run.admin
    
    gcloud projects add-iam-policy-binding my-small-gcp-project \
    --member \
    serviceAccount:[email protected] \
    --role roles/storage.admin
    

Terraform

Why Terraform

I like working with IaaC approach and I need some testing machine. Especially for docker part, I don’t like to work with Docker for Windows. Also if we want to reproduce solutions everywhere that’s the fastest solution. Also sometimes ago I decided that running playgrounds in clouds is cheaper and faster than taking care of the workstation. With IaaC I need to have only terminal and code editor.

Let’s code - Terraform

  1. Getting project credentials in JSON.

    gcloud iam service-accounts keys create auth.json \
    --iam-account [email protected]
    
  2. Add auth.json to .gitigonre

    echo "auth.json" > .gitignore
    echo ".terraform/" >> .gitignore
    
  3. Create main.tf

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    
    locals {
        region_eu = "europe-west3-a"
        p_name = "my-small-gcp-project"
    }
    
    provider "google" {
        credentials = file("auth.json")
        project     = local.p_name
        region      = local.region_eu
    }
    
    // Terraform plugin for creating random ids
    resource "random_id" "instance_id" {
        byte_length = 8
    }
    
    // A single Google Cloud Engine instance
    resource "google_compute_instance" "default" {
    count = 1
    name         = "build-machine${random_id.instance_id.hex}"
    machine_type = "e2-medium"
    zone         = local.region_eu
    
    boot_disk {
    initialize_params {
        image = "ubuntu-1804-bionic-v20200129a"
    }
    }
    metadata = {
        // everyone has rsa key, right ?
        ssh-keys = "kuba:${file("~/.ssh/id_rsa.pub")}"
    }
    
    // Make sure flask is installed on all new instances for later steps
    metadata_startup_script = "sudo apt-get update; sudo apt-get upgrade -y; "
    
    network_interface {
    network = "default"
        access_config {
        // Include this section to give the VM an external ip address
        }
    }
    }
    
    resource "google_compute_firewall" "default" {
        name    = "app-firewall"
        network = "default"
    
    allow {
        protocol = "tcp"
        ports    = ["80"]
        }
    }
    
    // A variable for extracting the external ip of the instance
    output "m1" {
        value = "${google_compute_instance.default.0.network_interface.0.access_config.0.nat_ip}"
    }
    
  4. Initialize a working directory containing Terraform configuration files

    terraform init
    
  5. Apply the changes required to reach the desired state of the configuration

    terraform apply
    
  6. Connect to instance via ssh

    ssh [email protected]
    
    # user = form line `metadata` secion
    # ip = from `ip` variable output
    # Example
    # ssh [email protected]
    

Clojure

Why Clojure

Functional, dynamic type language. Dialect of Lisp, with Lisp the code-as-data philosophy and a powerful macro system. Not very popular, but I like it. Also working with a various solution is always fun.

Let’s code - Clojure

  1. Setup Clojure

    1. Install java on Linux

      sudo apt install openjdk-8-jre-headless -y
      java -version
      
    2. Install Clojure on Linux

      curl -O https://download.clojure.org/install/linux-install-1.10.1.536.sh
      chmod +x linux-install-1.10.1.536.sh
      sudo ./linux-install-1.10.1.536.sh
      
    3. Install Leiningen

      wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
      -O /usr/bin/lein
      chmod a+x /usr/bin/lein
      lein
      
  2. Run new project

    lein new app <app-name>
    cd <app-name>
    
    # example
    # lein new app clojure-raw-rest-api
    
  3. Check it works

    cd <app-name>
    lein run
    
    # example
    # cd clojure-raw-rest-api
    
    # output
    ➜  clojure-raw-rest-api lein run
    Hello, World!
    ➜  clojure-raw-rest-api
    
  4. Let’s do TDD so tests first

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    
    ;test/clojure_raw_rest_api/core_test.clj
    ...
    (ns clojure-raw-rest-api.core-test
    (:require [clojure.test :refer :all]
                [clojure-raw-rest-api.core :refer :all]
                [ring.mock.request :as mock]))
    
    (deftest simple-page-test
    (is (= (simple-page (mock/request :get "/"))
            {:status  200
            :headers {"Content-Type" "text/html"}
            :body    "<h1>Hello World</h1>"})))
    
    (deftest app-status-test
    (is (= (app-status (mock/request :get "/status"))
            {:status  200
            :headers {"Content-Type" "text/json"}
            :body    {:status "ok"}})))
    
    (deftest enviroment-name-test
    (is (= (enviroment-name (mock/request :get "/env"))
            {:status  200
            :headers {"Content-Type" "text/json"}
            :body    {:enviroment "dev"}})))
    
    (deftest enviroment-missing-handler
    (is (= (missing-handler (mock/request :get "/test"))
            {:status  404
            :headers {"Content-Type" "text/html"}
            :body {:status "Error, path not found!"}})))
    
  5. Make a basic REST API implementation

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    
    ;src/clojure_raw_rest_api/core.clj
    ...
    (ns clojure-raw-rest-api.core
    (:require [ring.adapter.jetty :as jetty]
        [ring.middleware.params :refer [wrap-params]]
        [ring.middleware.reload :refer [wrap-reload]]
        [ring.middleware.keyword-params :refer [wrap-keyword-params]]
        [ring.middleware.json :refer [wrap-json-params wrap-json-response]]
        [clojure.java.io :as io]
        [clj-http.client :as client])
    (:gen-class))
    
    ; Read enviroment variable
    (def env (or (System/getenv "env") "dev"))
    
    ; Simple page
    (defn simple-page [req]
    {:status  200
    :headers {"Content-Type" "text/html"}
    :body    "<h1>Hello World</h1>"})
    
    ; Return Health Check
    (defn app-status [req]
    {:status  200
    :headers {"Content-Type" "text/json"}
    :body    {:status "ok"}})
    
    ; Return env(env var)
    (defn enviroment-name [req]
    {:status  200
    :headers {"Content-Type" "text/json"}
    :body    {:enviroment env}})
    
    ; Return when no path
    (defn missing-handler [request]
    {:status 404
    :headers {"Content-Type" "text/html"}
    :body {:status "Error, path not found!"}})
    
    (def routes [
        {:methods #{:get} :path "/" :handler simple-page}
        {:methods #{:get} :path "/status" :handler app-status}
        {:methods #{:get} :path "/env" :handler enviroment-name}
    ])
    
    (defn route-match? [request route]
    (and ((:methods route) (:request-method request))
        (= (:path route) (:uri request))))
    
    (defn app [request]
    (let [route (first (filter (partial route-match? request) routes))
            handler (get route :handler missing-handler)]
        (println "app request "
        (:request-method request) (:uri request) (pr-str route))
        (handler request)))
    
    (defn with-middleware [handler]
    (-> handler
        (wrap-reload)
        (wrap-keyword-params)
        (wrap-json-params {})
        (wrap-params {})
        (wrap-json-response {:pretty true})))
    
    (defn -main []
    (let [port (Integer. (or (System/getenv "PORT") 8081))]
        (jetty/run-jetty (with-middleware app) {:port port :join? false})))
    
  6. Ahh and project.clj update

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    
    (defproject clojure-raw-rest-api "1.0.0"
    :license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
                :url "https://www.eclipse.org/legal/epl-2.0/"}
    :dependencies [[org.clojure/clojure "1.10.0"]
        [ring/ring-jetty-adapter "1.4.0"]
        [ring/ring-json "0.4.0"]
        [ring/ring-devel "1.4.0"]
        [clj-http "2.2.0"]
        [ring/ring-mock "0.4.0"]]
    :main ^:skip-aot clojure-raw-rest-api.core
    :target-path "target/%s"
    :profiles {:uberjar {:aot :all}})
    
  7. Then run test

    lein test
    
    # output
    ➜  clojure-raw-rest-api lein test
    
    lein test clojure-raw-rest-api.core-test
    
    Ran 4 tests containing 4 assertions.
    0 failures, 0 errors.
    ➜  clojure-raw-rest-api ✗
    
  8. Create jar

    lein uberjar
    
    # output
    ➜  clojure-raw-rest-api git:(master) ✗ lein uberjar
    Compiling clojure-raw-rest-api.core
    2020-03-19 11:36:40.298:INFO::main: Logging initialized @3998ms
    Created /home/kuba/Desktop/Projekty/clojure-raw-rest-api/target/uberjar/clojure-raw-rest-api-1.0.0.jar
    Created /home/kuba/Desktop/Projekty/clojure-raw-rest-api/target/uberjar/clojure-raw-rest-api-1.0.0-standalone.jar
    
  9. Great now I’m ready to pack it into Docker container

Docker

Why Docker

I want to have a container. At this point, I’m familiar with Docker, however, I need to take a look an another solution like distroless.

Let’s code - Docker

  1. Install docker on VM

    sudo apt install docker.io
    
  2. Add user to correct group

    sudo usermod -aG docker $USER
    
  3. Re-login to apply changes

  4. Check installation

    docker run hello-world
    

Dockerfile

  1. Create Dockerfile

    touch Dockerfile
    
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    
    FROM clojure as builder
    RUN mkdir -p /usr/src/app
    WORKDIR /usr/src/app
    COPY COPY clojure-raw-rest-api/ ./
    RUN lein test
    RUN mv "$(lein uberjar | sed -n 's/^Created \(.*standalone\.jar\)/\1/p')" app-standalone.jar
    
    FROM openjdk:8-jre-alpine
    
    COPY --from=builder /usr/src/app/app-standalone.jar ./
    ENTRYPOINT ["java", "-jar", "app-standalone.jar"]
    
    EXPOSE 8081
    
  2. Test build

    docker build . -t clojure-app
    
  3. Run app with an environment variable

    docker run -p 8081:80 -d -e env="staging" clojure-app
    
  4. Test it

    curl  $(curl -s -H "Metadata-Flavor: Google" http://metadata/computeMetadata/v1/instance/network-interfaces/0/access-configs/0/external-ip):8081/environment
    # {"environment": "staging"}
    
  5. Tag image

    export PROJECT_ID="my-small-gcp-project"
    docker tag clojure-app gcr.io/$PROJECT_ID/clojure-app:1.0.0
    
  6. Copy auth.json from workstation

    scp auth.json [email protected]<ip>:<user_home>
    
    # IP of remote machine
    # place to copy file
    # example
    # scp auth.json [email protected]:/home/kuba
    
  7. Push to Container Registry

    gcloud auth configure-docker
    gcloud auth activate-service-account \
    [email protected] \
    --key-file=auth.json
    
    gcloud auth print-access-token | docker login \
    -u oauth2accesstoken \
    --password-stdin https://gcr.io
    
    docker push gcr.io/$PROJECT_ID/clojure-app:1.0.0
    
  8. Create Cloud Run Service for prod

    gcloud run deploy prod-awesome-clojure-api \
    --platform managed \
    --allow-unauthenticated \
    --region europe-west1 \
    --port 8081 \
    --set-env-vars=env='production' \
    --image gcr.io/$PROJECT_ID/clojure-app:1.0.0
    
  9. Create Cloud Run Service for non-prod

    gcloud run deploy staging-awesome-clojure-api \
    --platform managed \
    --allow-unauthenticated \
    --region europe-west1 \
    --port 8081 \
    --set-env-vars=env='staging' \
    --image gcr.io/$PROJECT_ID/clojure-app:1.0.0
    
  10. Destroy infra that’s important

    terraform destroy
    

GitHub Action - CI/CD base

Why GitHub Action

I like GitHub Action, I enjoy working with it. It just works and is fast, well-documented, free, etc. It’s not a complicated application so I can’t see any reason to use Jenkins.

Let’s code - GitHub Action

  1. Add 3 variables as GitHub Secrets

    1. GCP_SA_EMAIL - service account email: [email protected]$my-small-gcp-project.iam.gserviceaccount.com

    2. GCP_SA_KEY - encoded auth.json

      cat auth.json | base64
      
    3. PROJECT_ID - project name: my-small-gcp-project

  2. Put two file into .github\workflows

    touch master.yml
    touch no-master.yml
    
  3. Push/PR to master - master.yml

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    61
    62
    63
    64
    65
    66
    67
    
    name: Build from master
    
    on:
    push:
        branches:
        - master
    
    env:
        STG_SERVICE: "staging-awesome-clojure-api"
        STG_URL: "https://staging-awesome-clojure-api-jsrwhbaamq-ew.a.run.app"
        PROD_SERVICE: "prod-awesome-clojure-api"
        PROD_URL: "https://prod-awesome-clojure-api-jsrwhbaamq-ew.a.run.app"
        APP_NAME: "clojure-app"
    
    jobs:
    build-the-app:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/[email protected]
        - name: Setup GCP
          uses: GoogleCloudPlatform/github-actions/[email protected]
          with:
            version: '281.0.0'
            service_account_email: ${{ secrets.GCP_SA_EMAIL }}
            service_account_key: ${{ secrets.GCP_SA_KEY }}
            export_default_credentials: true
        - name: auth to CR
          run: gcloud auth configure-docker
        - name: build app
          run: docker build . -t $APP_NAME
        - name: tag app
          run: docker tag $APP_NAME gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
        - name: push image
          run: docker push gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
    
    deploy-to-stg:
        needs: build-the-app
        runs-on: ubuntu-latest
        steps:
        - name: Setup GCP
          uses: GoogleCloudPlatform/github-actions/[email protected]
          with:
            version: '281.0.0'
            service_account_email: ${{ secrets.GCP_SA_EMAIL }}
            service_account_key: ${{ secrets.GCP_SA_KEY }}
            export_default_credentials: true
        - name: update staging
          run: gcloud run deploy $STG_SERVICE --project ${{ secrets.PROJECT_ID }} --platform managed --region europe-west1 --image gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
        - name: Check stg
          run: if [[ ! $(curl -s $STG_URL/status | grep ok) ]]; then exit 1; fi
    
    deploy-to-prod:
        needs: [build-the-app, deploy-to-stg]
        runs-on: ubuntu-latest
        steps:
        - name: Setup GCP
          uses: GoogleCloudPlatform/github-actions/[email protected]
          with:
            version: '281.0.0'
            service_account_email: ${{ secrets.GCP_SA_EMAIL }}
            service_account_key: ${{ secrets.GCP_SA_KEY }}
            export_default_credentials: true
        - name: update prod
          run: gcloud run deploy $PROD_SERVICE --project ${{ secrets.PROJECT_ID }} --platform managed --region europe-west1 --image gcr.io/${{ secrets.PROJECT_ID }}/$APP_NAME:${{ github.sha }}
        - name: Check stg
          run: if [[ ! $(curl -s $PROD_URL/status | grep ok) ]]; then exit 1; fi
        - run: "echo PROD status: ${{ steps.get_prod_status.response }}"
    
  4. Push/PR to not master - no-master.yml

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    
    name: Build from no-master
    
    on:
    push:
        branches:
        - '*'
        - '!master'
    pull_request:
        branches:
        - '*'
    
    jobs:
    build-test-docker:
        runs-on: ubuntu-latest
        steps:
        - uses: actions/[email protected]
        - name: build app
        run: docker build . -t clojure-app
    

Bonus - Telegram

Why Telegram

I use Telegram every day, build are long so I decided to add a notification.

Let’s code - Telegram

  1. Add to master.yaml

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    
    send-notification:
      needs: [build-the-app, deploy-to-stg, deploy-to-prod]
      runs-on: ubuntu-latest
      steps:
      - name: test telegram notification
        uses: appleboy/[email protected]
        with:
          to: ${{ secrets.TELEGRAM_TO }}
          token: ${{ secrets.TELEGRAM_TOKEN }}
          message: Build number ${{ github.run_number }} of ${{ github.repository }} is complete ;)
    
  2. Telegram configuration

    1. type /help

    2. type /newbot

    3. generate bot name like superbot, not unique

    4. generate bot username like super-uniqe-bot, must be unique

    5. get a token /token

    6. save token

    7. subscribe bot

    8. use REST API to received TELEGRAM_TO

      curl -s https://api.telegram.org/bot<token>/getUpdates | jq.
      
      # example URL
      # https://api.telegram.org/bot123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11/getUpdates
      
    9. TELEGRAM_TO is field chat.id - 353824382

  3. Configure notification

    I would like to get some notification after the build. Telegram is a nice tool, and there is already created [GH Action][7].

    1
    2
    3
    4
    5
    6
    
    - name: test telegram notification
      uses: appleboy/[email protected]
      with:
        to: ${{ secrets.TELEGRAM_TO }}
        token: ${{ secrets.TELEGRAM_TOKEN }}
        message: Build number ${{ github.run_number }} of ${{ github.repository }} is complete ;)
    

    That snipped contains two secrets secrets.TELEGRAM_TO and secrets.TELEGRAM_TOKEN. Again I can recommend this docs. But I received this value in the previous section. \

    There are also two context variable github.run_number and github.repository. And again docs are more than enough.

Logging

I use Google Cloud Run, so Google manages all logs from services. In the case of this architecture usage, normal logs collector is an overstatement. just logs

Monitoring and alerts

For this, I decided to use the basic features of GCP.

Uptime Checks

  1. Go to Monitoring -> Create uptime checks

just uptime

Alerting policies

  1. Go to Monitoring -> Create alerting policies

just alert

Summary

That was a nice project. I have the opportunity to learn about IAM and GCP Run. Also, I have a better understanding of GitHub Action pipelines. Clojure looks a bit strange, but testing functional code is sweet. I never write such readable test cases. The source code of the project is here, feel free to use.