Cloud-init with DigitalOcean

2021-09-20 4 min read 3sky

Welcome

Another build something yourself article. The last one was focused on Kubernetes. Now I will try to build infrastructure for a bit different side-project. I always want to learn SaltStack. To be honest, I’m a huge fan of Ansible which is boring, but works. However, sometimes I have a problem with keeping my developers account synced. Also adding multiple users and track their settings is hard without states. So I decided to get into the new toolbox. Additionally cloud-init with DigitalOcean API sounds like fun.

Components

No SaltStack for now, just an infra setup.

Setting the environment

To build our infrastructure we don’t need much stuff. The first one is Access Token. DigitalOcean has great docs so, I just refer to it. When we get our token, the next step is getting the region’s name for our server. That can be done via API, doctl, or just google search. However I very like their(DO) CLI tool, also there is ZSH autocompletion(yay!).

$ doctl compute region list
Slug    Name               Available
nyc1    New York 1         true
sfo1    San Francisco 1    false
nyc2    New York 2         false
ams2    Amsterdam 2        false
sgp1    Singapore 1        true
lon1    London 1           true
nyc3    New York 3         true
ams3    Amsterdam 3        true
fra1    Frankfurt 1        true
tor1    Toronto 1          true
sfo2    San Francisco 2    false
blr1    Bangalore 1        true
sfo3    San Francisco 3    true

We’re interested in slug variable. In my case it will be fra1, I’m based in Poland. Fortunately, you can choose whatever you want. Setup is ready, so let’s “code” something.

Cloud-init

Why cloud-init? Because it’s a popular init script mechanism. It can be used in AWS as well in DO, or OpenStack. It’s quite simple, started almost at boot time and what is important doesn’t require any ready VM image. To be clear, immutable infrastructure is awesome. Sometimes if we just want a cheap solution(no storage cost), or boot time is not an important factor, then init-scripts are very handy.

File content

#cloud-config
# first create a user with
# ssh key, permission, etc
users:
  - name: kuba
    ssh-authorized-keys:
      - [email protected] AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29tAAAAILjtneTPZiRcjxcM2xiAvvk1BvZGyBibtTx0i+szJutMAAAABHNzaDo= jakub.wolynko@yubi
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    groups: sudo
    shell: /bin/bash

# then upgrade base image
package_update: true
package_upgrade: true
package_reboot_if_required: true

# add security to ssh config
write_files:
  - path: /etc/ssh/sshd_config
    content: |
         Port 31
         Protocol 2
         HostKey /etc/ssh/ssh_host_dsa_key
         HostKey /etc/ssh/ssh_host_ecdsa_key
         HostKey /etc/ssh/ssh_host_ed25519_key
         UsePrivilegeSeparation yes
         KeyRegenerationInterval 3600
         ServerKeyBits 1024
         SyslogFacility AUTH
         LogLevel INFO
         LoginGraceTime 120
         PermitRootLogin no
         StrictModes yes
         RSAAuthentication yes
         PubkeyAuthentication yes
         IgnoreRhosts yes
         RhostsRSAAuthentication no
         HostbasedAuthentication no
         PermitEmptyPasswords no
         ChallengeResponseAuthentication no
         X11Forwarding no
         X11DisplayOffset 10
         PrintMotd no
         PrintLastLog yes
         TCPKeepAlive yes
         AcceptEnv LANG LC_*
         UsePAM yes
         AllowUsers kuba

# install salt-master and salt-ssh 
runcmd:
  - systemctl restart ssh
  - curl -fsSL -o /usr/share/keyrings/salt-archive-keyring.gpg https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest/salt-archive-keyring.gpg
  - echo "deb [signed-by=/usr/share/keyrings/salt-archive-keyring.gpg arch=amd64] https://repo.saltproject.io/py3/ubuntu/20.04/amd64/latest focal main" | tee /etc/apt/sources.list.d/salt.list
  - apt-get update -y
  - apt-get install salt-ssh salt-master -y

Terraformation

Terraform is industry standard. We can use DigitalOcean API, GCP API, or even CloudFlare API. Some people prefer CloudFormation, some Pulumi or Azure ARM. Personally I like terraform. It is stable and works. The bucket mechanism is helpful in case of working with teammates. So why not use it?

Content

To be honest, the project is really small. Everything goes to one file called main.tf.

terraform {

  required_version = ">= 1.0.6"

  required_providers {
    digitalocean = {
      source = "digitalocean/digitalocean"
      version = "~> 2.0"
    }
  }
}

variable "do_token" {
  type = string
}

provider "digitalocean" {
  token = var.do_token
}

resource "digitalocean_droplet" "droplet" {
  image      = "ubuntu-20-04-x64"
  name       = "saltmaster"
  region     = "FRA1"
  size       = "s-1vcpu-1gb"
  monitoring = true
  tags       = ["salt", "edu", "terrafom"]
  user_data  = "${file("cloud-init.yaml")}"
}

output "public_ip" {
  description = "Droplet public IP"
  value       = digitalocean_droplet.droplet.ipv4_address
}

Also, we need to put do_token somewhere. For example in a file called sample.tfvars or in CLI variable. Both solutions have some props and cons. The file is easy to use, but it’s important to keep it in your .gitignore.

One command leter

At the end of the day we just need to run our terraform project with:

terraform init && terraform plan -var-file=sample.tfvar

If everything looks fine we can build our infrastructure.

terrafom apply -var-file=sample.tfvars -auto-approve

ssh kuba@$(tf output -raw public_ip) -p 31

Now you can check the cloud-init logs. I can assume that some updates will be still running.

sudo tail -f /var/logs/cloud-init.log

Summary

There is no repository for the article. The project structure is simple.

.
├── cloud-init.yaml
├── main.tf
└── sample.tfvars

The most interesting tool here is cloud-init which could be very helpful in many, many cases. We can use it in cloud environment provisioning, small internal projects, or even rebuilding the DC fleet on OpenStack. In my opinion, is important for every DevOps person to be at least familiar with this concept. Even if you will never use it in production. Terraform is a terrafrom, nothing else. DigitalOcean ocean is simple, affordable, and stable. What next? I will try to focus on SaltStack.