3sky's notes

Minimal blog about IT
en pl

DNS as code

2024-05-20 8 min read Kuba Wolynko

Welcome

Looks like I have 10 days to create 2 blog posts. To be open, it’s another attempt to become a creator, who makes at least 12 posts per year. Also, I decided, to make my write-ups a bit shorter and focused on one particular topic. That is why, today I will show you how to migrate existing Cloudflare configuration into OpenTofu manifests and make it even more cool with Scalr.

For context, I’m the owner of 3 domains, where one is used for my home lab. What does that it mean? I like to access my services without VPN, from anywhere in the world. Hopefully, Cloudflare provides a solution called tunnels. It could be used for exposing private endpoints/services directly on the internet, without setting up static IP(for example my ISP requires an additional fee for that), also we can easily add multiple “data centers”, for example, NUC under our desk, home NAS, or regular server in your parent’s basement.

Import our resources

For importing our DNS settings I was using a tool called cf-terraforming.
As the first step, we need to become more familiar with the Cloudflare name convention., but don’t panic. Essential variables are:

  1. --zone string Target the provided zone ID for the command
  2. --account string Target the provided account ID for the command

The zone is an ID of your domain zone. Let’s assume that if you’re managing example.com and hello.comdomains, every one of them has a dedicated zone ID. Account ID is the same for all zones and represents your account identity.

zones

Great, now we need to generate tokens for our process.

  1. Login to your account
  2. Go to this page and generate the token.
  3. Try to grant the lowest possible permission, ideally one token per zone, per needed resource.

Now with the token stored in buffer, we can try to generate our first DNS entry.

$ cf-terraforming generate -e your_email \ 
    -t your_token \
    -z your_zone_id \
    --resource-type cloudflare_record
FATA[0003] failed to detect provider installation

Wait, what? Provider? Yes, for this operation we already need to have an initialised Tofu project. Let’s then take a quick break, and talk about Tofu and Scalr configuration.

OpenTofu and Scalr

Now all of you know, that IBM acquired Hashicorp, unfortunately when I performing my migration, I have no idea about Hashicorp feature. Also, Terraform Cloud is ugly and unfriendly to use IMHO. That I why I decided to use OpenTofu as a drop-in replacement and try Scalr as one of many Terraform Cloud alternatives.

Ok, now it’s a good moment to talk about the final project structure. Based on my research and experience managing resources in the following structure will be very flexible and easy to use based on my scale and needs.

❯ tree .
.
├── account_a
│   ├── zone_1
│   │   ├── dns
│   │   │   ├── dns.tf
│   │   │   ├── dns.tf.bck
│   │   │   ├── providers.tf
│   │   │   └── vars.tf
│   │   └── tunnels
│   │       ├── outputs.tf
│   │       ├── providers.tf
│   │       ├── tunnels.tf
│   │       └── vars.tf
│   ├── zone_2
│   │   └── dns
│   │       ├── dns.tf
│   │       ├── providers.tf
│   │       └── vars.tf
│   └── zone_3
│       └── dns
│           ├── dns.tf
│           ├── providers.tf
│           └── vars.tf
├── LICENSE
├── README.md
├── main.tf
└── vars.tf

9 directories, 18 files

As you can see, I have dedicated vars and providers per resource in the zone, but it gives me full control over my critical at the end part of the infrastructure. Especially in the case of using “a bit like beta” type of resources, for example, tunnels or zero trust modules. However, let’s start from scratch. First our main.tf is very simple:

terraform {
  cloud {
    hostname     = "<hostname>"
    organization = "<org>"

    workspaces {
      name = "<ws>"
    }
  }
}

module "zone_1_dns" {
  source    = "./account_a/zone_1/dns"
  api_token = var.zone_1_token
}

In the beginning, we have Scalr config, which is very easy and similar to any other service provider. Also, I’m using Scalr as Terraform Backend for my service. Here you can find a dedicated post how to configure it. I will do it with the usage of fewer details, with the usage of pictures.

  1. Go to Workspaces and click ‘Create Workspaces’
    workspace
  2. Fill in the name, and add GitHub provider according to your preferences.
  3. Remember to use OpenTofu IaC Platform.
  4. Then create the workspace.

Now that we have our account configured we can fill our main.tf according to description:

The hostname argument refers to our instance of Scalr, the organization refers to the environment within our instance, and a workspace is just a workspace!

Ok, now we need to add our first module.

mkdir -pv ./account_a/zone_1/dns

Then let’s add ./account_a/zone_1/dns/providers.tf file:

terraform {
  required_version = "~> 1.5"
  required_providers {
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "4.28.0"
    }
  }
}

provider "cloudflare" {
  api_token = var.api_token
}

./account_a/zone_1/dns/vars.tf file:

variable "api_token" {
  type        = string
  description = "Cloudflare API token"
  sensitive   = true
}

./vars.tf file:

variable "zone_a_token" {
  type        = string
  description = "Cloudflare API token for zone A"
  sensitive   = true
}

And to avoid unnecessary typing .auto.tfvars

zone_1_token = "secret-token"

After that, we can try to init our project (and plan to make sure that Scalr connection is fine)

$ tofu init
[...]
$ tofu plan
tofu plan
[...]
No changes. Your infrastructure matches the configuration.

OpenTofu has compared your real infrastructure against your configuration and
found no differences, so no changes are needed.
[...]

Great, we’re finally ready to generate our DNS entries.

$ cf-terraforming generate \
    -e your_email \
    -t your_token \
    -z your_zone_id \
    --resource-type cloudflare_record
FATA[0003] failed to read provider schemaexit status 1

Error: Failed to load plugin schemas

Error while loading schemas for plugin components: Failed to obtain provider
schema: Could not load the schema for the provider
registry.terraform.io/cloudflare/cloudflare: failed to instantiate provider
"registry.terraform.io/cloudflare/cloudflare" to obtain schema: unavailable
provider "registry.terraform.io/cloudflare/cloudflare"..

Ah, drop-in replacement right? Yes, but actually no, if terraform is required by another tool. But hey! It’s only a home project.

DNS generation

Small fix, and bum!

$ rm -rf .terraform .terraform.lock.hcl
$ terraform version
Terraform v1.5.7
on darwin_arm64

[...]
$ terraform init
[...]
$ cf-terraforming generate -e your_email -t your_token -z your_zone_id --resource-type cloudflare_record

resource "cloudflare_record" "terraform_managed_resource_16e2c78bf8ed9b72215f56cc7fd" {
  name    = "example.com"
  proxied = true
  ttl     = 1
  type    = "A"
  value   = "23.21.20.120"
  zone_id = "your_zone_id"
}

resource "cloudflare_record" "terraform_managed_resource_74824bf72d4607c154097887d66" {
  name    = "example.com"
  proxied = true
  ttl     = 1
  type    = "A"
  value   = "23.21.20.19"
  zone_id = "your_zone_id"
}

resource "cloudflare_record" "terraform_managed_resource_06a882256f1ed1b393253633d6c1" {
  name    = "www"
  proxied = true
  ttl     = 1
  type    = "CNAME"
  value   = "example.com"
  zone_id = "your_zone_id"
}

In general, we can even redirect command output to file and populate ./account_a/zone_1/dns/dns.tf

$ cf-terraforming generate \
    -e your_email \
    -t your_token \
    -z your_zone_id \
    --resource-type cloudflare_record > ./account_a/zone_1/dns/dns.tf

Fast terraform plan , and please notice to things.

  1. Scalr is still using OpenTofu), even if you used terraform plan command on your workstation.
  2. We’re going to create 3 records, even if they already exist… and that’s the problem we need to resolve.
$ terraform plan
Running plan in Terraform Cloud. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan from running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://example.scalr.io/app/prod/my-awesome-cloudflare-config/runs/run-v0oce2teqmu10eo0k

Waiting for the plan to start...

OpenTofu v1.7.1
[...]
Plan: 3 to add, 0 to change, 0 to destroy.
[...]

Importing config

Thankfully we can use import flag of the cf-terraforming tool. Let’s use it then:

cf-terraforming import \
  -e your_email \
  -t your_token \
  -z your_zone_id \
  --resource-type cloudflare_record --modern-import-block >> main.tf

Now we should be able to import our resources right? Not yet. Cf-terraforming does not respect file structure, so our imports have the following format:

import {
  to = cloudflare_record.terraform_managed_resource_16e2c78bf8ed9b72215f56cc7fd
  id = "1fca8e4a1d16a9cdccc3f9f30ebb6317/06a882256f1ed1b39322f7453633d6c1"
}

Unfortunately, this will produce an error:

 terraform plan
[...]
------------------------------------------------------------------------

╷
│ Error: Configuration for import target does not exist
│
│   on main.tf line 16:
│   16: import {
│
│ The configuration for the given import
│ cloudflare_record.terraform_managed_resource_16e2c78bf8ed9b72215f56cc7fd
│ does not exist. All target instances must have an associated configuration
│ to be imported.

And that is very easy to solve, in our main.tf file just add a module.<module_alis_> prefix. (It could be painful in case of big import).

import {
  to = module.zone_1_dns.cloudflare_record.terraform_managed_resource_16e2c78bf8ed9b72215f56cc7fd
  id = "1fca8e4a1d16a9cdccc3f9f30ebb6317/16e2c78bfd13568ed9b72215f56cc7fd"
}

Now we can test our manifests by executing terraform plan:

$ terraform plan
[...]
Plan: 3 to import, 0 to add, 0 to change, 0 to destroy.
[...]

Now stop! Do not apply by hand. We have Scalr for it, right? The only thing you need to do is push your code to the repository(and add a token as a variable in the Scalr console). After push you should see something like this in your web console:

plan

Final notes

  1. As we planned first, the pushing code does nothing, and does not populate the state, but further declaration will do as well as polish the state.
  2. You can migrate to OpenTofu again, as soon as you migrate all Cloudflare blocks from Cloudflare’s API. 
  3. Not all objects are supported by cf-terraforming tool, for example, tunnels. You can check supported resources here
  4. You can visit this repo template on GitHub.

Summary

Uff, that was an interesting adventure, right? As you can see, Scalr is just fine and provides all the features I can expect from my “remote backend” service. OpenTofu in case of regular projects can act as a drop-in Terraform replacement, at least for today. Cloudflare is great, even if you’re on the free tier, and you’re the product.

CloudFlare Homelab OpenTofu Scalr
© 2024 by Lednerb
Bilberry Hugo Theme