Aiven databases and Terraform for fun and profit

In the dawn of prehistory, infrastructure teams managed their systems by physically logging into each machine. Nowadays, with Terraform, infra teams can make changes to entire datacenters at once with a few lines of configuration code, in a versionable, reproducible, and predictable way.

For example, you can use the Aiven's CLI to deploy a PostgreSQL database on GCP:

$ avn service create --service-type pg --plan startup-4 --cloud google-europe-west3 postgresql
Copy to clipboard

And you can achieve the same with Terraform code:

resource "aiven_pg" "postgresql" {
  project                = "my-project-name"
  service_name           = "postgresql"
  cloud_name             = "google-europe-west3"
  plan                   = "startup-4"
}
Copy to clipboard

Luckily, Aiven develops and maintains its own Aiven Terraform Provider. With it, users can manage their entire open source data infrastructure with Terraform.

Today, our challenge is to get a PostgreSQL database up and running. Let's get to it. Here's the high-level process:

1. Configure your Aiven account
2. Set up Terraform
3. Configure the Aiven Provider
4. Give me databases!

Note: Terraform can be very picky about the resources it manages. If you create an Aiven service with Terraform, avoid changing it manually through other tools, like the Aiven Console or CLI.

1. Configure your Aiven account

First, you will need an Aiven API access token to authenticate with Terraform.

Log in to the Aiven platform. If you don't have an account, sign up and take advantage of our free trial. It's worth it, I promise.

Then you can follow our documentation to generate your authentication token.

Also, note down your Project Name (located on the upper left side of the page).

2. Set up Terraform

Go to the Terraform page and download the latest Terraform CLI for your platform – you're going to need version v0.13.0 or newer. You can check your installation by running the following command:

$ terraform version
Terraform v0.15.3
Copy to clipboard

3. Configure the Aiven Provider

Terraform Providers are plugins that connect the power of Terraform with external services and cloud providers. We will now bootstrap our Terraform codebase using the Aiven Terraform Provider. Check the official Aiven documentation for more details on each resource.

Create a new directory for your project and add a file named main.tf with the content below:

variable "aiven_api_token" {
  type = string
}

variable "aiven_project_name" {
  type = string
}

terraform {
  required_providers {
    aiven = {
      source  = "aiven/aiven"
      version = "2.1.12" # check out the latest release in the docs page
    }
  }
}

provider "aiven" {
  api_token = var.aiven_api_token
}

data "aiven_project" "my_project" {
  project = var.aiven_project_name
}
Copy to clipboard

In the two first blocks, we define variables to receive the previously created Aiven access token and Project Name. The next two blocks configure the Provider. Lastly, we use a data source to retrieve all the information regarding the project.

Note: You can also create Aiven Projects with the aiven_project resource.

Create another file named terraform.tfvars to add the Aiven access token and Project Name:

aiven_api_token    = "your_access_token_here"
aiven_project_name = "your_project_name_here"
Copy to clipboard

Finally, initialize Terraform (command output shortened for brevity):

$ terraform init
Copy to clipboard

4. Give me databases!

Alright, enough setup, Let's deploy some databases, starting with Aiven for PostgreSQL.

Beware! Terraform is exceptionally efficient in creating and destroying resources. With databases, every resource removal may result in data loss. In a production environment, read the Terraform plan output carefully before applying it.

First, we will add the aiven_pg resource in the main.tf file:

resource "aiven_pg" "postgresql" {
  project                = data.aiven_project.my_project.project
  service_name           = "postgresql"
  cloud_name             = "google-europe-west3"
  plan                   = "startup-4"

  termination_protection = true

  pg_user_config {
    pg_version = 13
    admin_username = "admin"

    pgbouncer {
      autodb_max_db_connections = 200
    }
  }
}

output "postgresql_service_uri" {
  value     = aiven_pg.postgresql.service_uri
  sensitive = true
}
Copy to clipboard

Here, we have configured the database location, size and parameters. The Aiven website tells you more about clouds, plans and pricing and all the PostgreSQL related configurations can be found in the official Terraform documentation.

Also, the termination_process parameter guarantees that Terraform won't delete the resource until the user sets the flag to false.

At the end, Terraform will output the PostgreSQL connection string.

In your terminal run terraform plan and see what it will create for us:

$ terraform plan
[...]

Terraform will perform the following actions:

  # aiven_pg.postgresql will be created
  + resource "aiven_pg" "postgresql" {
      + cloud_name             = "google-europe-west3"
      + components             = (known after apply)
      + id                     = (known after apply)
      + plan                   = "startup-4"
      + project                = "your_project_name_here"
      + service_host           = (known after apply)
      + service_name           = "postgresql"
      + service_password       = (sensitive value)
      + service_port           = (known after apply)
      + service_type           = (known after apply)
      + service_uri            = (sensitive value)
      + service_username       = (known after apply)
      + state                  = (known after apply)
      + termination_protection = true

      + pg {
          + dbname      = (known after apply)
          + host        = (known after apply)
          + password    = (sensitive value)
          + port        = (known after apply)
          + replica_uri = (sensitive value)
          + sslmode     = (known after apply)
          + uri         = (sensitive value)
          + user        = (known after apply)
        }

      + pg_user_config {
          + admin_username = "admin"
          + pg_version     = "13"

          + pgbouncer {
              + autodb_max_db_connections = "200"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + postgresql_service_uri = (sensitive value)

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now.
Copy to clipboard

The command output tells how the Terraform plan looks like and what changes will be made. Please, for all that is holy, do a sanity check before applying it with the next command:

$ terraform apply
[...]

aiven_pg.postgresql: Creating...
aiven_pg.postgresql: Creation complete after 4m38s [id=my_project/postgresql]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

postgresql_service_uri = <sensitive>
Copy to clipboard

Awesome, our PostgreSQL is up and running. Now let's try to access it with psql using the postgresql_service_uri output:

$ psql "$(terraform output -raw postgresql_service_uri)"
psql (13.2)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

defaultdb=>
Copy to clipboard

This is everything you need to have a production-ready PostgreSQL server at GCP, managed by Aiven and deployed with code. Cool, right?!

Next, read the sequel Observe your PostgreSQL metrics with Terraform, InfluxDB and Grafana!

Wrapping up

I hope you enjoyed this ride with me. Terraform and infrastructure as code can be daunting at first, but once we see all of its benefits, it's hard to move away from it.

Please check Aiven's Terraform GitHub repository to see the whole code and any future updates for it.

If you want to learn more about the tools and services used here, make sure to check the links below:

• An introduction to PostgreSQL
• Introduction to Terraform

If you have any issues with Aiven's Terraform Provider or want to help us improve it, please check the GitHub repository and the official documentation.

--

Not using Aiven services yet? Sign up now for your free trial at https://console.aiven.io/signup!

In the meantime, make sure you follow our changelog and blog RSS feeds or our LinkedIn and Twitter accounts to stay up-to-date with product and feature-related news.