Enforce fine-grained policy control across your data infrastructure with Open Policy Agent and Terraform

Use Open Policy Agent, an open source project and Terraform to control how your organization creates resources in development and production

Subscribe to RSS

In this tutorial, you'll use Open Policy Agent (OPA) to enforce fine-grained policy control across development and production environments for Terraform deployments. This tutorial assumes that you are already using Terraform.

The challenge

Rapu started at Crab Inc. as a Junior DevOps Engineer. He is shadowing a senior engineer on the team to learn how the team deploys PostgreSQL®, Aiven for Caching, Apache Kafka®, and other data-related services across development and production environments.

The development team is based in Montreal, Canada and they should only create cloud resources in the Google Cloud Montreal region. However, to ensure high availability for the company's North American customers, the production environment supports multiple AWS cloud regions in the US East location. Previously, there were no guardrails in place and Rapu deployed to cloud regions where he wasn't supposed to deploy.

These are the specific regions for Prod and Dev.

Prod: aws-us-east1
Dev: google-northamerica-northeast1

Your goal is to help Rapu enforce these policies so that resources don't get created in the wrong cloud or region.

Prerequisites

The concept of the tutorial is agnostic to the Terraform provider you choose. For the sake of a demo, I'll choose Aiven Terraform Provider. Aiven provides highly-available and scalable data infrastructure based on open-source technologies. For this tutorial, you'll create a free Aiven account and an Aiven authentication token.

Install OPA and Terraform.

Installing jq is optional.

The Story

In this section, you'll help Rapu create Terraform files that describe an Aiven for Caching resource. This can be any cloud resource for your use case and Aiven for Caching is used as an example. Besides the Aiven for Caching resource in the services.tf file, create a provider.tf file for the provider and version details. You'll also create a variables.tf to define the required variables for Aiven Terraform Provider.

Our protagonist Rapu will learn how to decouple and enforce policies using some common tools. When decoupling policies using Open Policy Agent, the structure is pretty consistent no matter the tool or service.

There is a tool/service, in this case we will be using Terraform. This tool will generate some data that will be used as Input for our decision. The input file will be sent to OPA to be compared against the Policy(written in Rego) and any additional Data.

As an added bonus Rapu will learn how to write Unit tests for his policies, which is part of clean code and best practicies.

Here's a high-level overview of the system:

Here's a detailed version of the same system:

In this tutorial, you run terraform and opa commands manually and from your local machine. The above diagram shows how these tools can be used in an automated way. For example, a CI/CD pipeline can deploy cloud resources using Terraform if the OPA policy allows. A deny from OPA might result in a Slack or email notification to the developer.

Set up Terraform files

In an empty directory, create these three files:

provider.tf file:

terraform {
  required_providers {
    aiven = {
      source  = "aiven/aiven"
      version = "~> 4.1.0"
    }
  }
}
provider "aiven" {
  api_token = var.aiven_api_token
}

variables.tf file:

variable "aiven_api_token" {
  description = "Aiven console API token"
  type        = string
}
variable "project_name" {
  description = "Aiven console project name"
  type        = string
}

service.tf file:

resource "aiven_redis" "redis-demo" {
    project = var.project_name # Find your Aiven project name from top-left of Aiven console
    plan = "hobbyist" # For this exercise, the hobbyist plan will do
    service_name = "redis-demo" # Choose any service name
    cloud_name = "google-northamerica-northeast1" # Choose any cloud region from https://aiven.io/docs/platform/reference/list_of_clouds
}

To set the values for the environment variables like aiven_api_token or project_name, you can either use TF_VAR_name, use variables on the command line, or use a variable definition file.

Prepare the Terraform manifest for OPA

Before executing your Terraform manifest with OPA, it's important to add two environment variables so that Terraform client knows about two variables that Aiven Terraform Provider requires. Please replace the placeholder values with the actual values for your Aiven API token and Aiven project name.


export TF_VAR_aiven_api_token=YOUR_AIVEN_API_TOKEN

export TF_VAT_project_name=YOUR_AIVEN_PROJECT_NAME

Now initialize this directory with the terraform init command and ask Terraform to calculate what changes it will make and store the output in plan.binary.

terraform init
terraform plan --out tfplan.binary

Use the command terraform show to convert the Terraform plan into JSON so that OPA can read the plan.

terraform show -json tfplan.binary > tfplan.json

For improved readability you can pipe the output through jq before saving the file.

terraform show -json tfplan.binary | jq > tfplan.json

Here is a sample output of tfplan.json:


{
    "format_version": "1.1",
    "terraform_version": "1.2.8",
    "variables": {
        "aiven_api_token": {
            "value": "3Xi1J+E0G3vo0fwr2vWMLl0XgHwRyA6pCX8C6rQQZVhoyFfz9WAMreaGZPAI+jRUGWgtslQKQtIZTICCDZlZkQn3sRYHGBcAxgXqoiT3l9cYVbVvyPNSVPGHrSvBhSCXIYgWX3AXkOG/kQiJ1r0CZn0Y0gK/pRyiti6dImIzyEsZWja9FZk+mV/M/6BAZMKpa/EokkKUj4puMpUX4B3//slU9yUdicr2wCe/uyx53K64rU/OWZYCbqfTI6QcsjZc1wd8/a+0aLsv651qZwxmgTAenmj0JC5tXWD+Dx89NaiZcUdGxhyg58ZYfYh6U3YDm5S/ovDcvq9m/ffMKbb2Sut2vVELPO1l6AA70U1besBR0dE="
        },
        "project_name": {
            "value": "devrel-dewan"
        }
    },
    "planned_values": {
        "root_module": {
            "resources": [
                {
                    "address": "aiven_redis.redis-demo",
                    "mode": "managed",
                    "type": "aiven_redis",
                    "name": "redis-demo",
                    "provider_name": "registry.terraform.io/aiven/aiven",
                    "schema_version": 0,
                    "values": {
                        "additional_disk_space": null,
                        "cloud_name": "google-northamerica-northeast1",
                        "disk_space": null,
                        "maintenance_window_dow": null,
                        "maintenance_window_time": null,
                        "plan": "hobbyist",
                        "project": "devrel-dewan",
                        "project_vpc_id": null,
                        "redis_user_config": [],
                        "service_integrations": [],
                        "service_name": "redis-demo",
                        "service_type": "redis",
                        "static_ips": null,
                        "tag": [],
                        "termination_protection": null,
                        "timeouts": null
                    },
                    "sensitive_values": {
                        "components": [],
                        "redis": [],
                        "redis_user_config": [],
                        "service_integrations": [],
                        "tag": []
                    }
                }
            ]
        }
    },
    "resource_changes": [
        {
            "address": "aiven_redis.redis-demo",
            "mode": "managed",
            "type": "aiven_redis",
            "name": "redis-demo",
            "provider_name": "registry.terraform.io/aiven/aiven",
            "change": {
                "actions": [
                    "create"
                ],
                "before": null,
                "after": {
                    "additional_disk_space": null,
                    "cloud_name": "google-northamerica-northeast1",
                    "disk_space": null,
                    "maintenance_window_dow": null,
                    "maintenance_window_time": null,
                    "plan": "hobbyist",
                    "project": "devrel-dewan",
                    "project_vpc_id": null,
                    "redis_user_config": [],
                    "service_integrations": [],
                    "service_name": "redis-demo",
                    "service_type": "redis",
                    "static_ips": null,
                    "tag": [],
                    "termination_protection": null,
                    "timeouts": null
                },
                "after_unknown": {
                    "components": true,
                    "disk_space_cap": true,
                    "disk_space_default": true,
                    "disk_space_step": true,
                    "disk_space_used": true,
                    "id": true,
                    "redis": true,
                    "redis_user_config": [],
                    "service_host": true,
                    "service_integrations": [],
                    "service_password": true,
                    "service_port": true,
                    "service_uri": true,
                    "service_username": true,
                    "state": true,
                    "tag": []
                },
                "before_sensitive": false,
                "after_sensitive": {
                    "components": [],
                    "redis": [],
                    "redis_user_config": [],
                    "service_integrations": [],
                    "service_password": true,
                    "service_uri": true,
                    "tag": []
                }
            }
        }
    ],
    "configuration": {
        "provider_config": {
            "aiven": {
                "name": "aiven",
                "full_name": "registry.terraform.io/aiven/aiven",
                "version_constraint": "~> 4.1.0",
                "expressions": {
                    "api_token": {
                        "references": [
                            "var.aiven_api_token"
                        ]
                    }
                }
            }
        },
        "root_module": {
            "resources": [
                {
                    "address": "aiven_redis.redis-demo",
                    "mode": "managed",
                    "type": "aiven_redis",
                    "name": "redis-demo",
                    "provider_config_key": "aiven",
                    "expressions": {
                        "cloud_name": {
                            "constant_value": "google-northamerica-northeast1"
                        },
                        "plan": {
                            "constant_value": "hobbyist"
                        },
                        "project": {
                            "constant_value": "devrel-dewan"
                        },
                        "service_name": {
                            "constant_value": "redis-demo"
                        }
                    },
                    "schema_version": 0
                }
            ],
            "variables": {
                "aiven_api_token": {
                    "description": "Aiven console API token"
                },
                "project_name": {
                    "description": "Aiven console project name"
                }
            }
        }
    }
}

The api token in the above example is invalid and is shown as an example only.

Write OPA policies in Rego

OPA policies are written in Rego. The following Rego checks if Rapu can deploy to a development environment or a production environment based on the type of Terraform resource and the cloud region they choose.

In the same folder, create a sub-folder called policy and create a file within called terraform.rego. Add the following code to that file:

terraform.rego file:

package terraform.analysis
import input as tfplan
import future.keywords
dev_env_cloud_prefix := "google-northamerica-northeast1"
prod_env_cloud_prefix := "aws-us-east"
resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}
default allow_dev_deployment := false
default allow_prod_deployment := false
allow_dev_deployment if {
	some resource in tfplan.planned_values.root_module.resources
	resource.type in resource_types
	startswith(resource.values.cloud_name, dev_env_cloud_prefix)
}
allow_prod_deployment if {
	some resource in tfplan.planned_values.root_module.resources
	resource.type in resource_types
	startswith(resource.values.cloud_name, prod_env_cloud_prefix)
}

Let's analyze this file. Crab Inc. uses PostgreSQL for their relational database, Aiven for Caching, OpenSearch® for search and analytics, and Apache Kafka as the central message bus. Therefore, the resource_types field limits the use to these four resources. Crab Inc. allows developers to deploy in the Montreal, Canada region only. The dev_env_cloud_prefix field takes care of that requirement. Similarly, production deployments are allowed at any one of Aiven's supported AWS cloud regions in the US East coast which the prod_env_cloud_prefix field takes care of.

Execute the following command from the main directory to find out if OPA allows the Terraform deployment to go through:

./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json

With the current Terraform service definition, the output of the above command is:

{
  "result": [
    {
      "path": "tfplan.json",
      "result": false
    }
  ]
}

If you have jq installed on your machine, you can find the exact result with:

./opa exec --decision terraform/analysis/allow_prod_deployment --bundle policy/ tfplan.json | jq '.result[0].result'

The opa exec command is taking in the tfplan.json as an input and validating this against the policy we defined in the allow_prod_deployment section under policy/terraform.rego file. terraform/analysis is denoting the package name in that Rego.

Let's make a change in the services.tf file and change the cloud_name field to aws-us-east1. Now if you repeat the previous steps to create the tfplan.json and run opa exec command, the output should be true.

hint:

terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json | jq '.result[0].result'

Create data block in Rego

Rapu has done a great job implementing his first policy. However, typically data isn't hard coded in the policy. So let's rewrite the current policy and create some news ones.

Create the following data.json file in the policy folder this should be right next to our rego file:

Filename data.json:

{
    "team": "devrel",
    "app": "crab_cast",
    "dev": {
        "cloud": "google-northamerica-northeast1"
    },
    "prod": {
        "cloud": "aws-us-east1"
    }
}

Now that we have a data file, we can go back and update our Rego policy.

file name terraform.rego:

package terraform.analysis
import input as tfplan
import future.keywords
# notice we removed the hard coded variables
resource_types := {"aiven_kafka", "aiven_pg", "aiven_opensearch", "aiven_redis"}
default allow_dev_deployment := false
default allow_prod_deployment := false
allow_dev_deployment if {
	some resource in tfplan.planned_values.root_module.resources
	resource.type in resource_types
	startswith(resource.values.cloud_name, data.dev.cloud)  # referencing the new data block
}
allow_prod_deployment if {
	some resource in tfplan.planned_values.root_module.resources
	resource.type in resource_types
	startswith(resource.values.cloud_name, data.prod.cloud)  # referencing the new data block
}

Now we can rerun the policy check. Remember we added the data file to our policy folder so OPA should be aware of the new data.

terraform plan --out tfplan.binary
terraform show -json tfplan.binary | jq > tfplan.json
opa exec --decision terraform/analysis/allow_prod_deployment --bundle ./policy tfplan.json

Unit testing in Rego

Now that we have a few policies in place, we are going to add unit tests to ensure our policies are good before we enforce them in production.

Create a rego file for our tests.

file policy/test_terraform.rego:

package terraform.test_analysis
import data.terraform.analysis
test_allow_dev_deployment {
	analysis.allow_dev_deployment with input as {"planned_values": {"root_module": {"resources": [{
		"address": "aiven_redis.redis-demo",
		"mode": "managed",
		"type": "aiven_redis",
		"name": "redis-demo",
		"provider_name": "registry.terraform.io/aiven/aiven",
		"schema_version": 1,
		"values": {
			"cloud_name": "google-northamerica-northeast1",
			"plan": "hobbyist",
			"project": "devrel-dewan",
			"service_name": "redis-demo",
			"service_type": "redis",
		},
	}]}}}
}
test_not_allow_prod_deployment {
	not analysis.allow_prod_deployment with input as {"planned_values": {"root_module": {"resources": [{
		"address": "aiven_redis.redis-demo",
		"mode": "managed",
		"type": "aiven_redis",
		"name": "redis-demo",
		"provider_name": "registry.terraform.io/aiven/aiven",
		"schema_version": 1,
		"values": {
			"cloud_name": "google-northamerica-northeast1",
			"plan": "hobbyist",
			"project": "devrel-dewan",
			"service_name": "redis-demo",
			"service_type": "redis",
		},
	}]}}}
}

With our testing file in place let's run the tests. In this command we are calling the OPA binary, with the subcommand test on the target folder policy.

opa test policy

Some homework for you

Now that you have learned about writing and testing OPA policies for your data infrastructure in place, please write the following two policies and two unit tests:

Policies to add

project must start with team name
service_name must include app name

Unit tests to add

test that Aiven project name must contain team name
test that Aiven service name must contain app name

If you need a hint or two, take a look at the solutions.

Great job helping Rapu succeed

Let's look at all the things Rapu has accomplished on his first project at Crab, Inc.

Created an Aiven Terraform file
Converted the Terraform plan into binary
Converted the binary output into JSON
Created a Rego policy to verify the resource configuration
Tested our Rego policy on our local CLI
Cleaned up our Rego policy by moving hard coded data to a data file
Wrote addition policies to very more specifics of the resources
Added unit tests for each of our policies

Thanks for spending the time learning with us today. Here are some additional resources to help you learn about the tools in this tutorial.

Aiven Docs
Terraform Docs
Rego Playground: The Rego playground is available to interactively test your rego policies
Open Policy Agent Docs

If you have any question on this tutorial, please check out the FAQ page, ask on Aiven community forum, or raise an issue.

Table of contents

The challenge
Prerequisites
The Story
Set up Terraform files
Prepare the Terraform manifest for OPA
Write OPA policies in Rego
Create data block in Rego
Unit testing in Rego
Some homework for you
Policies to add
Unit tests to add
Great job helping Rapu succeed