Terraform — Remote Backend with S3 and State Locking with DynamoDB

In this article, We are discussing how can we store state files in remote locations and state locking using the DynamoDB table.

What is Remote State?

By default, Terraform stores state files locally with the file named terraform.tfstate. When working with Terraform in a team, the use of a local file makes Terraform state management complicated because each time when users apply infrastructure changes they always have the latest state data and make sure that nobody else runs Terraform at the same time.

With remote state, Terraform writes the state data to a remote location, which can then be shared between all team members.

What is State Locking?

If supported by your backend, Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state.

State locking happens automatically on all operations that could write state. You won’t see any message that it is happening. If state locking fails, Terraform will not continue. You can disable state locking for most commands with the -lock flag but it is not recommended.

Why Terraform State Locking is important?

It prevents Terraform state file terraform.tfstate from accidental updates by putting a lock on file so the current update can be finished before processing the new change. The feature of Terraform state locking is supported by AWS S3 and Dynamo DB.

Let’s Start!

Prerequisites

• We require AWS IAM API keys (access and secret keys) with full access to create/delete the S3 bucket, DynamoDB table as well as EC2 instances.
• Terraform should be installed on the machine. If Terraform does not exist you can download and install it from here.

The project directory structure that we follow for this demo

We have 2 layer directory structure here

1. The first layer defines code resources for our remote backend(S3 bucket) and state-locking(DynamoDB table) mechanism.
2. The second layer defines application architecture resources and the configuration of the remote backend and state-locking resource that is created in the first layer.

Let’s start with the first layer (S3 bucket and DynamoDB table) so the project architecture that I show above is a standard project structure as it has the following files

• provider.tf: Provider configurations should be declared in any .tf file. I would recommend either putting them in the main.tf , creating a providers.tf file specifically for the Providers and nothing else, or alternatively a versions.tf file that would include the required_providers block and specify the Terraform version.

• variables.tf: Terraform variables allow you to write configuration that is flexible and easier to re-use.

• output.tf: Terraform outputs let you share data between Terraform configurations and with other tools and automation. Outputs are the only way to share data from a child module to your configuration's root module.

• terraform.tfvars: Tfvars files allow us to manage variable assignments systematically in a file with the extension .tfvars or .tfvars.json. Even though there are numerous ways to manage variables in Terraform, tfvars files are the best and most common way to do so due to their simplicity and effectiveness.

• main.tf (S3.tf and Dynamodb.tf): Resources are the most important element in the Terraform language. Each resource block describes one or more infrastructure objects, such as virtual networks, compute instances, or higher-level components such as DNS records.

It will create S3 Bucket and DynamoDB table to our remote backend and state locking.

Note: here var.bucket_name and var.table_name both variables value comes from terraform.tfvars file

So our core resources are ready, just run the following commands to create all resources.

terraform init

terraform plan

terraform apply

let's configure the remote backend and state-locking with S3 and DynamoDB table that we have created in the above step.

• terraform.tf: contains terraform configuration information like terraform backend and state-locking, required provider/version info, and terraform version info.

emote-backend and state-locking with S3 and DynamoDB table

I have created the Terraform project and pushed it to the GitHub repository that contains a code for backend resource creation and configuration clone and follows the below steps.

GitHub — LazziBear/tf-remote-backend-state-locking: This is the terraform script that demonstrates…

Note: If you have already have infrastructure ready and adding remote-backend and state locking may be it return following error

Solution: Run terraform init -migrate-state to migrate your state to the remote-backend.

Nice, our remote backend and state locking are configured.

Now, Lets test

For state locking Login to your AWS Account and go to Dynamodb and open created table.

It shows 0 item Count but when you run terraform plan or apply it shows your state-locking entry

and if two or more developers try to run a terraform plan to apply the second one get an error like

Thank you for reading, if you have anything to add please send a response or add a note!

Prashant Bhatasana