anuj, Author at For all your terraform needs https://terraformarchitect.com/author/anuj/ Automation cannot be an afterthought (TM) Wed, 24 Jul 2024 23:11:05 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.5 Workspaces in TerraforM https://terraformarchitect.com/cicd-pipelines/workspaces-in-terraform/?utm_source=rss&utm_medium=rss&utm_campaign=workspaces-in-terraform https://terraformarchitect.com/cicd-pipelines/workspaces-in-terraform/#respond Wed, 24 Jul 2024 23:11:05 +0000 https://terraformarchitect.com/?p=383 workspaces allow you to manage multiple environments or configurations within a single Terraform configuration directory. Each workspace maintains its own state file, enabling you to isolate environments such as development, […]

The post Workspaces in TerraforM appeared first on For all your terraform needs.

]]>
workspaces allow you to manage multiple environments or configurations within a single Terraform configuration directory. Each workspace maintains its own state file, enabling you to isolate environments such as development, staging, and production.

Key Concepts

  1. Default Workspace: When you initialize a Terraform project, a default workspace is created. This is where Terraform commands operate if no other workspace is specified.
  2. Custom Workspaces: You can create additional workspaces to manage separate states for different environments or configurations.

Creating and Switching Workspaces

  • Create a Workspace:
    sh

    terraform workspace new <workspace_name>
  • List Workspaces:
    sh

    terraform workspace list
  • Switch to a Workspace:
    sh

    terraform workspace select <workspace_name>
  • Delete a Workspace:
    sh

    terraform workspace delete <workspace_name>

Example Use Cases

Example 1: Managing Multiple Environments

Suppose you have a Terraform configuration that sets up infrastructure for a web application. You want to manage separate environments (development, staging, production) using workspaces.

  1. Create Workspaces:
    sh

    terraform workspace new development
    terraform workspace new staging
    terraform workspace new production
  2. Configure Variables: Use workspace-specific variable files or conditionals within your configuration files to manage environment-specific settings.
    hcl

    variable "environment" {
    description = "The environment for this configuration"
    type = string
    }

    locals {
    environment = terraform.workspace
    }

    resource "aws_instance" "web" {
    count = local.environment == "production" ? 3 : 1
    ami = "ami-0c55b159cbfafe1f0"
    instance_type = "t2.micro"
    tags = {
    Name = "web-server-${local.environment}"
    }
    }

  3. Switch and Apply Configuration:
    sh

    terraform workspace select development
    terraform apply

    terraform workspace select staging
    terraform apply

    terraform workspace select production
    terraform apply

Each workspace maintains a separate state file, ensuring that the resources for each environment are managed independently.

Example 2: Isolating Client Configurations

Imagine you are managing infrastructure for multiple clients, and you want to keep each client’s resources isolated.

  1. Create Workspaces for Clients:
    sh

    terraform workspace new clientA
    terraform workspace new clientB
  2. Client-Specific Variables: Define variables or use conditionals based on the workspace name to configure resources for each client.
    hcl

    variable "client_name" {
    description = "The client for this configuration"
    type = string
    }

    locals {
    client = terraform.workspace
    }

    resource "aws_s3_bucket" "client_bucket" {
    bucket = "myapp-${local.client}-bucket"
    acl = "private"
    }

  3. Switch and Apply Configuration:
    sh

    terraform workspace select clientA
    terraform apply

    terraform workspace select clientB
    terraform apply

This ensures that each client’s resources are managed separately, preventing any accidental overlap or interference.

Summary

Workspaces in Terraform provide a powerful way to manage multiple environments or configurations within a single codebase. By using workspaces, you can maintain separate state files and easily switch between different setups, whether for different environments (like development, staging, production) or different clients. This isolation helps to ensure that changes in one workspace do not affect resources in another, providing a more organized and secure way to manage infrastructure.

The post Workspaces in TerraforM appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/cicd-pipelines/workspaces-in-terraform/feed/ 0
GitHub Actions versus GitLab CI/CD https://terraformarchitect.com/gitlab/github-versus-gitlab/?utm_source=rss&utm_medium=rss&utm_campaign=github-versus-gitlab https://terraformarchitect.com/gitlab/github-versus-gitlab/#respond Thu, 13 Jun 2024 14:31:15 +0000 https://terraformarchitect.com/?p=367 GitHub Actions and GitLab CI/CD are both popular tools used for Continuous Integration (CI) and Continuous Deployment (CD) in software development. They automate the process of building, testing, and deploying […]

The post GitHub Actions versus GitLab CI/CD appeared first on For all your terraform needs.

]]>

GitHub Actions and GitLab CI/CD are both popular tools used for Continuous Integration (CI) and Continuous Deployment (CD) in software development. They automate the process of building, testing, and deploying code. Here’s an explanation of each and a comparison between them.

GitHub Actions

GitHub Actions is a CI/CD and automation tool integrated into GitHub. It allows you to automate workflows directly from your GitHub repository. Workflows are defined in YAML files located in the .github/workflows directory of the repository.

Key Features:

  • Integration: Deep integration with GitHub repositories and services.
  • Workflows: Defined as YAML files, specifying jobs and steps.
  • Jobs and Steps: Jobs can run on different runners (e.g., Linux, Windows, macOS) and consist of steps that execute commands or actions.
  • Actions: Reusable units of code that can be combined to create a workflow. They can be created by the community or custom-made.
  • Marketplace: A vast repository of pre-built actions created by the GitHub community.
  • Matrix Builds: Allows running jobs in parallel with different configurations (e.g., testing against multiple versions of a language).

Example Workflow:

yaml

name: CI

on: [push, pull_request]

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [10, 12, 14]

steps:
- name: Checkout code
uses: actions/checkout@v2

- name: Set up Node.js
uses: actions/setup-node@v2
with:
node-version: ${{ matrix.node-version }}

- name: Install dependencies
run: npm install

- name: Run tests
run: npm test

GitLab CI/CD

GitLab CI/CD is a part of GitLab, a web-based DevOps lifecycle tool that provides a Git repository manager. GitLab CI/CD allows you to define and run CI/CD pipelines using .gitlab-ci.yml files in the repository.

Key Features:

  • Integration: Built into GitLab, providing seamless integration with GitLab repositories.
  • Pipelines: Defined in YAML files, containing stages, jobs, and steps.
  • Stages and Jobs: Pipelines are divided into stages (e.g., build, test, deploy), and each stage contains jobs that run in parallel.
  • Runners: Agents that run the jobs, which can be shared or specific to a project/group.
  • Auto DevOps: Predefined CI/CD configurations for common languages and frameworks.
  • Docker Integration: Strong support for Docker, including building and pushing Docker images.

Example Pipeline:

yaml

stages:
- build
- test
- deploy

build:
stage: build
script:
- npm install
- npm run build

test:
stage: test
script:
- npm test

deploy:
stage: deploy
script:
- echo "Deploying to production..."
only:
- master

Comparison Between GitHub Actions and GitLab CI/CD

Feature GitHub Actions GitLab CI/CD
Integration Deeply integrated with GitHub Deeply integrated with GitLab
Configuration .github/workflows/*.yml files .gitlab-ci.yml file
Runners Hosted by GitHub or self-hosted Hosted by GitLab or self-hosted
Marketplace Extensive marketplace for actions Fewer community-contributed scripts
Workflow Jobs and steps within workflows Stages and jobs within pipelines
Ease of Use Simple setup for GitHub users Simple setup for GitLab users
Flexibility Highly flexible with custom actions Flexible with strong Docker support
Auto DevOps No equivalent feature Built-in Auto DevOps for common setups
Matrix Builds Native support Can be achieved using custom scripts
Artifacts Supported Supported
Security GitHub’s security features GitLab’s security features

Summary

  • GitHub Actions is ideal for projects already hosted on GitHub, providing a rich set of features and a large marketplace of actions.
  • GitLab CI/CD is best for projects hosted on GitLab, offering strong integration with GitLab’s features and extensive support for Docker.

Both tools are powerful and can handle complex CI/CD workflows, but the choice between them often comes down to the platform your repository is hosted on and the specific features you need.

Basic Git commands

  • git init
  • git status
  • git checkout -b <branch-name>
  • git add
  • git commit -m "your message goes here"
  • git remote add
  • git push
  • git pull
  • git clone

The post GitHub Actions versus GitLab CI/CD appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/gitlab/github-versus-gitlab/feed/ 0
Resource Shutdown via terraform https://terraformarchitect.com/quick-recipes-terraform/resource-shutdown-via-terraform/?utm_source=rss&utm_medium=rss&utm_campaign=resource-shutdown-via-terraform https://terraformarchitect.com/quick-recipes-terraform/resource-shutdown-via-terraform/#respond Tue, 22 Aug 2023 13:31:40 +0000 https://terraformarchitect.com/?p=372 Terraform is often used to destroy environments – which keeps the state file consistent, when the next create event needs to occur. However, sometimes, instead of destroying resources, all you […]

The post Resource Shutdown via terraform appeared first on For all your terraform needs.

]]>
Terraform is often used to destroy environments – which keeps the state file consistent, when the next create event needs to occur.

However, sometimes, instead of destroying resources, all you want to do is shut them down or disable them. This is also doable using remote-exec in terraform

Shutting down an EC2 instance on aws

resource "aws_instance" "app" {
  # ...

  provisioner "remote-exec" {
    when    = "destroy"
    inline = [ "systemctl stop service" ]
  }
}

The post Resource Shutdown via terraform appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/quick-recipes-terraform/resource-shutdown-via-terraform/feed/ 0
Terraform for IAM resource creation https://terraformarchitect.com/gcp/terraform-for-iam-resource-creation/?utm_source=rss&utm_medium=rss&utm_campaign=terraform-for-iam-resource-creation https://terraformarchitect.com/gcp/terraform-for-iam-resource-creation/#respond Mon, 21 Aug 2023 14:06:43 +0000 https://terraformarchitect.com/?p=364 Overview Is terraform a suitable tool for creating and managing cloud IAM resources? The short answer is – no. Especially, if you are going to be dealing with a large […]

The post Terraform for IAM resource creation appeared first on For all your terraform needs.

]]>
Overview

Is terraform a suitable tool for creating and managing cloud IAM resources? The short answer is – no. Especially, if you are going to be dealing with a large number of such creation requests.

Time taken to apply terraform for IAM

Check whether the IAM resource already exists – across all buckets that store state. This could take hours. Can you afford to wait for hours for a simple identity creation or a role assignment?

State drift – Console based roles and assignments

Certain IAM actions performed via the console will cause state drift. The next run of your IAM script will not pick these up – and essentially wipe them out.

Of course, if you have IAM creation etc. locked out for console users, you will not face this particular issue.

Summary

For smaller scoped IAM requests – say you have a dozen or so requests to deal with – terraform for IAM may be a workable solution.

 

 

The post Terraform for IAM resource creation appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/gcp/terraform-for-iam-resource-creation/feed/ 0
Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts: Failure sending request: StatusCode=0 — Original Error: context deadline exceeded https://terraformarchitect.com/known-issues-terraform/error-retrieving-contact-for-keyvault-keyvault-baseclientgetcertificatecontacts-failure-sending-request-statuscode0-original-error-context-deadline-exceeded/?utm_source=rss&utm_medium=rss&utm_campaign=error-retrieving-contact-for-keyvault-keyvault-baseclientgetcertificatecontacts-failure-sending-request-statuscode0-original-error-context-deadline-exceeded https://terraformarchitect.com/known-issues-terraform/error-retrieving-contact-for-keyvault-keyvault-baseclientgetcertificatecontacts-failure-sending-request-statuscode0-original-error-context-deadline-exceeded/#comments Mon, 13 Feb 2023 04:32:15 +0000 https://terraformarchitect.com/?p=248 When creating a key vault in Azure using terraform,  you may encounter this error Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts data "azurerm_client_config" "current" {} resource “azurerm_key_vault” “my_key_vault” {  name   […]

The post Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts: Failure sending request: StatusCode=0 — Original Error: context deadline exceeded appeared first on For all your terraform needs.

]]>
When creating a key vault in Azure using terraform,  you may encounter this error
Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts

data "azurerm_client_config" "current" {}

resource “azurerm_key_vault” “my_key_vault” {  name                        = “sample-keyvault-av”

  location                    = azurerm_resource_group.rg_des.location
resource_group_name         = azurerm_resource_group.rg_des.name
tenant_id                   = data.azurerm_client_config.current.tenant_id
sku_name                    = “standard”
#sku_name                    = “premium”
enabled_for_disk_encryption = true
#soft_delete_enabled         = true
purge_protection_enabled    = true

}

}

The issue has been fixed in version 3.3.0 of the azure provider

terraform {
  required_providers {
    azurerm = {
      source  = “hashicorp/azurerm”
      version = “=3.3.0”
    }
  }
}




Need an experienced Cloud Networking or a Cloud Data Protection Expert?  Anuj has successfully delivered over a dozen deployments on each of the public clouds (AWS/GCP/Azure) including several DevSecOps engagements. Set up a time with Anuj Varma.

The post Error: retrieving `contact` for KeyVault: keyvault.BaseClient#GetCertificateContacts: Failure sending request: StatusCode=0 — Original Error: context deadline exceeded appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/known-issues-terraform/error-retrieving-contact-for-keyvault-keyvault-baseclientgetcertificatecontacts-failure-sending-request-statuscode0-original-error-context-deadline-exceeded/feed/ 2
Retrieving Resource IDs in Terraform – The Terraform Data Block https://terraformarchitect.com/terraform-basics/the-terraform-data-block/?utm_source=rss&utm_medium=rss&utm_campaign=the-terraform-data-block https://terraformarchitect.com/terraform-basics/the-terraform-data-block/#respond Mon, 13 Feb 2023 04:23:30 +0000 https://terraformarchitect.com/?p=327 Also read – Terraform external data querying The Terraform Data Block – Retrieving a Project Id for a GCP project data “google_project” “project” { } output "project_number" { value = […]

The post Retrieving Resource IDs in Terraform – The Terraform Data Block appeared first on For all your terraform needs.

]]>
Also read – Terraform external data querying

The Terraform Data Block – Retrieving a Project Id for a GCP project

data “google_project” “project” {
}

output "project_number" {
  value = data.google_project.project.number
}

Terraform Destroy and the Data Block

When executing the terraform destroy command on our Terraform configuration, Terraform does not perform a destroy action on the resource called by the data block. It is a read only block.

The data block is also called when executing the terraform plan command, so your external resource must be present before you execute the terraform plan and terraform apply commands.

Searching for Projects using the terraform data block

One can apply a filter to search for specific projects – e.g. projects about to be deleted

data "google_projects" "my-organization-projs" {
  filter = "parent.id:23232323 lifecycleState:DELETE_REQUESTED"
}

data "google_project" "deletion-candidate" {
  project_id = data.google_projects.my-organization-projs.projects[0].project_id
}

Summary

Instead of hard coding IDs of resources, the terraform data block allows dynamic retrieval of resource IDs. It also allows searching / filtering based a filter inside the data block.





Need an experienced Cloud Networking or a Cloud Data Protection Expert?  Anuj has successfully delivered over a dozen deployments on each of the public clouds (AWS/GCP/Azure) including several DevSecOps engagements. Set up a time with Anuj Varma.

The post Retrieving Resource IDs in Terraform – The Terraform Data Block appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/terraform-basics/the-terraform-data-block/feed/ 0
tfvars versus variables.tf https://terraformarchitect.com/terraform-basics/tfvars-versus-variables-tf/?utm_source=rss&utm_medium=rss&utm_campaign=tfvars-versus-variables-tf https://terraformarchitect.com/terraform-basics/tfvars-versus-variables-tf/#respond Tue, 31 Jan 2023 02:24:13 +0000 https://terraformarchitect.com/?p=322 Prompting a user for Input in Terraform To me, the primary use case is that I want to prompt a user to provide the value for the input variable. In […]

The post tfvars versus variables.tf appeared first on For all your terraform needs.

]]>
Prompting a user for Input in Terraform

To me, the primary use case is that I want to prompt a user to provide the value for the input variable. In that case, I HAVE to use variables.tf.. I simply define the variable and leave it’s value blank (of course, within the resource, this variable has to be a REQUIRED value).

 

The post tfvars versus variables.tf appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/terraform-basics/tfvars-versus-variables-tf/feed/ 0
VS Code does not pick up the latest terraform version https://terraformarchitect.com/known-issues-terraform/vs-code-does-not-pick-up-the-latest-terraform-version/?utm_source=rss&utm_medium=rss&utm_campaign=vs-code-does-not-pick-up-the-latest-terraform-version https://terraformarchitect.com/known-issues-terraform/vs-code-does-not-pick-up-the-latest-terraform-version/#respond Wed, 09 Nov 2022 13:26:30 +0000 https://terraformarchitect.com/?p=299 From a powershell prompt, get the current executable path for terraform using: Get-Command terraform If this is pointing to the older terraform version, simply delete the older terraform.exe file. ReInstall […]

The post VS Code does not pick up the latest terraform version appeared first on For all your terraform needs.

]]>
From a powershell prompt, get the current executable path for terraform using:

Get-Command terraform

If this is pointing to the older terraform version, simply delete the older terraform.exe file.

ReInstall the newer version.  Now, do Get-Command again to see the executable path.

VS Code should also pick up whatever version exe is returned by the Get-Command terraform.

 

 

 

The post VS Code does not pick up the latest terraform version appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/known-issues-terraform/vs-code-does-not-pick-up-the-latest-terraform-version/feed/ 0
VS Code Scripts Error: Cannot be loaded because running scripts is disabled on this system https://terraformarchitect.com/known-issues-terraform/vs-code-scripts-error-cannot-be-loaded-because-running-scripts-is-disabled-on-this-system/?utm_source=rss&utm_medium=rss&utm_campaign=vs-code-scripts-error-cannot-be-loaded-because-running-scripts-is-disabled-on-this-system https://terraformarchitect.com/known-issues-terraform/vs-code-scripts-error-cannot-be-loaded-because-running-scripts-is-disabled-on-this-system/#respond Thu, 03 Nov 2022 14:59:31 +0000 https://terraformarchitect.com/?p=293 VS Code Terminal displays this error Cannot be loaded because running scripts is disabled on this system  If you want to continue using powershell as the terminal, you will need […]

The post VS Code Scripts Error: Cannot be loaded because running scripts is disabled on this system appeared first on For all your terraform needs.

]]>
VS Code Terminal displays this error
Cannot be loaded because running scripts is disabled on this system 
If you want to continue using powershell as the terminal, you will need to tweak settings.json (ctrl shft p and search for settings). Add the following json code to this file.
{
    “terminal.integrated.profiles.windows”: {
        “PowerShell”: {
          “source”: “PowerShell”,
          “icon”: “terminal-powershell”,
          “args”: [“-ExecutionPolicy”, “Bypass”]
        }
      },
      “terminal.integrated.defaultProfile.windows”: “PowerShell”,
}
If you can make do with the windows cmd prompt,  change the VS Code terminal from powerShell to cmd (cmd already has the privileges for running scripts).
  1. Terminal -> New Terminal –> “Default Shell” –> Windows

That’s it.

 

The post VS Code Scripts Error: Cannot be loaded because running scripts is disabled on this system appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/known-issues-terraform/vs-code-scripts-error-cannot-be-loaded-because-running-scripts-is-disabled-on-this-system/feed/ 0
Error – ‘registry.terraform.io/hashicorp/local: there is no package for registry.terraform.io/hashicorp/local cached in .terraform/providers’ https://terraformarchitect.com/gcp/terraform-apply-fails/?utm_source=rss&utm_medium=rss&utm_campaign=terraform-apply-fails https://terraformarchitect.com/gcp/terraform-apply-fails/#respond Wed, 19 Oct 2022 13:11:20 +0000 https://terraformarchitect.com/?p=276 Terraform init or terraform init -upgrade or terraform apply fails with this error message: The installed provider plugins are not consistent with the packages selected in the dependency lock file: […]

The post Error – ‘registry.terraform.io/hashicorp/local: there is no package for registry.terraform.io/hashicorp/local cached in .terraform/providers’ appeared first on For all your terraform needs.

]]>
Terraform init or terraform init -upgrade or terraform apply fails with this error message:

The installed provider plugins are not consistent with the packages selected in the dependency lock file:
│ – registry.terraform.io/hashicorp/azurerm: the cached package for registry.terraform.io/hashicorp/azurerm x.y.x. (in .terraform\providers) does not match any of the checksums recorded in the dependency lock file
│ – registry.terraform.io/hashicorp/random: there is no package for registry.terraform.io/hashicorp/random 3.4.3 cached in .terraform\providers

Resolution

Step 1 – Look for this lock file (terraform.lock.hcl) in your current module. Delete it.

Step 2 – Re Run terraform init -upgrade

Terraform apply fails

The post Error – ‘registry.terraform.io/hashicorp/local: there is no package for registry.terraform.io/hashicorp/local cached in .terraform/providers’ appeared first on For all your terraform needs.

]]>
https://terraformarchitect.com/gcp/terraform-apply-fails/feed/ 0