[{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/aws/","section":"Tags","summary":"","title":"Aws","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/devops/","section":"Tags","summary":"","title":"Devops","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/github/","section":"Tags","summary":"","title":"Github","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/github-actions/","section":"Tags","summary":"","title":"Github-Actions","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/github-apps/","section":"Tags","summary":"","title":"Github-Apps","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/go/","section":"Tags","summary":"","title":"Go","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/lambda/","section":"Tags","summary":"","title":"Lambda","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/serverless/","section":"Tags","summary":"","title":"Serverless","type":"tags"},{"content":"This post explains a reference setup for running a GitHub App on AWS Lambda. It is not tied to one use case. The idea is a pattern you can copy: take signed webhooks, prove they came from GitHub, call the API as the app, and return the right HTTP status when GitHub sends the same event again.\nThe code is here: github.com/salsiy/serverless-github-app. It has Terraform for Lambda and a Go handler you can swap out while keeping the rest.\nGo libraries # The handler is plain Go (1.24). These are the main dependencies:\nLibrary What it does here aws-lambda-go Lambda handler and Function URL request type aws-sdk-go-v2 (SSM) Read app ID, key, and webhook secret from Parameter Store ghinstallation GitHub App auth and installation access tokens go-github GitHub REST API (Contents API, repository_dispatch, etc.) viper Parse .github/app-config.yaml in the sample zap Structured logs to CloudWatch Webhook HMAC uses the Go standard library (crypto/hmac, crypto/sha256). There is no extra signing package.\nHow the pieces fit # A GitHub App is its own app on GitHub. It has keys, permissions, and webhooks. When something happens in an org or repo, GitHub POSTs JSON to your URL and signs it with a secret. Your code checks the signature, reads installation from the payload, gets a short-lived token, and uses that token for API calls for that install only.\nIn this repo the URL is a Lambda Function URL with authorization_type = \u0026quot;NONE\u0026quot;. There is no API Gateway and no server that runs all the time. On startup, init() loads the app ID, private key, and webhook secret from SSM. Environment variables hold SSM paths, not the secrets themselves. The middle of the handler is sample code. In a fork you usually keep verify, auth, secrets, and status codes. You replace the business logic.\nflowchart TB subgraph github [GitHub] App[GitHub App] Webhook[Webhook POST] ContentsAPI[Contents API] RestAPI[GitHub REST API] end subgraph aws [AWS] FuncURL[Lambda Function URL] Lambda[Go bootstrap] SSM[SSM Parameter Store] CW[CloudWatch Logs] end App --\u003e Webhook Webhook --\u003e|HTTPS| FuncURL FuncURL --\u003e Lambda Lambda --\u003e|init| SSM Lambda --\u003e|HMAC verify| Lambda Lambda --\u003e|sample| ContentsAPI Lambda --\u003e|sample| RestAPI Lambda --\u003e CW One webhook, step by step # sequenceDiagram participant GH as GitHub participant URL as LambdaFunctionURL participant L as GoHandler participant SSM as SSM participant API as GitHubAPI Note over L,SSM: init on warm container L-\u003e\u003eSSM: app-id, private key, webhook secret GH-\u003e\u003eURL: POST body + x-hub-signature-256 URL-\u003e\u003eL: LambdaFunctionURLRequest L-\u003e\u003eL: HMAC-SHA256 over raw body alt unsupported event L--\u003e\u003eGH: 200 else supported L-\u003e\u003eAPI: installation token + handler L--\u003e\u003eGH: 200 or 500 end GitHub signs the raw body. The handler checks x-hub-signature-256 before it parses JSON. Lambda Function URLs turn header names lowercase. The code must read x-hub-signature-256, not X-Hub-Signature-256. That is why verification often works on your laptop but fails in Lambda.\nThe header looks like sha256= plus hex. The code strips sha256=, hashes the body with the webhook secret, and compares with hmac.Equal. No header returns 400. Bad signature returns 401. Bad JSON returns 400. Events the handler does not support return 200 so GitHub does not retry. Errors in processWebhook (like missing config) return 500 and GitHub will try again.\nFor API calls, ghinstallation wraps the HTTP transport and go-github is the client. Together they use the app ID, installation ID from the payload, and PEM from SSM. The sample uses that client to read .github/app-config.yaml and call the REST API. Change those calls. Keep the client setup.\nStatus codes # What happened HTTP GitHub Event not supported 200 No retry Bad signature or body 400 / 401 No retry Handler failed 500 Retries In the sample, if one repository_dispatch fails, the code logs it and moves on. The webhook can still return 200. That way one bad repo does not make GitHub resend the whole event. Change this if you need all targets to succeed.\nDeploy # Terraform deploys Go on provided.al2023 (arm64), 256 MB, 30 second timeout, plus a Function URL. IAM can only read three SSM parameters. No VPC. Example parameters:\naws ssm put-parameter --name \u0026#34;/dev/github-app/app-id\u0026#34; --value \u0026#34;YOUR_APP_ID\u0026#34; --type \u0026#34;String\u0026#34; aws ssm put-parameter --name \u0026#34;/dev/github-app-private-key\u0026#34; --value file://app.private-key.pem --type SecureString aws ssm put-parameter --name \u0026#34;/dev/github-app-webhook-secret\u0026#34; --value \u0026#34;YOUR_SECRET\u0026#34; --type SecureString Run make deploy. Set the GitHub App webhook URL to the Function URL output. Use the same webhook secret in GitHub and in SSM.\nWhat to change in the repo # File Role app/webhook.go Keep. Signature check. app/main.go Keep. Entry point, init(), status codes. app/github_auth.go Keep. GitHub client per installation. app/config.go Keep. SSM reads. app/webhook_processor.go Replace. Event routing (sample uses release). app/repo_config_loader.go, app/github_dispatch.go Replace. Sample config and API calls. The sample only runs when the payload has a release object. It reads .github/app-config.yaml as an example. You do not have to use that file. Run make test for checks. Use terraform output for the function name when tailing logs.\nWrap-up # A GitHub App on Lambda is mostly the same chores every time. Trust the webhook. Load secrets from SSM. Stay within the installation. Return status codes GitHub understands. This repo does that work once so you can focus on the logic in the middle. Fork it, replace the middle, keep the rest.\n","date":"31 May 2026","externalUrl":null,"permalink":"/posts/tech-logs/serverless-github-apps-aws-reference-architecture/","section":"All Posts","summary":"\u003cp\u003eThis post explains a reference setup for running a \u003ca href=\"https://docs.github.com/en/apps\" target=\"_blank\"\u003eGitHub App\u003c/a\u003e on AWS Lambda. It is not tied to one use case. The idea is a pattern you can copy: take signed webhooks, prove they came from GitHub, call the API as the app, and return the right HTTP status when GitHub sends the same event again.\u003c/p\u003e\n\u003cp\u003eThe code is here: \u003ca href=\"https://github.com/salsiy/serverless-github-app\" target=\"_blank\"\u003egithub.com/salsiy/serverless-github-app\u003c/a\u003e. It has Terraform for Lambda and a Go handler you can swap out while keeping the rest.\u003c/p\u003e","title":"Serverless GitHub Apps on AWS: A Reference Architecture","type":"posts"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/categories/tech-logs/","section":"Categories","summary":"","title":"Tech Logs","type":"categories"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/posts/tech-logs/","section":"All Posts","summary":"","title":"Tech Logs","type":"posts"},{"content":"","date":"31 May 2026","externalUrl":null,"permalink":"/tags/webhooks/","section":"Tags","summary":"","title":"Webhooks","type":"tags"},{"content":"Hi, I’m Siyad (nickname: Ziad)\nWelcome to my personal space where I share stories about my life in India and Poland, along with insights from my tech journey\nExplore my technical articles.\n","date":"31 May 2026","externalUrl":null,"permalink":"/","section":"Welcome","summary":"\u003cp\u003eHi, I’m \u003cstrong\u003eSiyad\u003c/strong\u003e (nickname: \u003cstrong\u003eZiad\u003c/strong\u003e)\u003c/p\u003e\n\u003cp\u003eWelcome to my personal space where I share stories about my life in India and Poland, along with insights from my tech journey\u003c/p\u003e\n\u003cp\u003eExplore my \u003ca href=\"/posts/tech-logs\"\u003etechnical articles\u003c/a\u003e.\u003c/p\u003e","title":"Welcome","type":"page"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/automation/","section":"Tags","summary":"","title":"Automation","type":"tags"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/cicd/","section":"Tags","summary":"","title":"Cicd","type":"tags"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/gitops/","section":"Tags","summary":"","title":"Gitops","type":"tags"},{"content":"This is Part 6, the final installment of my series on Production-Grade Terraform Patterns. In Part 5, I closed the loop on updates. But one question remains: When you merge a PR, who actually runs terraform apply?\nIn valid \u0026ldquo;GitOps\u0026rdquo;, humans should never touch the cloud console. And ideally, they shouldn\u0026rsquo;t even run terraform apply from their laptops.\nThe Evolution of Execution # Stage 1: Crypto-ClickOps (The Dark Ages) # You run terraform apply from your laptop.\nRisk: You have AdministratorAccess keys on your disk. Bug: You forgot to git pull before applying. You just overwrote your colleague\u0026rsquo;s changes. Stage 2: Generic CI (Jenkins/GitHub Actions) # You put terraform apply in a pipeline that runs on merge to main.\nBenefit: Centralized, audited execution. Friction: You only see the failure after you merge. \u0026ldquo;Fixing broken main\u0026rdquo; becomes a daily ritual. This is where many teams land first. It works until PR-time plan review and locking start to matter.\nStage 3: TACOS (Terraform Automation and Collaboration Software) # This is the modern standard. TACOS tools move plan and apply into the Pull Request, so you review infrastructure changes before they touch the cloud.\nThe Workflow # sequenceDiagram participant Dev as Developer participant Git as GitHub PR participant TACOS as TACOS participant Cloud as AWS Dev-\u003e\u003eGit: Open Pull Request Git-\u003e\u003eTACOS: Webhook Event TACOS-\u003e\u003eTACOS: Plan TACOS-\u003e\u003eGit: Comment: \"Plan: 3 to add\" Dev-\u003e\u003eGit: Comment: \"apply\" Git-\u003e\u003eTACOS: Webhook Event TACOS-\u003e\u003eCloud: Apply Changes TACOS-\u003e\u003eGit: Comment: \"Apply Successful\" TACOS-\u003e\u003eGit: Merge PR (Optional) TACOS in the wild # The category includes many products. Below are the ones worth knowing, with official links only.\nPR-native and self-hosted # Atlantis: Open-source PR automation server. OpenTaco: Successor to Digger. Atlantis-style automation on your existing CI runners. Burrito: Kubernetes operator for Terraform PR/MR workflows and drift detection. Orchestration and platform # Terramate: Stacks, orchestration, CI/CD integration, and observability. Managed platforms # HCP Terraform (formerly Terraform Cloud) / Terraform Enterprise: HashiCorp\u0026rsquo;s managed offering. Spacelift: Multi-IaC orchestration platform. Env0: Cloud governance and IaC automation. Scalr: Terraform Cloud alternative with PR-native GitOps. Digger rebranded to OpenTaco. Same category, new name. If you evaluated Digger in the past, start with OpenTaco.\nPick based on team size, existing CI, and whether you want self-hosted or SaaS. I have not deployed any of these in my reference repos. Treat this as a starting point, not a recommendation.\nWhy You Need This # If you have more than 3 engineers, locking is essential. TACOS provide:\nState Locking: Prevents race conditions. Plan Review: The plan output is right there in the PR comment, immutable and searchable. Gatekeeping: You can require \u0026ldquo;1 approval\u0026rdquo; before the apply command is allowed to run. Conclusion # That closes the series. A complete platform looks like this:\nSplit Repos (Part 1) isolate failure domains. Modules (Part 2) provide reusable logic. Release Please (Part 3) versions those modules reliably. Git Tags (Part 4) wire the live repo to versioned modules. Renovate (Part 5) keeps dependencies fresh. TACOS (Part 6) automate the final mile: plan and apply in the PR, with locking and review. You have now graduated from \u0026ldquo;running scripts\u0026rdquo; to building a Product.\n","date":"15 February 2026","externalUrl":null,"permalink":"/posts/tech-logs/part-6-scaling-execution-tacos/","section":"All Posts","summary":"\u003cp\u003eThis is Part 6, the final installment of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. In \u003ca href=\"/posts/tech-logs/part-5-automating-dependency-updates-renovate/\"\u003ePart 5\u003c/a\u003e, I closed the loop on updates. But one question remains: \u003cstrong\u003eWhen you merge a PR, who actually runs \u003ccode\u003eterraform apply\u003c/code\u003e?\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn valid \u0026ldquo;GitOps\u0026rdquo;, humans should never touch the cloud console. And ideally, they shouldn\u0026rsquo;t even run \u003ccode\u003eterraform apply\u003c/code\u003e from their laptops.\u003c/p\u003e\n\n\n\u003ch2 class=\"relative group\"\u003eThe Evolution of Execution \n    \u003cdiv id=\"the-evolution-of-execution\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#the-evolution-of-execution\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\n\n\u003ch3 class=\"relative group\"\u003eStage 1: Crypto-ClickOps (The Dark Ages) \n    \u003cdiv id=\"stage-1-crypto-clickops-the-dark-ages\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#stage-1-crypto-clickops-the-dark-ages\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cp\u003eYou run \u003ccode\u003eterraform apply\u003c/code\u003e from your laptop.\u003c/p\u003e","title":"Part 6: Scaling Execution — The Case for TACOS","type":"posts"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/tacos/","section":"Tags","summary":"","title":"Tacos","type":"tags"},{"content":"","date":"15 February 2026","externalUrl":null,"permalink":"/tags/terraform/","section":"Tags","summary":"","title":"Terraform","type":"tags"},{"content":"This is Part 5 of my series on Production-Grade Terraform Patterns. I have split my repositories, enforced versioning, automated releases, and set up my consumption model.\nIn a traditional setup, upgrading infrastructure is painful. A developer releases vpc-v1.1.0, sends a Slack message, and\u0026hellip; silence. Three months later, Prod is still on vpc-v0.9.0.\nTo build a robust infrastructure, upgrades should be pushed to the consumer, not pulled.\nI achieve this with Renovate Bot.\nThe Workflow # Release: Release Please tags vpc-v1.1.0 in your Modules Repo. Detection: Renovate scans your Live Repo, sees vpc-v1.0.0, and detects the new tag. Proposal: Renovate opens a Pull Request: chore(deps): update module vpc to vpc-v1.1.0. Validation: CI triggers terragrunt plan on that PR. Merge: You review the plan and merge. graph TD Tag[New Tag vpc-v1.1.0 Released] --\u003e|Scanned by| Reno[Renovate Bot] subgraph Live_Repo [Live Infrastructure Repo] Reno --\u003e|Opens PR| PR[PR: Update to vpc-v1.1.0] end subgraph CI_Pipeline [GitHub Actions] PR --\u003e|Triggers| Plan[terragrunt run-all plan] Plan --\u003e|Posts Comment| PR end eng[Engineer] --\u003e|Reviews Plan| PR eng --\u003e|Merges| Live_Repo Live_Repo --\u003e|Apply| AWS[AWS Cloud] style Reno fill:#00796b,stroke:#004d40,color:white style PR fill:#ffecb3,stroke:#ff6f00 style AWS fill:#232f3e,stroke:#ff9900,color:white A real Renovate pull request looks like this in GitHub:\nConfiguring Renovate for Terragrunt # Renovate is ideal because it has a dedicated Terragrunt manager. It parses HCL and finds underlying versions.\nHere is the renovate.json from my live repo, terraform-patterns-live (source on GitHub):\n{ \u0026#34;$schema\u0026#34;: \u0026#34;https://docs.renovatebot.com/renovate-schema.json\u0026#34;, \u0026#34;extends\u0026#34;: [ \u0026#34;config:recommended\u0026#34;, \u0026#34;:dependencyDashboard\u0026#34; ], \u0026#34;enabledManagers\u0026#34;: [ \u0026#34;terragrunt\u0026#34;, \u0026#34;terraform\u0026#34; ], \u0026#34;terragrunt\u0026#34;: { \u0026#34;managerFilePatterns\u0026#34;: [ \u0026#34;/\\\\.hcl$/\u0026#34; ], \u0026#34;versioning\u0026#34;: \u0026#34;regex:^((?\u0026lt;compatibility\u0026gt;.*)-v|v*)(?\u0026lt;major\u0026gt;\\\\d+)\\\\.(?\u0026lt;minor\u0026gt;\\\\d+)\\\\.(?\u0026lt;patch\u0026gt;\\\\d+)$\u0026#34; }, \u0026#34;packageRules\u0026#34;: [ { \u0026#34;matchDatasources\u0026#34;: [ \u0026#34;git-tags\u0026#34;, \u0026#34;github-tags\u0026#34;, \u0026#34;terraform-module\u0026#34; ], \u0026#34;groupName\u0026#34;: \u0026#34;infrastructure modules\u0026#34;, \u0026#34;matchPackageNames\u0026#34;: [ \u0026#34;/terraform-patterns-modules/\u0026#34; ] }, { \u0026#34;matchFileNames\u0026#34;: [ \u0026#34;**/development-account/**\u0026#34; ], \u0026#34;automerge\u0026#34;: true }, { \u0026#34;matchFileNames\u0026#34;: [ \u0026#34;**/production-account/**\u0026#34; ], \u0026#34;minimumReleaseAge\u0026#34;: \u0026#34;7 days\u0026#34; } ] } config:recommended plus :dependencyDashboard gives sensible defaults and a single GitHub Issue that lists pending updates. terragrunt.managerFilePatterns limits scanning to .hcl files; the versioning regex matches tags like vpc-v1.2.0 from this series (compatibility prefix + semver). packageRules: module updates under terraform-patterns-modules are grouped as “infrastructure modules”; paths under development-account may automerge when CI passes; production-account changes wait 7 days after release before Renovate proposes them (minimumReleaseAge). The Dependency Dashboard # The \u0026quot;:dependencyDashboard\u0026quot; preset prevents PR noise. Renovate creates a single GitHub Issue listing all available updates. You verify them there before checking a box to generate the actual PR.\nValidation Pipeline (CI) # A PR updating a version number is useless if you don\u0026rsquo;t know what it changes.\nI need a CI workflow (.github/workflows/plan.yaml) that runs terragrunt run-all plan on every PR.\nname: Terragrunt Plan on: [pull_request] jobs: plan: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: hashicorp/setup-terraform@v2 - uses: autero1/action-terragrunt@v1.3.0 - name: Terragrunt Plan run: | terragrunt run-all plan --terragrunt-non-interactive -out=tfplan.binary \u0026gt; plan.txt continue-on-error: true # Step to post plan.txt as a PR comment (using github-script) Deployment strategy: dev vs prod # You rarely want production to move at the same pace as development. In the config above, matchFileNames ties behavior to your repo layout: anything under development-account can automerge after CI; anything under production-account must pass a 7-day minimumReleaseAge before Renovate opens a PR, which gives a soak period on new module releases.\nAdjust the path globs to match your own account or folder names. The idea is the same: fast feedback in lower environments, slower, explicit promotion toward production.\nMoving to execution # You now have a pipeline that proposes updates automatically. But who approves and applies them?\nIf you merge a PR, does a human run terraform apply? In Part 6, I will introduce TACOS — Terraform Automation and Collaboration Software — and the tools that automate the final mile of execution safely.\n","date":"5 January 2026","externalUrl":null,"permalink":"/posts/tech-logs/part-5-automating-dependency-updates-renovate/","section":"All Posts","summary":"\u003cp\u003eThis is Part 5 of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. I have split my repositories, enforced versioning, automated releases, and set up my consumption model.\u003c/p\u003e\n\u003cp\u003eIn a traditional setup, upgrading infrastructure is painful. A developer releases \u003ccode\u003evpc-v1.1.0\u003c/code\u003e, sends a Slack message, and\u0026hellip; silence. Three months later, Prod is still on \u003ccode\u003evpc-v0.9.0\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eTo build a robust infrastructure, upgrades should be \u003cstrong\u003epushed to the consumer\u003c/strong\u003e, not pulled.\u003c/p\u003e\n\u003cp\u003eI achieve this with \u003cstrong\u003eRenovate Bot\u003c/strong\u003e.\u003c/p\u003e","title":"Part 5: Automating Dependency Updates with Renovate","type":"posts"},{"content":"","date":"5 January 2026","externalUrl":null,"permalink":"/tags/renovate/","section":"Tags","summary":"","title":"Renovate","type":"tags"},{"content":"","date":"5 January 2026","externalUrl":null,"permalink":"/tags/terragrunt/","section":"Tags","summary":"","title":"Terragrunt","type":"tags"},{"content":"","date":"1 January 2026","externalUrl":null,"permalink":"/tags/git/","section":"Tags","summary":"","title":"Git","type":"tags"},{"content":"","date":"1 January 2026","externalUrl":null,"permalink":"/tags/infrastructure-as-code/","section":"Tags","summary":"","title":"Infrastructure-as-Code","type":"tags"},{"content":"This is Part 4 of my series on Production-Grade Terraform Patterns. In Part 3, I automated tagging and releases for my modules. Now I need to consume those modules from a separate live repository.\nPrerequisite: This guide assumes you are using the split repo layout from Part 1 and that modules are versioned as in Part 2 and Part 3.\nSuppose the modules repo has a tag vpc-v1.1.0 and the live repo should deploy that infrastructure. There are two primary ways to wire that up:\nGit references: Point Terraform at a URL (usually with ?ref= set to a tag or commit). Module registry: Use Terraform\u0026rsquo;s module registry protocol. The CLI lists versions, matches constraints, downloads a tarball from a registry (Terraform Cloud/Enterprise, Artifactory, GitLab Terraform Registry, or any implementation of that protocol). That protocol is what makes registry installs different from plain Git: you get version metadata and a standard download path. The HashiCorp doc above is the place to read if you care about the HTTP details.\nThis is not only a syntax choice. It is a trade-off between keeping things simple and paying the cost of running a registry (metadata, semver constraints, a UI).\n%%{init: {\"themeVariables\": {\"fontSize\": \"22px\", \"fontFamily\": \"system-ui, Segoe UI, sans-serif\", \"primaryTextColor\": \"#111\"}, \"flowchart\": {\"nodeSpacing\": 28, \"rankSpacing\": 56, \"padding\": 36, \"curve\": \"basis\", \"htmlLabels\": true}}}%% flowchart TB subgraph Opt2 [Option 2: Git reference] direction TB gA[\"Live repoTerragrunt / Terraform\"] gB[\"Git hostmodules monorepo\"] gC[\"Module sourcetag + subfolder\"] gA --\u003e gB gB --\u003e gC end subgraph Opt1 [Option 1: Module registry] direction TB rA[\"Live repoTerragrunt / Terraform\"] rB[\"RegistryTerraform registry,gitlab,jfrog, …\"] rC[\"Module packageversioned .tar.gz\"] rA --\u003e rB rB --\u003e rC end classDef mid fill:#fafafa,stroke:#78909c,stroke-width:2px classDef gitLast fill:#e3f2fd,stroke:#1565c0,stroke-width:2px classDef regLast fill:#ffebee,stroke:#c62828,stroke-width:2px class gA,gB,rA,rB mid class gC gitLast class rC regLast class Opt1 fill:#f5f9ff,stroke:#1565c0,stroke-width:3px class Opt2 fill:#fff8f7,stroke:#c62828,stroke-width:3px Option 1: Git reference # For private, internal modules this is usually what I reach for first. Terraform pulls source straight from GitHub or GitLab. There is no registry in the middle.\nSyntax (Terragrunt) # terraform { # After the repo URL, // is the module root inside the clone (here, modules/vpc). source = \u0026#34;git::https://github.com/my-org/infra-modules.git//modules/vpc?ref=vpc-v1.1.0\u0026#34; } Pros: No registry to run; pin with a tag or commit SHA; the // path fits a monorepo layout.\nCons: You cannot put a Terraform-style semver constraint on the Git URL itself. You choose the ref, so every bump is explicit. Very large repos can mean heavier fetches than downloading a single module tarball.\nOption 2: Private registry # If you publish modules to a registry, consumers use a registry address instead of git::.\nSyntax (Terragrunt tfr://) # terraform { source = \u0026#34;tfr://app.terraform.io/my-org/vpc/aws?version=1.1.0\u0026#34; } In plain Terraform (a module block in .tf), the same module is often declared with a separate version argument. Terraform resolves version constraints (for example ~\u0026gt; 1.0) against the versions the registry publishes. That resolution flow is what the module registry protocol describes.\nPros: When you use Terraform\u0026rsquo;s module plus version pattern, you get constraint-aware resolution; downloads are typically tarballs; many registries expose docs and search in a UI.\nCons: You need a pipeline (or process) to publish versions. If you use loose constraints, one apply on Tuesday and another on Wednesday can resolve to different patch versions unless you treat upgrades as an explicit change. That is the same discipline problem as floating refs anywhere.\nWith Terragrunt, the practical approach for tfr:// is usually an exact ?version= in the URL unless your Terragrunt version and docs say otherwise. Do not assume terragrunt init resolves ~\u0026gt; 1.0 the same way a Terraform module block does.\nWhat I use in this series # For most internal teams, and for this walkthrough, Git references with an explicit tag or SHA stay simple, skip extra infrastructure, and keep what we deployed obvious in the live repo.\nUsing ~\u0026gt; 1.0-style implicit upgrades in production is risky: two applies on different days can pick different patch releases without anyone editing config. I prefer explicit bumps, whether the source is Git or a registry.\nGit authentication (CI/CD) # Do not embed tokens in module URLs. Configure Git once in the pipeline so plain https://github.com/... sources still work. For example:\n- name: Configure Git credentials run: | git config --global url.\u0026#34;https://oauth2:${{ secrets.GITHUB_TOKEN }}@github.com\u0026#34;.insteadOf https://github.com GITHUB_TOKEN only reaches other private repos when your org and workflow permissions allow it. Otherwise use a PAT or a GitHub App with repository access.\nSummary # Release Please tags vpc-v1.1.0. Terragrunt points at that tag with a git:: source (no credentials in HCL). In Part 5, I use Renovate so those pins update across many environments without hand-editing every file.\n","date":"1 January 2026","externalUrl":null,"permalink":"/posts/tech-logs/part-4-git-tags-vs-registry/","section":"All Posts","summary":"\u003cp\u003eThis is Part 4 of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. In \u003ca href=\"/posts/tech-logs/part-3-automating-releases-release-please/\"\u003ePart 3\u003c/a\u003e, I automated tagging and releases for my modules. Now I need to \u003cstrong\u003econsume\u003c/strong\u003e those modules from a separate live repository.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite\u003c/strong\u003e: This guide assumes you are using the \u003cstrong\u003esplit repo layout\u003c/strong\u003e from \u003ca href=\"/posts/tech-logs/part-1-split-repository-pattern/\"\u003ePart 1\u003c/a\u003e and that modules are versioned as in \u003ca href=\"/posts/tech-logs/part-2-production-ready-modules/\"\u003ePart 2\u003c/a\u003e and \u003ca href=\"/posts/tech-logs/part-3-automating-releases-release-please/\"\u003ePart 3\u003c/a\u003e.\u003c/p\u003e\u003c/blockquote\u003e\n\u003cp\u003eSuppose the modules repo has a tag \u003ccode\u003evpc-v1.1.0\u003c/code\u003e and the live repo should deploy that infrastructure. There are two primary ways to wire that up:\u003c/p\u003e","title":"Part 4: Consuming Terraform Modules: Git Tags vs Private Registry","type":"posts"},{"content":"","date":"1 January 2026","externalUrl":null,"permalink":"/tags/security/","section":"Tags","summary":"","title":"Security","type":"tags"},{"content":"This is Part 3 of my series on Production-Grade Terraform Patterns. In Part 2, I established standards for module creation. Now, I solve the biggest friction point in module management: Release Engineering.\nPrerequisite: This guide assumes you have defined your modules using the Production-Ready Standards from Part 2.\nTreating infrastructure as software means you must version it. But if an engineer updates a module, they shouldn\u0026rsquo;t have to manually calculate semantic versions (\u0026ldquo;Is this a minor or patch?\u0026rdquo;), write changelogs, or tag releases.\nIn high-velocity teams, manual release steps lead to:\n\u0026ldquo;Big Bang\u0026rdquo; Releases: Engineers delay releases to avoid the hassle, batching weeks of changes. Unstable References: People reference main because there isn\u0026rsquo;t a recent tag, breaking production reliability. The solution is to automate this entire lifecycle using Release Please.\nWhy Release Please? # There are many tools for this (like semantic-release), and any of them can work depending on your preference.\nThe critical point is that manually managing and calculating tags is difficult. I choose Release Please because:\nGoogle-Backed: It is maintained by Google. Easy Integration: It integrates seamlessly with GitHub Actions, automating the complex task of versioning without requiring heavy custom scripting. The Strategy: Conventional Commits # Automation requires structured data. You cannot automate versioning if commit messages are \u0026ldquo;fixed stuff\u0026rdquo; or \u0026ldquo;updated vpc\u0026rdquo;.\nI adopt Conventional Commits:\nPrefix SemVer Impact Example Result fix: Patch (0.0.x) fix(vpc): correct typo in subnet tag 1.0.0 -\u0026gt; 1.0.1 feat: Minor (0.x.0) feat(eks): add fargate support 1.0.0 -\u0026gt; 1.1.0 feat!: Major (x.0.0) feat!(s3): force encryption 1.0.0 -\u0026gt; 2.0.0 The ! indicates a breaking change (Major), regardless of the prefix.\nConfiguring Release Please # I often keep multiple modules in one repository. A change to the VPC module should not trigger a release for the RDS module.\nFor reference, see my terraform-patterns-modules repository.\nRelease Please uses a Manifest-Driven approach to handle this.\n1. The State: .release-please-manifest.json # This file tracks the current version of every component.\n{ \u0026#34;modules/vpc\u0026#34;: \u0026#34;1.0.0\u0026#34;, \u0026#34;modules/eks-cluster\u0026#34;: \u0026#34;2.1.0\u0026#34; } 2. The Configuration: release-please-config.json # This defines the strategy. I use the terraform-module type, which knows how to update Terraform files and READMEs.\n{ \u0026#34;packages\u0026#34;: { \u0026#34;modules/vpc\u0026#34;: { \u0026#34;release-type\u0026#34;: \u0026#34;terraform-module\u0026#34;, \u0026#34;package-name\u0026#34;: \u0026#34;aws-vpc-module\u0026#34;, \u0026#34;changelog-path\u0026#34;: \u0026#34;CHANGELOG.md\u0026#34; }, \u0026#34;modules/eks-cluster\u0026#34;: { \u0026#34;release-type\u0026#34;: \u0026#34;terraform-module\u0026#34;, \u0026#34;package-name\u0026#34;: \u0026#34;aws-eks-cluster-module\u0026#34; } } } The Workflow: GitHub Actions # I create a workflow .github/workflows/release.yaml.\nThis workflow utilizes a specific pattern: The Persistent Release PR. When you merge a feat: into main, Release Please doesn\u0026rsquo;t release immediately. It opens (or updates) a dedicated \u0026ldquo;Release PR\u0026rdquo;. This PR contains the calculated Changelog and version bump.\nThe release is only \u0026ldquo;cut\u0026rdquo; when you merge this specific Release PR.\nname: Release Please on: push: branches: - main jobs: release-please: runs-on: ubuntu-latest steps: - uses: google-github-actions/release-please-action@v4 with: config-file: release-please-config.json manifest-file: .release-please-manifest.json token: ${{ secrets.GITHUB_TOKEN }} sequenceDiagram participant Dev as Developer participant Main as Main Branch participant RP as Release Please (Bot) participant PR as Release PR participant Tag as Git Tag Dev-\u003e\u003eMain: git commit -m \"feat: new vpc\" activate Main Main-\u003e\u003eRP: Trigger Action deactivate Main activate RP RP-\u003e\u003eRP: Analyze Commits (feat = minor) RP-\u003e\u003ePR: Open/Update \"chore: release 1.1.0\" deactivate RP Note over PR: Contains CHANGELOG.mdand version bumps Dev-\u003e\u003ePR: Review \u0026 Merge activate PR PR-\u003e\u003eMain: Merge Pull Request deactivate PR activate Main Main-\u003e\u003eRP: Trigger Action (on Merge) deactivate Main activate RP RP-\u003e\u003eTag: Create Tag v1.1.0 RP-\u003e\u003eRP: Publish GitHub Release deactivate RP Summary # This workflow transforms the developer experience:\nCode: Engineer commits feat: add private subnet. Merge: PR merged to main. Propose: Release Please opens a PR: \u0026ldquo;chore: release modules/vpc 1.1.0\u0026rdquo;. Release: Team Lead merges that PR. Publish: Tag is created, Release is published. I now have strictly versioned, immutable artifacts. In Part 4, I will decide how to consume these artifacts: via simple Git Tags or a Private Registry.\n","date":"28 December 2025","externalUrl":null,"permalink":"/posts/tech-logs/part-3-automating-releases-release-please/","section":"All Posts","summary":"\u003cp\u003eThis is Part 3 of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. In \u003ca href=\"/posts/tech-logs/part-2-production-ready-modules/\"\u003ePart 2\u003c/a\u003e, I established standards for module creation. Now, I solve the biggest friction point in module management: \u003cstrong\u003eRelease Engineering\u003c/strong\u003e.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite\u003c/strong\u003e: This guide assumes you have defined your modules using the \u003cstrong\u003eProduction-Ready Standards\u003c/strong\u003e from \u003ca href=\"/posts/tech-logs/part-2-production-ready-modules/\"\u003ePart 2\u003c/a\u003e.\u003c/p\u003e\u003c/blockquote\u003e\n\u003cp\u003eTreating infrastructure as software means you must version it. But if an engineer updates a module, they shouldn\u0026rsquo;t have to manually calculate semantic versions (\u0026ldquo;Is this a minor or patch?\u0026rdquo;), write changelogs, or tag releases.\u003c/p\u003e","title":"Part 3: Automating Semantic Versioning with Release Please","type":"posts"},{"content":"","date":"28 December 2025","externalUrl":null,"permalink":"/tags/release-please/","section":"Tags","summary":"","title":"Release-Please","type":"tags"},{"content":"","date":"24 December 2025","externalUrl":null,"permalink":"/tags/best-practices/","section":"Tags","summary":"","title":"Best-Practices","type":"tags"},{"content":"","date":"24 December 2025","externalUrl":null,"permalink":"/tags/modules/","section":"Tags","summary":"","title":"Modules","type":"tags"},{"content":"This is Part 2 of my series on Production-Grade Terraform Patterns. In Part 1, I established the architecture for scaling infrastructure. Now, I focus on the building blocks: the Modules.\nPrerequisite: This guide builds upon the Split Repository Pattern defined in Part 1. I highly recommend reading it first to understand the architectural context.\nIn many tutorials, a Terraform module is treated as a simple folder of .tf scripts. However, in a professional engineering environment, a module must be treated as a Software Product. It requires a well-defined API (Variables), strict guarantees (Validation), and a stable lifecycle (Versioning).\nIf you write \u0026ldquo;lazy\u0026rdquo; modules, your infrastructure will be fragile. Building Production-Ready Modules creates an infrastructure that is stable, reusable, and safe.\nThe Anatomy of a Module # graph TD %% Actors User(Consumer) Cloud(AWS Provider) %% The Module subgraph Module [\"Terraform Module\"] direction TB Vars(\"variables.tf(Inputs)\") Main(\"main.tf(Logic)\") Outs(\"outputs.tf(Outputs)\") Vars --\u003e Main Main --\u003e Outs end %% Data Flow User --\u003e|Step 1: Define Inputs| Vars Main --\u003e|Step 2: Create Resources| Cloud Outs --\u003e|Step 3: Return Attributes| User %% Styling style Module fill:#f4f6f7,stroke:#bdc3c7,stroke-width:2px,rx:10,ry:10 style Vars fill:#fff,stroke:#e67e22,stroke-width:2px,rx:5,ry:5 style Main fill:#fff,stroke:#3498db,stroke-width:2px,rx:5,ry:5 style Outs fill:#fff,stroke:#9b59b6,stroke-width:2px,rx:5,ry:5 style User fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5 style Cloud fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5 A production module must be standardized. When an engineer opens any module (e.g., modules/s3-secure), they should immediately understand the structure. I never dump everything into main.tf.\nFor a comprehensive set of rules on naming and organization, I recommend the official HashiCorp Terraform Style Guide.\n1. Standard File Structure # Every module must contain these three files at a minimum:\nmain.tf: The Logic. Contains the resources (e.g. aws_s3_bucket, aws_instance). Keep it focused. variables.tf: The Interface. Defines every input the module accepts. This is your API contract. outputs.tf: The Return Values. Exposes IDs, ARNs, and endpoints to the consumer. 2. Don\u0026rsquo;t Reinvent the Wheel: The Wrapper Pattern # After years of writing modules, my biggest recommendation is: Do not write modules from scratch.\nUnless you have very specific requirements, use the open-source community modules (like terraform-aws-modules) and wrap them. This gives you the stability of a battle-tested module while keeping your specific defaults (Standardization).\nFor example, instead of defining resource \u0026quot;aws_s3_bucket\u0026quot; \u0026quot;main\u0026quot; {...} with 50 lines of configuration, wrap the community module:\nmodule \u0026#34;s3_bucket\u0026#34; { source = \u0026#34;terraform-aws-modules/s3-bucket/aws\u0026#34; version = \u0026#34;5.9.1\u0026#34; bucket = var.bucket_name # Security Defaults: Enforced for everyone using this module block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true versioning = { enabled = var.versioning } server_side_encryption_configuration = { rule = { apply_server_side_encryption_by_default = { sse_algorithm = \u0026#34;AES256\u0026#34; } } } } This ensures that every bucket created via your platform has block_public_acls enabled by default, without every developer needing to remember it.\n2. Input Validation (The Contract) # The biggest difference between a script and a product is Validation.\nIf a user tries to create a storage bucket with an invalid retention period, the module should fail fast—before it even talks to the cloud API.\nTerraform 1.0+ allows me to enforce this contract natively in variables.tf:\nvariable \u0026#34;retention_days\u0026#34; { type = number description = \u0026#34;Number of days to retain logs. Must be greater than 7.\u0026#34; default = 30 validation { condition = var.retention_days \u0026gt; 7 error_message = \u0026#34;Retention period must be greater than 7 days to meet compliance standards.\u0026#34; } } By adding validation, you shift compliance left. You prevent \u0026ldquo;garbage in\u0026rdquo; from ever reaching your cloud provider.\nManaging Dependencies: versions.tf # In production, you cannot assume that \u0026ldquo;Terraform\u0026rdquo; means the same thing to everyone. Your module might rely on a feature added in Terraform 1.3.\nEvery module must have a versions.tf file that explicitly defines its requirements.\nterraform { # 1. Require a minimum Terraform binary version required_version = \u0026#34;\u0026gt;= 1.5.0\u0026#34; # 2. Pin Provider Versions required_providers { aws = { source = \u0026#34;hashicorp/aws\u0026#34; # Allow any 5.x version, but do not allow 6.0 (Breaking Changes) version = \u0026#34;~\u0026gt; 5.0\u0026#34; } } } The Strategy: Broad Constraints # Notice I used ~\u0026gt; 5.0 (Lazy Constraint) instead of = 5.12.0 (Exact Pin).\nIn Live Infrastructure: I pin exactly to ensure reproducibility. In Modules: I use broad constraints. I want the module to be compatible with a wide range of provider versions so that consuming teams aren\u0026rsquo;t forced to upgrade their entire stack just to use a minor module update.\nThe \u0026ldquo;Diamond Dependency\u0026rdquo; Problem # Why am I so obsessed with versioning? Because of dependencies.\nImagine you have a live environment that uses two modules:\nModule A (Network) depends on hashicorp/aws version 4.0. Module B (Database) depends on hashicorp/aws version 5.0. If you try to use them together, Terraform will fail to initialize. By maintaining strict versions of your modules (e.g., releasing v1.0 compatible with AWS v4, and v2.0 compatible with AWS v5), you allow consumers to upgrade incrementally.\nStatic Analysis and Testing # Before releasing a module, I must ensure it is correct. While full integration testing is expensive, static analysis is free and fast.\nYour CI pipeline for the module repository should run these two commands on every Pull Request:\nterraform fmt -check: Ensures code style consistency. terraform validate: Checks for syntax errors and valid references. However, built-in tools aren\u0026rsquo;t enough. I highly recommend adding these two advanced scanners:\n3. TFLint (Quality Assurance) # terraform validate only checks syntax. tflint checks for semantics and provider-specific issues. It acts like a spell-checker for your cloud resources.\nFor example, terraform validate thinks instance_type = \u0026quot;t9.large\u0026quot; is fine (it\u0026rsquo;s a valid string). TFLint knows that t9.large doesn\u0026rsquo;t exist in AWS and will warn you immediately.\nIt is also excellent at detecting unused declarations. If you define a variable \u0026quot;foo\u0026quot; but never use it, TFLint will flag it, keeping your codebase clean.\nTo enable this, add a .tflint.hcl to your repository root:\nplugin \u0026#34;aws\u0026#34; { enabled = true version = \u0026#34;0.32.0\u0026#34; source = \u0026#34;github.com/terraform-linters/tflint-ruleset-aws\u0026#34; } plugin \u0026#34;terraform\u0026#34; { enabled = true preset = \u0026#34;recommended\u0026#34; } tflint --init tflint 4. Checkov (Security Compliance) # Infrastructure as Code allows you to build insecure things very quickly. Checkov is a static code analysis tool for infrastructure that scans for security misconfigurations.\nIt has hundreds of built-in policies. If you try to create an unencrypted S3 bucket or a security group open to 0.0.0.0/0, Checkov will fail the build.\ncheckov -d . By adding these to your CI pipeline, you ensure that Quality and Security are baked into every module version.\nBy adding these to your CI pipeline, you ensure that Quality and Security are baked into every module version.\n5. Automated Integration Testing (Terratest) # Static analysis is great, but it can\u0026rsquo;t prove that your infrastructure actually works. For that, you need integration tests. I use Terratest, a Go library that helps you write automated tests for your infrastructure code.\nIt works by:\nDeploying your real infrastructure (using terraform apply). Validating it works (e.g., making an HTTP request to your load balancer or checking if an S3 bucket exists). Destroying it (using terraform destroy). A simple test for our S3 module might look like this:\npackage test import ( \u0026#34;testing\u0026#34; \u0026#34;github.com/gruntwork-io/terratest/modules/terraform\u0026#34; \u0026#34;github.com/stretchr/testify/assert\u0026#34; ) func TestS3Module(t *testing.T) { terraformOptions := \u0026amp;terraform.Options{ // The path to where our Terraform code is located TerraformDir: \u0026#34;../examples/s3-private\u0026#34;, } // At the end of the test, run `terraform destroy` to clean up any resources that were created defer terraform.Destroy(t, terraformOptions) // This will run `terraform init` and `terraform apply` and fail the test if there are any errors terraform.InitAndApply(t, terraformOptions) // Validate your code works output := terraform.Output(t, terraformOptions, \u0026#34;bucket_arn\u0026#34;) assert.Contains(t, output, \u0026#34;arn:aws:s3\u0026#34;) } This gives you the confidence to refactor and upgrade versions without fear of breaking production.\nI have defined what makes a module \u0026ldquo;Production-Ready\u0026rdquo;:\nStandardized: Predictable file structure. Safe: Inputs are validated. Compatible: Dependencies are broad but bounded. But currently, the process is manual. To release v1.0.0, humans have to edit files, tag commits, and update changelogs. In Part 3, I will implement Release Please to automate the versioning process completely.\n","date":"24 December 2025","externalUrl":null,"permalink":"/posts/tech-logs/part-2-production-ready-modules/","section":"All Posts","summary":"\u003cp\u003eThis is Part 2 of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. In \u003ca href=\"/posts/tech-logs/part-1-split-repository-pattern/\"\u003ePart 1\u003c/a\u003e, I established the architecture for scaling infrastructure. Now, I focus on the building blocks: the \u003cstrong\u003eModules\u003c/strong\u003e.\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003ePrerequisite\u003c/strong\u003e: This guide builds upon the \u003cstrong\u003eSplit Repository Pattern\u003c/strong\u003e defined in \u003ca href=\"/posts/tech-logs/part-1-split-repository-pattern/\"\u003ePart 1\u003c/a\u003e. I highly recommend reading it first to understand the architectural context.\u003c/p\u003e\u003c/blockquote\u003e\n\u003cp\u003eIn many tutorials, a Terraform module is treated as a simple folder of \u003ccode\u003e.tf\u003c/code\u003e scripts. However, in a professional engineering environment, a module must be treated as a \u003cstrong\u003eSoftware Product\u003c/strong\u003e. It requires a well-defined API (Variables), strict guarantees (Validation), and a stable lifecycle (Versioning).\u003c/p\u003e","title":"Part 2: Writing Production-Ready Terraform Modules","type":"posts"},{"content":"\ngraph TD %% REPOS LiveRepo(terraform-patterns-live) ModRepo(terraform-patterns-modules) %% MODULE VERSIONS ModV1(\"ecs-cluster-v0.1.0(Stable)\") ModV2(\"ecs-cluster-v0.2.0(Beta)\") ModRepo --- ModV1 ModRepo --- ModV2 %% LIVE BRANCH 1: PRODUCTION LiveRepo --\u003e ProdAcc[Account: production-account] --\u003e ProdReg[Region: us-east-1] --\u003e ProdApp[ecs-cluster] %% LIVE BRANCH 2: STAGING LiveRepo --\u003e StageAcc[Account: staging-account] --\u003e StageReg[Region: us-east-1] --\u003e StageApp[ecs-cluster] %% LIVE BRANCH 3: DEV LiveRepo --\u003e DevAcc[Account: development-account] --\u003e DevReg[Region: us-east-1] --\u003e DevApp[ecs-cluster] %% VERSION BINDINGS ProdApp -.-\u003e|binds to| ModV1 StageApp -.-\u003e|binds to| ModV2 DevApp -.-\u003e|binds to| ModV2 %% STYLING style LiveRepo fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5 style ModRepo fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5 style ModV1 fill:#fff,stroke:#34495e,stroke-width:2px,rx:5,ry:5 style ModV2 fill:#fff,stroke:#34495e,stroke-width:2px,rx:5,ry:5 style ProdAcc fill:#e8f8f5,stroke:#1abc9c,color:#000,rx:5,ry:5 style ProdReg fill:#a2d9ce,stroke:#16a085,color:#000,rx:5,ry:5 style ProdApp fill:#fff,stroke:#1abc9c,stroke-width:2px,rx:5,ry:5 style StageAcc fill:#fef9e7,stroke:#f1c40f,color:#000,rx:5,ry:5 style StageReg fill:#f9e79f,stroke:#f39c12,color:#000,rx:5,ry:5 style StageApp fill:#fff,stroke:#f1c40f,stroke-width:2px,rx:5,ry:5 style DevAcc fill:#f4ecf7,stroke:#9b59b6,color:#000,rx:5,ry:5 style DevReg fill:#e8daef,stroke:#8e44ad,color:#000,rx:5,ry:5 style DevApp fill:#fff,stroke:#9b59b6,stroke-width:2px,rx:5,ry:5 This is Part 1 of my series on Production-Grade Terraform Patterns. I am moving beyond basic tutorials to build an infrastructure capable of handling hundreds of resources, multiple environments, and dozens of engineers.\nThis approach is heavily inspired by the reference architectures provided by Gruntwork. Specifically, it adapts the patterns demonstrated in their Infrastructure Catalog and Live Stacks examples.\nSource Code:\nModules: https://github.com/salsiy/terraform-patterns-modules Live: https://github.com/salsiy/terraform-patterns-live Before we start, I assume you have some basic familiarity with Terraform and Terragrunt. If you are just getting started, I highly recommend checking out the official HashiCorp Terraform Tutorials and the Terragrunt Quick Start Guide. Also, for the purpose of this demonstration, I will be using AWS Cloud.\nIf you have used Terraform for personal projects, you know how satisfying terraform apply can be. You write a main.tf, run a command, and infrastructure appears.\nBut in a production with a growing team, that simplicity disappears.\nThe Scaling Problem # As your infrastructure scales, you will inevitably face the pitfalls of a monolithic architecture:\nBlast Radius: You want to update a security group in Dev, but your state file includes Prod. One mistake destroys the production database. With Terragrunt, each unit (module) will have a separate state file, strictly isolating the impact of any change.\nEnvironment Drift: You apply a fix in Dev, but forget to apply it to Prod. Over time, your environments diverge, and deployments become a guessing game. This setup solves this by packing infrastructure as Versioned Modules, ensuring the exact same code is promoted from environment to environment.\nCode Duplication: Dev, Stage, and Prod are 99% identical. You end up copy-pasting code three times, making maintenance a nightmare.\nTo solve this, you don\u0026rsquo;t just need better code; you need a better Architecture. I recommend the Split Repository Pattern powered by Terragrunt.\nThe Development Workflow: Scaled Trunk-Based # Architecture is useless without a workflow. To manage these repositories effectively, I adopt Scaled Trunk-Based Development.\nIn Infrastructure as Code, long-lived feature branches are dangerous. If you branch off for 2 weeks to build a VPC, and I branch off to build ECS, whoever merges last faces a massive, risky conflict resolution that could break production.\nThe Rules:\nShort-Lived Branches: Features are merged to main within hours, not days. Main is Production: The main branch of the Live Repo should always reflect what is currently deployed (or being deployed). This aligns perfectly with the Split-Repo pattern: you iterate rapidly on Modules (Logic) using releases, while your Live Repo (State) moves forward in small, incremental steps.\nThe Architecture: Split Repositories # The most vital decision you will make is to separate your Definition (Modules) from your Implementation (Live State).\nInstead of one giant repo, I split my world in two:\n1. The Modules Repository (The Logic) # This allows us to treat infrastructure code like software libraries.\nContent: Pure Terraform HCL (.tf files). Purpose: Reusable logic. E.g., \u0026ldquo;This is how I build a standard ECS cluster.\u0026rdquo; Key Trait: It knows nothing about your specific environments. No account IDs, no \u0026ldquo;prod\u0026rdquo; strings. Management: Strictly versioned using semantic versioning (Tags). 2. The Live Repository (The State) # This represents your actual deployable environments.\nContent: Terragrunt configuration (.hcl files). Purpose: To call the modules and pass in specific inputs. Key Trait: If a folder exists here, it exists in the cloud. Management: Organized hierarchically by Account, Region, and Environment. Why Terragrunt? # Terragrunt is a thin wrapper that significantly enhances Terraform without replacing it. Its primary goal is to keep your configuration DRY (Don\u0026rsquo;t Repeat Yourself). By automating remote state setup and enforcing consistency, it eliminates code duplication across your Dev, Stage, and Prod environments, making complex infrastructure maintainable and scalable.\nYou might ask, \u0026ldquo;Why can\u0026rsquo;t I just use Terraform workspaces or standard .tfvars files?\u0026rdquo;\nYou can, but Terraform is not designed to be DRY regarding backend configuration.\nWithout Terragrunt, every component in your live repo needs a hardcoded backend \u0026quot;s3\u0026quot; {...} block. If you have 50 components, you have 50 backend configs to maintain. Terragrunt allows you to write this once in a root file and inherit it everywhere.\nThe Directory Hierarchy # In your Live repository, the folder structure is your source of truth. I follow the Account -\u0026gt; Region -\u0026gt; Environment hierarchy to physically isolate failure domains. This approaches uses a Multi-Account Strategy, ensuring strict isolation between Development, Staging, and Production. This aligns with the AWS Best Practices for Organizing Your AWS Environment, designed to minimize blast radius and simplify access management.\nterraform-patterns-modules/ # 1. The Logic ├── ecs-cluster/ │ ├── main.tf │ ├── variables.tf │ └── outputs.tf ├── vpc/ └── ... terraform-patterns-live/ # 2. The State ├── root.hcl # 1. Global Configuration (State Bucket, Locking) ├── tags.yaml # 2. Global Tags ├── _envcommon/ # 3. DRY Module Configs (Global) │ ├── vpc.hcl │ └── ecs-cluster.hcl ├── production-account/ # 4. Account Isolation │ └── us-east-1/ # 5. Region Isolation │ ├── vpc/ # 6. Component │ │ └── terragrunt.hcl │ └── ecs-cluster/ │ └── terragrunt.hcl ├── development-account/ │ └── us-east-1/ │ └── ... └── staging-account/ └── ... Why this works:\nIf you run a command inside production-account/us-east-1/prod/vpc, Terragrunt can only see that specific folder. It is physically impossible for a command run there to accidentally delete resources in staging-account.\nImplementation: How Inheritance Works # The magic of Terragrunt lies in the include block.\n1. The Root Config (terraform-patterns-live/root.hcl) # This file sits at the top of your repo. It ensures every component stores its state in the correct place automatically.\nlocals { # Automatically load account \u0026amp; region variables account_vars = read_terragrunt_config(find_in_parent_folders(\u0026#34;account.hcl\u0026#34;)) region_vars = read_terragrunt_config(find_in_parent_folders(\u0026#34;region.hcl\u0026#34;)) account_name = local.account_vars.locals.account_name aws_region = local.region_vars.locals.aws_region } # Generate an AWS provider block generate \u0026#34;provider\u0026#34; { path = \u0026#34;provider.tf\u0026#34; if_exists = \u0026#34;overwrite_terragrunt\u0026#34; contents = \u0026lt;\u0026lt;EOF provider \u0026#34;aws\u0026#34; { region = \u0026#34;${local.aws_region}\u0026#34; # ... } EOF } # Configure Terragrunt to automatically store tfstate files in an S3 bucket remote_state { backend = \u0026#34;s3\u0026#34; config = { encrypt = true bucket = \u0026#34;terragrunt-example-tf-state-${local.account_name}-${local.aws_region}\u0026#34; key = \u0026#34;${path_relative_to_include()}/tf.tfstate\u0026#34; # ... } } # Pass these to all child modules inputs = merge( local.account_vars.locals, local.region_vars.locals, ) 2. The DRY Config (_envcommon/ecs-cluster.hcl) # This is where the magic happens. We define the module source and common variables once. All environments (Dev, Stage, Prod) inherit from here.\nsource = \u0026#34;git::https://github.com/my-org/terraform-patterns-modules.git//ecs-cluster?ref=ecs-cluster-v0.1.0\u0026#34; } inputs = { # Common inputs for ALL environments cluster_name = \u0026#34;main-cluster\u0026#34; } 3. The Component Config (.../us-east-1/ecs-cluster/terragrunt.hcl) # This file lives in the specific environment folder. It does two things:\nInherits the backend config (so you don\u0026rsquo;t type it again). Points to a specific version of your module. # The Child Configuration include \u0026#34;root\u0026#34; { path = find_in_parent_folders(\u0026#34;root.hcl\u0026#34;) } include \u0026#34;envcommon\u0026#34; { path = \u0026#34;${dirname(find_in_parent_folders(\u0026#34;root.hcl\u0026#34;))}/_envcommon/ecs-cluster.hcl\u0026#34; } inputs = { env_name = \u0026#34;production\u0026#34; services = { app-service = { cpu = 256 memory = 512 # ... container definitions ... } } } 4. Global Tagging: Consistent Metadata # In the root of the repo, you\u0026rsquo;ll notice a tags.yaml. This file defines tags that every single resource in your infrastructure must have (e.g., Project, Owner, ManagedBy).\n# tags.yaml Project: \u0026#34;terraform-patterns\u0026#34; Owner: \u0026#34;DevOps Team\u0026#34; ManagedBy: \u0026#34;Terraform/Terragrunt\u0026#34; In root.hcl, we read this file and inject it into every module. This guarantees that whether you deploy a database in Prod or a load balancer in Dev, they all carry consistent metadata for billing and auditing.\nConclusion # Transitioning to the Split Repository Pattern is the difference between maintaining a hobby project and operating a professional infrastructure platform.\nZero Ambiguity: The file structure tells you exactly what is deployed where. Zero Drift: Versioned modules ensure that Staging and Production run the exact same logic. Total Confidence: Physical isolation means you can break Dev without ever risking Prod. In Part 2: Production-Ready Modules, we will dive deep into the Modules Repository and learn how to write clean, reusable, and versioned Terraform code.\n","date":"20 December 2025","externalUrl":null,"permalink":"/posts/tech-logs/part-1-split-repository-pattern/","section":"All Posts","summary":"\u003cbr\u003e\n\u003cdiv class=\"mermaid\" align=\"center\"\u003e\n  \ngraph TD\n    %% REPOS\n    LiveRepo(terraform-patterns-live)\n    ModRepo(terraform-patterns-modules)\n\n    %% MODULE VERSIONS\n    ModV1(\"ecs-cluster-v0.1.0\u003cbr/\u003e(Stable)\")\n    ModV2(\"ecs-cluster-v0.2.0\u003cbr/\u003e(Beta)\")\n    \n    ModRepo --- ModV1\n    ModRepo --- ModV2\n\n    %% LIVE BRANCH 1: PRODUCTION\n    LiveRepo --\u003e ProdAcc[Account: production-account] --\u003e ProdReg[Region: us-east-1] --\u003e ProdApp[ecs-cluster]\n    \n    %% LIVE BRANCH 2: STAGING\n    LiveRepo --\u003e StageAcc[Account: staging-account] --\u003e StageReg[Region: us-east-1] --\u003e StageApp[ecs-cluster]\n\n    %% LIVE BRANCH 3: DEV\n    LiveRepo --\u003e DevAcc[Account: development-account] --\u003e DevReg[Region: us-east-1] --\u003e DevApp[ecs-cluster]\n\n    %% VERSION BINDINGS\n    ProdApp -.-\u003e|binds to| ModV1\n    StageApp -.-\u003e|binds to| ModV2\n    DevApp -.-\u003e|binds to| ModV2\n\n    %% STYLING\n    style LiveRepo fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5\n    style ModRepo fill:#34495e,color:#fff,stroke-width:0px,rx:5,ry:5\n    style ModV1 fill:#fff,stroke:#34495e,stroke-width:2px,rx:5,ry:5\n    style ModV2 fill:#fff,stroke:#34495e,stroke-width:2px,rx:5,ry:5\n\n    style ProdAcc fill:#e8f8f5,stroke:#1abc9c,color:#000,rx:5,ry:5\n    style ProdReg fill:#a2d9ce,stroke:#16a085,color:#000,rx:5,ry:5\n    style ProdApp fill:#fff,stroke:#1abc9c,stroke-width:2px,rx:5,ry:5\n\n    style StageAcc fill:#fef9e7,stroke:#f1c40f,color:#000,rx:5,ry:5\n    style StageReg fill:#f9e79f,stroke:#f39c12,color:#000,rx:5,ry:5\n    style StageApp fill:#fff,stroke:#f1c40f,stroke-width:2px,rx:5,ry:5\n\n    style DevAcc fill:#f4ecf7,stroke:#9b59b6,color:#000,rx:5,ry:5\n    style DevReg fill:#e8daef,stroke:#8e44ad,color:#000,rx:5,ry:5\n    style DevApp fill:#fff,stroke:#9b59b6,stroke-width:2px,rx:5,ry:5\n\n\u003c/div\u003e\n\n\u003cbr\u003e\n\u003cp\u003eThis is Part 1 of my series on \u003ca href=\"/series/production-grade-terraform-patterns/\"\u003eProduction-Grade Terraform Patterns\u003c/a\u003e. I am moving beyond basic tutorials to build an infrastructure capable of handling hundreds of resources, multiple environments, and dozens of engineers.\u003c/p\u003e","title":"Part 1: Structuring Terraform at Scale — The Split Repository Pattern","type":"posts"},{"content":"Welcome to the Production-Grade Terraform Patterns series.\nIn this 6-part guide, I walk through the journey of maturing your infrastructure from simple scripts to a robust, self-updating platform.\nWhat you will learn: # Part 1: How to structure repositories using the Split-Repo Pattern to isolate failure domains. Part 2: How to write \u0026ldquo;Production-Ready\u0026rdquo; modules that are strictly versioned, validated, and tested. Part 3: How to automate the entire release lifecycle (Changelogs, Tags) using Release Please. Part 4: Why you should favor secure Git Tags over Private Registries for internal code. Part 5: How to automate dependency upgrades across 100+ environments using Renovate. Part 6: The series finale — scale execution with TACOS: plan and apply in the pull request, with locking and review. ","date":"20 December 2025","externalUrl":null,"permalink":"/series/production-grade-terraform-patterns/","section":"Series","summary":"\u003cp\u003eWelcome to the \u003cstrong\u003eProduction-Grade Terraform Patterns\u003c/strong\u003e series.\u003c/p\u003e\n\u003cp\u003eIn this 6-part guide, I walk through the journey of maturing your infrastructure from simple scripts to a robust, self-updating platform.\u003c/p\u003e\n\n\n\u003ch3 class=\"relative group\"\u003eWhat you will learn: \n    \u003cdiv id=\"what-you-will-learn\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#what-you-will-learn\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003ePart 1\u003c/strong\u003e: How to structure repositories using the Split-Repo Pattern to isolate failure domains.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePart 2\u003c/strong\u003e: How to write \u0026ldquo;Production-Ready\u0026rdquo; modules that are strictly versioned, validated, and tested.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePart 3\u003c/strong\u003e: How to automate the entire release lifecycle (Changelogs, Tags) using Release Please.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePart 4\u003c/strong\u003e: Why you should favor secure Git Tags over Private Registries for internal code.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePart 5\u003c/strong\u003e: How to automate dependency upgrades across 100+ environments using Renovate.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003ePart 6\u003c/strong\u003e: The series finale — scale execution with TACOS: plan and apply in the pull request, with locking and review.\u003c/li\u003e\n\u003c/ul\u003e","title":"Production-Grade Terraform Patterns","type":"series"},{"content":"Last month, I spent four hours tracking down why our AWS bill increased. The culprit? A few forgotten resources across multiple regions that no one remembered creating.\nI realized we needed automated infrastructure monitoring, but every solution I looked at cost more than the resources we were trying to optimize. That\u0026rsquo;s when I decided to build something different.\nThe monitoring problem # Most AWS monitoring tools are either free but useless, or useful but expensive:\nFree tools: CloudWatch alerts exist but won\u0026rsquo;t tell you which S3 buckets are public or which IAM users lack MFA Paid tools: AWS Config charges $2 per rule. Monitor 50 things? That\u0026rsquo;s $100/month. Datadog? Try $500/month for medium infrastructure I needed something I could customize without writing boto3 code for every check.\nDiscovering Steampipe # Steampipe turns AWS APIs into SQL tables. Instead of this:\nimport boto3 client = boto3.client(\u0026#39;ec2\u0026#39;) response = client.describe_instances() # ... 20 more lines You write this:\nSELECT instance_id, instance_type FROM aws_ec2_instance WHERE instance_state = \u0026#39;running\u0026#39;; Anyone who knows SQL can now write AWS queries.\nSystem architecture # Here\u0026rsquo;s the complete system I built:\nEventBridge Schedule triggers the task daily at 9 AM UTC ECS Fargate Task runs Steampipe with 256 CPU / 512 MB memory Steampipe Engine executes SQL queries against AWS APIs (EC2, S3, RDS, IAM) SNS Topic receives query results and task completion events AWS Chatbot forwards SNS messages to Slack EventBridge Rule captures task state changes for completion notifications CloudWatch Logs stores execution logs for debugging The system orchestrates scheduled queries, processes results, and delivers real-time notifications—all serverless.\ngraph TB EB1[EventBridge ScheduleDaily 9 AM UTC] ECS[ECS Fargate Task256 CPU / 512 MB] SP[run_queries.sh] STEAM[Steampipe Engine] AWS[AWS APIsEC2, S3, RDS, IAM] SNS[SNS Topicsteampipe-reports] CB[AWS ChatbotAmazon Q] SLACK[Slack Channel] CW[CloudWatch Logs] EB2[EventBridge RuleTask State Change] EB1 --\u003e|Trigger Daily| ECS ECS --\u003e|Execute| SP SP --\u003e|Run Queries| STEAM STEAM -.Query.-\u003e AWS SP --\u003e|Query Results| SNS ECS --\u003e|Logs| CW ECS --\u003e|Task Stops| EB2 EB2 --\u003e|Completion Event| SNS SNS --\u003e|Forward| CB CB --\u003e|Notify| SLACK style EB1 fill:#ff9900,stroke:#232f3e,stroke-width:2px,color:#fff style ECS fill:#ff9900,stroke:#232f3e,stroke-width:2px,color:#fff style SP fill:#3b82f6,stroke:#1e40af,stroke-width:2px,color:#fff style STEAM fill:#10b981,stroke:#059669,stroke-width:2px,color:#fff style AWS fill:#232f3e,stroke:#ff9900,stroke-width:2px,color:#fff style SNS fill:#ff4b4b,stroke:#232f3e,stroke-width:2px,color:#fff style CB fill:#232f3e,stroke:#ff9900,stroke-width:2px,color:#fff style SLACK fill:#611f69,stroke:#4a154b,stroke-width:2px,color:#fff style CW fill:#ff9900,stroke:#232f3e,stroke-width:2px,color:#fff style EB2 fill:#ff9900,stroke:#232f3e,stroke-width:2px,color:#fff How it works # The workflow is straightforward:\nEventBridge Schedule triggers the ECS Fargate task daily Container starts and executes the shell script Steampipe queries run against AWS APIs, results go to SNS AWS Chatbot forwards query results to your Slack channel Task completes, EventBridge captures the state change and sends a completion notification All infrastructure is defined in Terraform with five focused modules: SNS, IAM, networking, ECS, and Slack notifications. Each does one thing well.\nThe container runs a 90-line shell script:\nfor QUERY_FILE in $QUERY_FILES; do steampipe query \u0026#34;$QUERY_FILE\u0026#34; --output json \u0026gt; results.json aws sns publish --topic-arn \u0026#34;$SNS_TOPIC_ARN\u0026#34; --message \u0026#34;$NOTIFICATION\u0026#34; done Find SQL files. Run them. Send to SNS. That\u0026rsquo;s it.\nHere\u0026rsquo;s what the Slack notifications look like in action:\nReal use cases # To find public RDS instances:\nSELECT db_instance_identifier, engine, region FROM aws_rds_db_instance WHERE publicly_accessible = true; To find unused Elastic IPs:\nSELECT public_ip, allocation_id, region FROM aws_ec2_elastic_ip WHERE association_id IS NULL; Found four. Saved $144/year.\nCost breakdown # Here\u0026rsquo;s where it gets interesting:\nService Configuration Monthly Cost ECS Fargate 0.25 vCPU, 0.5 GB, 5 min/day $0.03 CloudWatch Logs Standard retention $0.01 SNS, EventBridge, ECR, Chatbot Standard usage Free tier Total $0.04 Four cents a month—that\u0026rsquo;s it.\nFour cents. I monitor EC2, S3, RDS, IAM, security groups, and EBS volumes across all regions for less than a penny per day.\nComplete working example # GitHub Repository: https://github.com/salsiy/steampipe-aws-monitor\nThe complete source code, infrastructure, and deployment guide are available in the repository above. It includes:\nFull Terraform modules (SNS, IAM, networking, ECS, Slack notifications) Ready-to-use SQL queries for common AWS checks Docker configuration for Steampipe EventBridge scheduling setup Step-by-step deployment guide If I left the team tomorrow, someone could understand and modify this in 30 minutes.\nWhat you could build # This works for any Steampipe plugin - same architecture, different SQL queries. Monitor GitHub repos, Kubernetes clusters, Azure resources, or any of the 140+ available plugins.\nBottom line # You don\u0026rsquo;t need expensive monitoring tools. You don\u0026rsquo;t need custom Lambda functions for every check. You don\u0026rsquo;t need to maintain boto3 code.\nWrite SQL. Run it on a schedule. Send results to Slack. Deploy with Terraform.\nTotal cost: 4 cents per month.\nSometimes the best solutions are the simple ones.\nFrequently asked questions # What AWS services does this monitoring system use? # The system uses six core AWS services:\nECS Fargate – Runs the containerized Steampipe engine EventBridge – Schedules daily tasks and captures task completion events SNS – Receives query results and forwards them to Chatbot AWS Chatbot – Sends formatted notifications to Slack CloudWatch Logs – Stores execution logs for debugging ECR – Hosts the Docker image How much does it cost to run this system each month? # Total monthly cost is $0.04, broken down as:\nECS Fargate: $0.03 (0.25 vCPU, 0.5 GB RAM, running 5 minutes per day) CloudWatch Logs: $0.01 (standard log retention) All other services: Free tier (SNS, EventBridge, ECR, Chatbot) For comparison, AWS Config would cost $2 per rule, and enterprise monitoring tools like Datadog start at $15 per host.\nCan I monitor multiple AWS regions or accounts? # Yes. Configure Steampipe with multiple AWS connection profiles:\nMultiple regions: The AWS plugin queries all regions by default Multiple accounts: Add assume-role configurations in steampipe.conf pointing to cross-account IAM roles Aggregated results: SQL queries can join data across accounts and regions in a single result set This makes it ideal for organizations managing dozens of AWS accounts.\nDo I need to write code to add a new check? # No. Adding a check is as simple as creating a .sql file in the queries/ folder:\n-- queries/check-unencrypted-ebs.sql SELECT volume_id, region FROM aws_ebs_volume WHERE encrypted = false; Rebuild the Docker image, push to ECR, and the next scheduled run will include your new check. No Python, no Lambda functions, no CloudFormation updates.\nHow do Slack notifications reach my channel? # The notification flow has three steps:\nSteampipe → SNS: Shell script publishes query results to an SNS topic SNS → AWS Chatbot: SNS topic triggers the Chatbot subscription Chatbot → Slack: Chatbot formats the message and posts it to your configured channel AWS Chatbot handles authentication via your workspace authorization. You invite @Amazon Q to the target channel during setup, and it delivers all future notifications automatically.\n","date":"12 October 2025","externalUrl":null,"permalink":"/posts/tech-logs/serverless-aws-monitoring-steampipe/","section":"All Posts","summary":"\u003cp\u003eLast month, I spent four hours tracking down why our AWS bill increased. The culprit? A few forgotten resources across multiple regions that no one remembered creating.\u003c/p\u003e\n\u003cp\u003eI realized we needed automated infrastructure monitoring, but every solution I looked at cost more than the resources we were trying to optimize. That\u0026rsquo;s when I decided to build something different.\u003c/p\u003e\n\n\n\u003ch2 class=\"relative group\"\u003eThe monitoring problem \n    \u003cdiv id=\"the-monitoring-problem\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#the-monitoring-problem\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cp\u003eMost AWS monitoring tools are either free but useless, or useful but expensive:\u003c/p\u003e","title":"Building a Serverless AWS Monitoring System for Less Than a Coffee","type":"posts"},{"content":"","date":"12 October 2025","externalUrl":null,"permalink":"/tags/ecs/","section":"Tags","summary":"","title":"Ecs","type":"tags"},{"content":"","date":"12 October 2025","externalUrl":null,"permalink":"/tags/fargate/","section":"Tags","summary":"","title":"Fargate","type":"tags"},{"content":"","date":"12 October 2025","externalUrl":null,"permalink":"/tags/monitoring/","section":"Tags","summary":"","title":"Monitoring","type":"tags"},{"content":"","date":"12 October 2025","externalUrl":null,"permalink":"/tags/slack/","section":"Tags","summary":"","title":"Slack","type":"tags"},{"content":"","date":"12 October 2025","externalUrl":null,"permalink":"/tags/steampipe/","section":"Tags","summary":"","title":"Steampipe","type":"tags"},{"content":"","date":"18 May 2025","externalUrl":null,"permalink":"/tags/cloud/","section":"Tags","summary":"","title":"Cloud","type":"tags"},{"content":" Overcoming AWS Tagging Limits in Terraform \u0026amp; Terragrunt # Recently I ran into an annoying AWS limitation while working with Terraform. Turns out S3 objects can only have a maximum of 10 tags, which becomes a problem when you\u0026rsquo;re using provider-level tagging. The Problem I like to use default tags at the provider level to make sure all\nprovider is defined here\nhttps://github.com/salsiy/terragrunt-examples/blob/master/root.hcl\nprovider \u0026#34;aws\u0026#34; { region = \u0026#34;${local.aws_region}\u0026#34; default_tags { tags = ${jsonencode(local.bucket_tags)} } } Tags for environment is defined here\nhttps://github.com/salsiy/terragrunt-examples/blob/master/dev/env.hcl\nlocals { environment = \u0026#34;dev\u0026#34; primary_region = \u0026#34;us-east-1\u0026#34; # S3 bucket with MORE than 10 tags (to test limitation) bucket_tags = { Environment = \u0026#34;dev\u0026#34; Application = \u0026#34;MyApp\u0026#34; Team = \u0026#34;Engineering\u0026#34; CostCenter = \u0026#34;Development\u0026#34; Owner = \u0026#34;hello@bohobot.com\u0026#34; DataClass = \u0026#34;Internal\u0026#34; Backup = \u0026#34;Daily\u0026#34; Monitoring = \u0026#34;Enabled\u0026#34; Compliance = \u0026#34;Standard\u0026#34; Version = \u0026#34;1.0\u0026#34; Purpose = \u0026#34;Testing\u0026#34; CreatedBy = \u0026#34;Terraform\u0026#34; } object_tags = { Environment = \u0026#34;dev\u0026#34; Type = \u0026#34;Config\u0026#34; Application = \u0026#34;MyApp\u0026#34; Version = \u0026#34;1.0\u0026#34; Owner = \u0026#34;Engineering\u0026#34; # Only 5 tags - well under the 10 tag limit for S3 objects } } This works great for most AWS resources since they support up to 50 tags. But S3 objects? They only support 10 tags maximum. So if I have 8 default tags and want to add a couple more specific tags to my S3 object, I hit the limit and get an error.\nThe Solution: Multiple Providers\nThe trick is to create a second AWS provider with fewer default tags, and use that specifically for S3 objects:\nprovider \u0026#34;aws\u0026#34; { alias = \u0026#34;secondary\u0026#34; region = \u0026#34;${local.aws_region}\u0026#34; default_tags { tags = ${jsonencode(local.object_tags)} } } Why This Works\nMost AWS resources support 50 tags, so the main provider works fine S3 objects get only the essential tags, staying under the 10-tag limit You still get consistent tagging across your infrastructure No need to remember which resources have tag limits\nComplete Example You can find the complete working example in my GitHub repo:\nterragrunt-examples/modules/s3-bucket-object-tag\nThis simple approach saved me from having to restructure my entire tagging strategy. Sometimes the best solutions are the simplest ones :)\n","date":"18 May 2025","externalUrl":null,"permalink":"/posts/tech-logs/aws-tagging-limit-terraform-terragrunt/","section":"All Posts","summary":"\u003ch1 class=\"relative group\"\u003eOvercoming AWS Tagging Limits in Terraform \u0026amp; Terragrunt \n    \u003cdiv id=\"overcoming-aws-tagging-limits-in-terraform--terragrunt\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#overcoming-aws-tagging-limits-in-terraform--terragrunt\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h1\u003e\n\u003cp\u003eRecently I ran into an annoying AWS limitation while working with Terraform. Turns out S3 objects can only have a maximum of 10 tags, which becomes a problem when you\u0026rsquo;re using provider-level tagging.\nThe Problem\nI like to use default tags at the provider level to make sure all\u003c/p\u003e","title":"Overcoming AWS Tagging Limits in Terraform \u0026 Terragrunt","type":"posts"},{"content":"","date":"18 May 2025","externalUrl":null,"permalink":"/tags/s3/","section":"Tags","summary":"","title":"S3","type":"tags"},{"content":"","date":"18 May 2025","externalUrl":null,"permalink":"/tags/tagging/","section":"Tags","summary":"","title":"Tagging","type":"tags"},{"content":"","date":"2 November 2024","externalUrl":null,"permalink":"/posts/","section":"All Posts","summary":"","title":"All Posts","type":"posts"},{"content":" Senior DevOps Engineer Warsaw, Poland\nProfessional Summary # Cloud and DevOps professional experienced in automating infrastructure, optimizing CI/CD pipelines, and implementing cloud-native solutions. Holder of AWS Solutions Architect – Professional and HashiCorp Terraform Associate certifications\nTechnical Expertise # Professional Experience # Senior DevOps Engineer | Aug 2021 – Present | Remote (Warsaw, Poland)\nAutomated AWS infrastructure provisioning using Terraform, reducing deployment times by 60% Platform DevOps Engineer (R\u0026D) | Sep 2020 – Aug 2021 | Bengaluru, India\nsigned multi-cloud Kubernetes platform for 50+ microservices Implemented CI/CD pipelines reducing deployment failures by 30% Certifications # AWS Solutions Architect – Professional\nValid until Jun 2027 HashiCorp Terraform Associate\nValid until Dec 2025 AWS Solutions Architect – Associate\nValid until Aug 2026 Education # Visvesvaraya Technological University\nBachelor of Engineering (Computer Science) | 2015 ","date":"20 November 2023","externalUrl":null,"permalink":"/resume/","section":"Resume | Siyad Salam","summary":"\u003cstyle\u003e\n  /* ===== GLOBAL STYLES ===== */\n  .resume-header h1 { \n    margin-bottom: 0; \n    color: #2c3e50;\n    font-size: 2.2em;\n  }\n  .subtitle { \n    color: #7f8c8d; \n    font-size: 1.2em;\n    margin-top: 0.3em;\n  }\n  .contact a { \n    color: #3498db; \n    margin-right: 12px;\n    text-decoration: none;\n  }\n\n  /* ===== LOGO STYLES ===== */\n  .resume-logo {\n    height: 24px;\n    width: auto;\n    vertical-align: middle;\n    margin-right: 8px;\n    position: relative;\n    top: -2px; /* Fine-tune alignment */\n  }\n  .cert-logo {\n    height: 32px;\n    margin-right: 10px;\n  }\n .tech-stack {\n  display: flex;\n  flex-wrap: wrap;\n  align-items: center;\n  gap: 28px; /* Wider spacing */\n  margin: 25px 0 35px; /* More vertical space */\n}\n  \n .tech-item {\n  display: flex;\n  align-items: center;\n}\n\n  /* ===== CERTIFICATION GRID ===== */\n  .certifications {\n    display: grid;\n    grid-template-columns: repeat(auto-fill, minmax(280px, 1fr));\n    gap: 16px;\n    margin: 20px 0;\n  }\n  .cert-item {\n    display: flex;\n    align-items: center;\n    padding: 12px;\n    background: #f8f9fa;\n    border-radius: 8px;\n  }\n\n  /* ===== EXPERIENCE SECTION ===== */\n  .experience {\n    margin-bottom: 2em;\n  }\n  .experience-header {\n    display: flex;\n    align-items: center;\n    margin-bottom: 0.5em;\n  }\n\u003c/style\u003e\n\u003cdiv class=\"resume-header\"\u003e\n  \u003cp class=\"subtitle\"\u003eSenior DevOps Engineer \u003c/p\u003e","title":"Resume | Siyad Salam","type":"resume"},{"content":"Tech enthusiast exploring the world of DevOps, living in Poland, proudly rooted in India.\n","date":"2 November 1991","externalUrl":null,"permalink":"/about/","section":"Welcome","summary":"\u003cp\u003eTech enthusiast exploring the world of DevOps, living in Poland, proudly rooted in India.\u003c/p\u003e","title":"About Me","type":"page"},{"content":" Connect With Me # Email\nLocation: Warsaw, Poland (originally from Kerala, India)\n","date":"2 November 1991","externalUrl":null,"permalink":"/contact/","section":"Welcome","summary":"\u003ch2 class=\"relative group\"\u003eConnect With Me \n    \u003cdiv id=\"connect-with-me\" class=\"anchor\"\u003e\u003c/div\u003e\n    \n    \u003cspan\n        class=\"absolute top-0 w-6 transition-opacity opacity-0 ltr:-left-6 rtl:-right-6 not-prose group-hover:opacity-100\"\u003e\n        \u003ca class=\"group-hover:text-primary-300 dark:group-hover:text-neutral-700\"\n            style=\"text-decoration-line: none !important;\" href=\"#connect-with-me\" aria-label=\"Anchor\"\u003e#\u003c/a\u003e\n    \u003c/span\u003e        \n    \n\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003e\n\n  \u003cspan class=\"relative inline-block align-text-bottom icon\"\u003e\n    \u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 512 512\"\u003e\u003cpath fill=\"currentColor\" d=\"M48 64C21.5 64 0 85.5 0 112c0 15.1 7.1 29.3 19.2 38.4L236.8 313.6c11.4 8.5 27 8.5 38.4 0L492.8 150.4c12.1-9.1 19.2-23.3 19.2-38.4c0-26.5-21.5-48-48-48H48zM0 176V384c0 35.3 28.7 64 64 64H448c35.3 0 64-28.7 64-64V176L294.4 339.2c-22.8 17.1-54 17.1-76.8 0L0 176z\"/\u003e\u003c/svg\u003e\n  \u003c/span\u003e\n\n \u003ca href=\"mailto:siyadsalam@gmail.com\"\u003eEmail\u003c/a\u003e\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003e\u003cstrong\u003eLocation\u003c/strong\u003e: Warsaw, Poland (originally from Kerala, India)\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ul\u003e","title":"Contact","type":"page"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"}]