Back to All Concepts
DevOpsCICDAutomationDeploymentIntermediate

CI/CD Pipeline Architecture

Designing robust Continuous Integration and Continuous Deployment pipelines. Strategies for artifact promotion, testing pyramids, canary deployments, and rollback mechanisms.

The Goal

Continuous Integration (CI): Merge code changes frequently. Automated tests verify correctness. Continuous Deployment (CD): Automatically release changes to production if tests pass.

Goal: Reduce Mean Time To Recovery (MTTR) and Increase Deployment Frequency.

Interactive: The Pipeline Flow

CI/CD Pipeline Flow

Commit
Build
Test
Deploy
Waiting for changes... Click 'Commit' to start.

The Pipeline Anatomy

A modern pipeline isn't just "build and deploy". It's a series of quality gates.

mermaid
graph LR
    Dev[Developer Commit] --> CI[CI: Build & Test]
    CI --> Art[Artifact Registry]
    Art --> Staging[CD: Deploy to Staging]
    Staging --> E2E[E2E Tests]
    E2E --> Prod[CD: Deploy to Prod]
    Prod --> Monitor[Observability]
Click to expand code...

1. Build & Test (CI)

Triggers: Pull Request, Git Push. Tasks:

  • Linting (ESLint, Pylint)
  • Static Analysis (SonarQube)
  • Unit Tests (Jest, PyTest)
  • Build Container Image (Docker)

[!IMPORTANT] Ideally < 10 minutes. Slow CI kills developer velocity.

2. Artifact Promotion

Don't rebuild for every environment! Build once, deploy everywhere.

  1. CI builds Docker Image app:v1.0.0-sha123.
  2. Push to Registry (ECR/DockerHub) as app:v1.0.0-rc.
  3. Deploy app:v1.0.0-rc to Staging.
  4. If successful, retag as app:v1.0.0-stable and deploy to Prod.

Why? Ensures the exact binary tested in Staging goes to Prod.

Deployment Strategies

How do we update production without downtime?

Blue-Green Deployment

  • Method: Spin up a complete new environment (Green) alongside old (Blue).
  • Cutover: Switch Load Balancer to Green.
  • Rollback: Switch LB back to Blue instantly.
  • Cost: High (Double infrastructure resources).
mermaid
graph TD
    Traffic[User Traffic] --> LB{Load Balancer}
    
    subgraph Active
        LB -->|100%| V1[Blue: v1.0]
    end
    
    subgraph Idle
        V2[Green: v2.0]
    end
Click to expand code...

Canary Deployment

  • Method: Route small % of traffic (1-5%) to new version.
  • Validation: Monitor error rates/latency.
  • Rollout: Gradually increase traffic (1% -> 10% -> 50% -> 100%).
  • Risk: Lowest. Bugs only affect a small subset of users.

Rolling Update (Kubernetes Default)

  • Method: Replace instances one by one. maxUnavailable: 1, maxSurge: 1.
  • Cost: Low (No extra infra).
  • Risk: Harder to rollback instantly.

Pipeline as Code (GitHub Actions)

Defining pipelines in YAML allows version control of the build process itself.

yaml
name: Production Build

on:
  push:
    branches: [ "main" ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js
        uses: actions/setup-node@v3
        with:
          node-version: '18.x'
      - run: npm ci
      - run: npm test

  build-push:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Build Docker Image
        run: docker build -t myapp:${{ github.sha }} .
      
      - name: Push to Registry
        run: |
          echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u user --password-stdin
          docker push myapp:${{ github.sha }}

  deploy-staging:
    needs: build-push
    environment: staging
    steps:
      - name: Update K8s Manifest
        run: |
          sed -i "s|image: .*|image: myapp:${{ github.sha }}|" k8s/deployment.yaml
          kubectl apply -f k8s/
Click to expand code...

Testing Pyramid

Automated tests are the safety net of CI/CD.

TypeSpeedCostQuantityExample
UnitFast (ms)LowManyadd(1, 2) == 3
IntegrationMedium (s)MedSomeAPI calls Database
E2E (UI)Slow (min)HighFewSelenium login flow

Feature Flags

Decouple Deployment (technical) from Release (business).

  1. Deploy code with feature disabled: if (features.enableNewUI) { ... }
  2. Enable flag for internal users.
  3. Enable flag for 10% of users.
  4. Enable for everyone.

Tools: LaunchDarkly, Split.io, Unleash.

Interview Tips šŸ’”

  • "How do you handle schema changes?" — Backward compatible migrations. Add column -> Deploy Code -> Fill Column -> Remove old code.
  • "What if the deployment fails?" — Automated Rollback based on health checks.
  • "Immutable Infrastructure" — Don't patch servers. Replace them (Terraform + Packer).
  • "GitOps" — Infrastructure updates via Pull Request (ArgoCD).

Related Concepts

About ScaleWiki

ScaleWiki is an interactive educational platform dedicated to demystifying distributed systems, software architecture, and system design. Our mission is to provide high-quality, technically accurate resources for software engineers preparing for interviews or solving complex scaling challenges in production.

Read more about our Editorial Guidelines & Authorship.

Educational Disclaimer: The architectural patterns and system designs discussed in this article are based on common industry practices, technical whitepapers, and public engineering blogs. Actual implementations in enterprise environments may vary significantly based on specific product requirements, legacy constraints, and evolving technologies.

Related Articles