DevOps

Securing CI/CD Pipelines: supply‑chain best practices

Securing CICD Pipelines Supply‑chain Best Practices

Introduction

Modern CI/CD pipelines automate everything from building to testing to deploying code. This automation accelerates development but also introduces new attack surfaces. If your build environment or dependencies are compromised, attackers can silently inject malicious code into production. High-profile incidents like SolarWinds, Codecov, and the XZ Utils backdoor demonstrate that supply-chain attacks are not theoretical threats but active concerns for every development team.

In this comprehensive guide, you will learn how to identify common CI/CD security risks, implement defense-in-depth strategies, and adopt industry frameworks that protect your software supply chain. By the end, you will have practical techniques to secure every stage of your pipeline from code commit to production deployment.

Why CI/CD Security Matters

CI/CD pipelines connect every part of your delivery process: source code repositories, build servers, package registries, deployment credentials, and production environments. A single weak link can compromise your entire product and every customer who uses it.

  • Direct path to production: Pipelines have the credentials and permissions to deploy anywhere
  • Trust amplification: Code that passes CI/CD is implicitly trusted by downstream systems
  • Lateral movement: Compromised pipelines can access secrets, databases, and cloud infrastructure
  • Supply chain reach: If you publish libraries, your compromise becomes your consumers’ compromise
  • Stealth attacks: Build-time injections are harder to detect than runtime attacks

The SolarWinds attack demonstrated this perfectly: attackers compromised the build system and injected malware that shipped to 18,000 customers through legitimate update channels.

Common Threats in CI/CD Pipelines

Understanding attack vectors helps you prioritize defenses. These are the most common and dangerous threats.

Compromised Dependencies

Attackers inject malicious code through external packages using typosquatting, account takeover, or dependency confusion attacks. Once installed, these packages execute during builds with full pipeline permissions.

Leaked Secrets

API keys, tokens, and credentials exposed in logs, environment variables, or repositories give attackers direct access to your infrastructure. Secret sprawl across multiple systems makes this problem worse.

Malicious Pull Requests

Pull requests from forks or external contributors may contain hidden payloads that execute during CI builds. Workflow files themselves can be modified to exfiltrate secrets.

Insecure Build Runners

Shared or poorly isolated build servers can leak data between jobs. Persistent runners accumulate secrets and artifacts from previous builds.

Tampered Artifacts

Unsigned or unverified binaries, container images, or packages can be replaced anywhere between build and deployment. Without verification, you deploy whatever arrives.

Securing Source Code and Repositories

Security begins before code enters the pipeline. Protect your repositories as the foundation of your supply chain.

Branch Protection Rules

# GitHub branch protection configuration
name: main
protection:
  required_pull_request_reviews:
    required_approving_review_count: 2
    dismiss_stale_reviews: true
    require_code_owner_reviews: true
  required_status_checks:
    strict: true
    contexts:
      - security-scan
      - unit-tests
      - integration-tests
  enforce_admins: true
  required_signatures: true
  allow_force_pushes: false
  allow_deletions: false

Commit Signing

Require GPG or SSH signatures on all commits to verify author identity. This prevents impersonation attacks where attackers push commits appearing to come from trusted developers.

# Configure Git to sign commits
git config --global commit.gpgsign true
git config --global user.signingkey YOUR_KEY_ID

# Verify commits
git log --show-signature

Pre-Commit Security Hooks

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: detect-private-key
      - id: check-added-large-files
  - repo: https://github.com/zricethezav/gitleaks
    rev: v8.18.0
    hooks:
      - id: gitleaks

Protecting Secrets

Secrets management is critical. Never hardcode credentials in repositories, and minimize secret exposure throughout your pipeline.

Use Dedicated Secrets Management

# GitHub Actions with OIDC for AWS (no stored credentials)
jobs:
  deploy:
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789:role/GitHubActionsRole
          aws-region: us-east-1
          # No AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY needed!

Secret Management Best Practices

  • Use OIDC federation: Eliminate stored credentials by using identity federation with cloud providers
  • Scope secrets narrowly: Each secret should only be accessible to jobs that need it
  • Rotate regularly: Automate credential rotation on a fixed schedule
  • Audit access: Log every secret access and review anomalies
  • Use short-lived tokens: Prefer tokens that expire in hours, not months
# HashiCorp Vault dynamic secrets example
vault read aws/creds/deploy-role
# Returns temporary credentials that auto-expire
# Key             Value
# lease_id        aws/creds/deploy-role/abcd1234
# lease_duration  1h
# access_key      AKIAIOSFODNN7EXAMPLE
# secret_key      wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Securing Dependencies

Dependencies are the most common attack vector in supply chain compromises. Implement multiple layers of defense.

Pin Exact Versions

# package.json - pin exact versions
{
  "dependencies": {
    "express": "4.18.2",
    "lodash": "4.17.21"
  }
}

# Use lockfiles and verify integrity
npm ci --ignore-scripts  # Install from lockfile, skip postinstall scripts

Automated Vulnerability Scanning

# GitHub Actions dependency scanning
name: Security Scan
on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: '0 6 * * *'  # Daily scan

jobs:
  dependency-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'

      - name: Run Snyk to check for vulnerabilities
        uses: snyk/actions/node@master
        env:
          SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
        with:
          args: --severity-threshold=high

Private Package Registries

Host private mirrors of public registries to prevent dependency confusion attacks and control what packages enter your builds.

# .npmrc - use private registry with scoped packages
@mycompany:registry=https://npm.mycompany.com/
//npm.mycompany.com/:_authToken=${NPM_TOKEN}

# Block installation of unscoped packages from public registry
registry=https://npm.mycompany.com/

Isolating Build Environments

Each pipeline job should run in a fully isolated environment that cannot leak data to other jobs or persist between runs.

Use Ephemeral Runners

# GitHub Actions - ephemeral self-hosted runners
runs-on: [self-hosted, ephemeral]

# Or use GitHub-hosted runners (always fresh)
runs-on: ubuntu-latest

Container-Based Isolation

# GitLab CI with isolated containers
build:
  image: node:20-alpine
  script:
    - npm ci
    - npm run build
  variables:
    # Prevent network access during build
    FF_NETWORK_PER_BUILD: "true"
  tags:
    - docker
    - isolated

Runner Security Checklist

  • Use ephemeral runners that reset after each job
  • Never share runners between untrusted repositories
  • Disable outbound network access when possible
  • Run builds in unprivileged containers
  • Clear caches and temporary files between jobs

Signing and Verifying Artifacts

Code signing provides a cryptographic chain of trust from build to deployment. Use Sigstore and Cosign for container images.

# Sign container image with Cosign
cosign sign --key cosign.key myregistry.com/myapp:v1.0.0

# Verify before deployment
cosign verify --key cosign.pub myregistry.com/myapp:v1.0.0

# Keyless signing with GitHub Actions OIDC
- name: Sign image with Cosign
  uses: sigstore/cosign-installer@v3
- run: cosign sign --yes ${{ env.IMAGE_NAME }}@${{ env.IMAGE_DIGEST }}

Generate Software Bill of Materials (SBOM)

# Generate SBOM with Syft
syft myregistry.com/myapp:v1.0.0 -o spdx-json > sbom.json

# Attach SBOM to image
cosign attach sbom --sbom sbom.json myregistry.com/myapp:v1.0.0

Complete Secure Pipeline Example

# .github/workflows/secure-pipeline.yml
name: Secure CI/CD Pipeline

on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  id-token: write
  security-events: write

jobs:
  security-scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Gitleaks
        uses: gitleaks/gitleaks-action@v2
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Dependency Review
        uses: actions/dependency-review-action@v3
        if: github.event_name == 'pull_request'

      - name: Run Trivy
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          severity: 'CRITICAL,HIGH'
          exit-code: '1'

  build:
    needs: security-scan
    runs-on: ubuntu-latest
    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4

      - name: Build and push image
        id: build
        uses: docker/build-push-action@v5
        with:
          push: true
          tags: myregistry.com/myapp:${{ github.sha }}

      - name: Generate SBOM
        uses: anchore/sbom-action@v0
        with:
          image: myregistry.com/myapp:${{ github.sha }}

      - name: Sign image
        uses: sigstore/cosign-installer@v3
      - run: cosign sign --yes myregistry.com/myapp@${{ steps.build.outputs.digest }}

  deploy:
    needs: build
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ vars.AWS_ROLE_ARN }}
          aws-region: us-east-1

      - name: Verify image signature
        run: cosign verify myregistry.com/myapp@${{ needs.build.outputs.digest }}

      - name: Deploy to ECS
        run: |
          aws ecs update-service --cluster prod --service myapp \
            --force-new-deployment

Following the SLSA Framework

SLSA (Supply-chain Levels for Software Artifacts) provides a security framework with four levels of increasing protection. Aim for at least SLSA Level 2 for production systems.

  • Level 1: Documentation of build process, any build system
  • Level 2: Signed provenance, hosted build platform
  • Level 3: Hardened build platform, non-falsifiable provenance
  • Level 4: Hermetic builds, two-person review

Monitoring and Incident Response

Security requires continuous monitoring. Log everything and alert on anomalies.

  • Log all pipeline executions: Who triggered, what changed, what was deployed
  • Alert on unusual patterns: Builds outside business hours, new contributors, credential access
  • Retain logs immutably: Store in append-only systems that attackers cannot modify
  • Practice incident response: Know how to revoke credentials and roll back deployments quickly

Conclusion

A CI/CD pipeline is only as strong as its weakest stage. By enforcing least privilege access, protecting secrets with OIDC and vault solutions, verifying dependencies, isolating build environments, and signing all artifacts, you build a resilient delivery process that attackers cannot easily compromise. Continuous monitoring ensures that potential issues are caught before they reach production.

For deeper coverage of secrets management tools, read Secrets Management: Comparing Vault, AWS KMS and Other Tools. To understand authentication patterns that secure your APIs and services, see OAuth2, JWT, and Session Tokens Explained. For containerization security in your pipelines, explore Docker and Kubernetes Essentials for Developers. Reference the official SLSA framework documentation and the Sigstore documentation for the latest supply chain security practices.

1 Comment

Leave a Comment