CI/CD Has Evolved Beyond Build-Test-Deploy
In 2016, a CI/CD pipeline ran tests and deployed code. In 2026, an enterprise pipeline is a comprehensive quality, security, and compliance gateway that enforces organisational standards automatically. DORA's 2025 Accelerate report shows that elite performers run 15-20 automated checks per pipeline and still achieve sub-hour lead times from commit to production.
This guide covers the pipeline architecture and practices that we implement for enterprise clients. The goal: every merge to main is production-ready, and deploying to production is a non-event.
The Modern Enterprise Pipeline Architecture
A production-grade pipeline has five stages, each with specific checks:
# GitHub Actions - Enterprise pipeline structure
name: Production Pipeline
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
# Stage 1: Code Quality
lint-and-format:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run lint
- run: npm run format:check
- run: npx tsc --noEmit # Type checking
# Stage 2: Testing
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm test -- --coverage
- uses: codecov/codecov-action@v4
integration-tests:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:16
env:
POSTGRES_DB: test
POSTGRES_PASSWORD: test
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run test:integration
# Stage 3: Security
security-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: SAST (Static Analysis)
uses: github/codeql-action/analyze@v3
- name: Dependency Audit
run: npm audit --audit-level=high
- name: Secret Detection
uses: trufflesecurity/trufflehog@main
- name: Container Scan
uses: aquasecurity/trivy-action@master
with:
scan-type: 'image'
severity: 'CRITICAL,HIGH'
# Stage 4: Build and Preview
build:
needs: [lint-and-format, unit-tests, integration-tests, security-scan]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npm run build
- name: Build Container Image
run: docker build -t app:${{ github.sha }} .
- name: Infrastructure Cost Estimate
uses: infracost/actions/setup@v3
- run: infracost diff --path=./terraform
# Stage 5: Deploy
deploy-staging:
needs: [build]
if: github.ref == 'refs/heads/main'
environment: staging
runs-on: ubuntu-latest
steps:
- name: Deploy to staging
run: ./deploy.sh staging ${{ github.sha }}
- name: Smoke tests
run: npm run test:smoke -- --env=staging
deploy-production:
needs: [deploy-staging]
if: github.ref == 'refs/heads/main'
environment: production
runs-on: ubuntu-latest
steps:
- name: Canary deploy (10%)
run: ./deploy.sh production ${{ github.sha }} --canary=10
- name: Monitor canary (5 min)
run: ./monitor-canary.sh --duration=300 --error-threshold=1
- name: Full rollout
run: ./deploy.sh production ${{ github.sha }} --canary=100Best Practice 1: Trunk-Based Development
Long-lived feature branches are the enemy of continuous integration. By definition, if you have branches that live for weeks, you do not have CI — you have "intermittent integration."
Adopt trunk-based development:
- All engineers merge to main at least daily
- Feature branches live <24 hours
- Use feature flags for work-in-progress features, not long-lived branches
- PRs are small (<400 lines of diff) and focused on a single concern
DORA research shows that trunk-based development is one of the strongest predictors of elite performance. Teams using trunk-based development deploy 3x more frequently with lower change failure rates than teams using long-lived branches.
Best Practice 2: Pipeline as Code, Reviewed Like Code
Your pipeline definition is infrastructure. Treat it with the same rigour as application code:
- Pipeline changes go through code review
- Pipeline configurations are versioned alongside application code
- Shared pipeline components are published as reusable actions/templates
- Pipeline changes are tested in a staging pipeline before applying to production
Best Practice 3: Shift Security Left (But Not All the Way)
Security scanning should run in the pipeline, but not every security check belongs in every pipeline run:
| Check | When to Run | Block on Failure? |
|---|---|---|
| Secret detection | Every commit | Yes — always |
| Dependency audit (critical/high CVEs) | Every PR | Yes |
| SAST (static analysis) | Every PR | Yes for new findings |
| Container image scan | Every build | Yes for critical CVEs |
| DAST (dynamic analysis) | Post-deploy to staging | Block production deploy |
| License compliance | Weekly scheduled | Alert, don't block |
| Full penetration test | Quarterly | Manual review |
Best Practice 4: Fast Feedback Loops
If your pipeline takes 30 minutes, engineers will batch changes and deploy less frequently. Speed matters:
- Target: <10 minutes for the full PR pipeline (lint + test + security scan + build)
- Parallelise independent stages (tests, linting, and security scans can all run simultaneously)
- Cache aggressively: dependency caches (npm, pip), Docker layer caches, build artifact caches
- Test splitting: distribute test suites across parallel runners using tools like Jest's
--shardor pytest-split - Selective testing: in monorepos, only run tests for changed packages and their dependents
# GitHub Actions - Aggressive caching example
- uses: actions/cache@v4
with:
path: |
~/.npm
node_modules
.next/cache
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
restore-keys: ${{ runner.os }}-node-
# Parallel test sharding
test:
strategy:
matrix:
shard: [1, 2, 3, 4]
steps:
- run: npx jest --shard=${{ matrix.shard }}/4Best Practice 5: Progressive Deployment
Deploying 100% of traffic to a new version simultaneously is unnecessarily risky. Use progressive deployment:
- Canary: Deploy to 5-10% of traffic, monitor error rates and latency for 5-15 minutes
- Automated rollback: If error rate or latency exceeds thresholds during canary, automatically roll back
- Progressive rollout: 10% to 25% to 50% to 100%, with monitoring between each step
Tools: Argo Rollouts (Kubernetes), AWS CodeDeploy, Flagger, or custom scripts monitoring your APM tool.
Best Practice 6: Infrastructure Cost Gates
A 2026 addition to enterprise pipelines: estimate the cost impact of infrastructure changes before they are applied.
- Use Infracost to calculate the monthly cost delta of Terraform changes
- Post cost estimates as PR comments so reviewers can see the financial impact
- Set thresholds: changes that increase monthly costs by more than EUR 500 require platform team approval
Best Practice 7: Observability-Driven Deployment
Your deployment pipeline should consume observability data to make decisions:
- Post-deploy smoke tests verify critical user flows in the deployed environment
- Canary analysis compares error rates and latency between canary and baseline
- Deployment annotations in your monitoring tool (Datadog, Grafana) correlate deployments with metric changes
- Automated rollback triggers if key SLOs are breached within 15 minutes of deployment
Anti-Patterns to Avoid
- Manual approval gates for every deployment: If you require manual approval for routine production deployments, you do not trust your pipeline. Fix the pipeline instead.
- Environment-specific branches: (dev, staging, production branches). Use one branch (main) and promote the same artefact through environments.
- Building different artefacts per environment: Build once, deploy everywhere. Use environment variables for configuration, not separate builds.
- Ignoring flaky tests: A test suite with flaky tests is worse than no tests — it teaches engineers to ignore failures. Fix or remove flaky tests immediately.
- No pipeline metrics: Track pipeline duration, success rate, and MTTR for pipeline failures. Treat the pipeline as a product with SLOs.
Build Your Pipeline Right
If your CI/CD pipeline is a bottleneck — slow, unreliable, or missing critical checks — our DevOps consulting team builds enterprise-grade pipelines that teams actually trust. Get in touch for a pipeline architecture review.
