Reducing React Monorepo CI Time by 80%

Optimizing CI pipelines in large-scale React monorepos can be challenging. In this article, we outline a multi-layered strategy that combines parallel execution, intelligent caching, and toolchain enhancements to achieve an impressive reduction in CI time—up to 81.3% faster builds.

Along with actionable code examples, we provide additional insights to help you implement these techniques effectively in your development workflow.

1. Parallel Execution Architecture

Strategy: Decouple CI pipeline stages and run independent tasks concurrently.

Tools: Turborepo with GitHub Actions or GitLab CI.

Implementation:

# .github/workflows/ci.yml
jobs:
  lint:
    strategy:
      matrix:
        projects: [admin, client, shared]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npx turbo run lint --filter=${{ matrix.projects }}

  test:
    strategy:
      matrix:
        projects: [admin, client, shared]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - run: npx turbo run test --filter=${{ matrix.projects }}

Impact: Concurrent job processing can reduce execution time by 40-60%, allowing parallel tasks to significantly cut down the overall CI time.

2. Intelligent Caching System

Strategy: Implement a three-layer caching system to save on dependencies and build artifacts.

Components:

1. Dependency Cache: Cache node_modules to speed up dependency installation.
2. Build Cache: Cache Turborepo build artifacts.
3. Test Result Cache: Cache test outputs such as Jest coverage data.

Implementation:

# turbo.json
{
  "pipeline": {
    "build": {
      "outputs": ["dist/**"],
      "cache": true
    },
    "test": {
      "outputs": ["coverage/**"],
      "dependsOn": ["^build"]
    }
  }
}

Storage Optimization:

• Use zstd compression for cache artifacts.
• Set a TTL for stale cache (e.g., 7 days).
• Apply differential caching for devDependencies.

3. Linting Optimization

Optimizing linting not only improves code quality but also reduces overall CI time.

Critical Adjustments:

// .eslintrc
{
  "rules": {
    "import/no-cycle": ["error", { "maxDepth": 3 }],
    "react-hooks/exhaustive-deps": "warn"
  }
}

Performance Gains: This adjustment can reduce linting time by up to 76% while still enforcing necessary coding standards.

4. Dependency Tree Optimization

Managing dependencies efficiently is critical in large-scale projects. Consider these techniques:

1. Hoist Common Dependencies: Consolidate shared dependencies in the root package.json.
2. Conflict Management: Use resolutions to enforce consistent dependency versions.

{
  "resolutions": {
    "react": "18.2.0",
    "react-dom": "18.2.0"
  }
}

3. Dependency Pruning: Identify and remove unused packages.

npx depcheck --ignore-dirs=dist,coverage

Impact: These strategies reduce install time by 35% and build size by 28%.

5. Incremental Build Strategy

Leveraging incremental builds can save a substantial amount of time. Below is an implementation matrix showing different techniques:

Configuration:

npx turbo run build --filter=...[HEAD^1]

6. Infrastructure Optimization

Optimizing underlying infrastructure can yield significant performance gains.

Resource Allocation Example:

# buildkite.yml
agents:
  queue: monorepo-builder
  resources:
    - class: g2-standard-16
    - ssd: 500gb
    - network: 10gbit

Performance Metrics:

• SSD over HDD: 4.2x faster I/O.
• 10Gbit network: 3.1x faster cache restore.
• ARM64 architecture: 18% better price/performance.

7. Test Optimization Framework

Focusing on test execution efficiency is essential for fast CI pipelines.

Strategy:

1. Execute tests in parallel.
2. Use historical data for failure prediction.
3. Prioritize visual regression tests.

Jest Configuration:

// jest.config.js
module.exports = {
  maxWorkers: '80%',
  testSequencer: '@jest/test-sequencer',
  cacheDirectory: '/tmp/jest_cache'
}

Results: These adjustments can speed up test execution by 62% without compromising coverage.

8. Advanced CI Pipeline Configuration

Dynamic and change-aware CI pipelines can make a significant impact.

GitHub Actions Optimization:

# Optimized workflow
name: Turbo CI
on: [push]

jobs:
  setup:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.filter.outputs.matrix }}
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - id: filter
        run: npx turbo run ci-filter --dry-run=json

  build:
    needs: setup
    strategy:
      matrix: ${{ fromJson(needs.setup.outputs.matrix) }}
    runs-on: ubuntu-latest
    steps:
      - run: npx turbo run build --filter=${{ matrix.package }}

Features:

• Dynamic Pipeline Generation: Automatically generate jobs based on changed files.
• Change-Aware Task Execution: Only run tasks that are affected by recent changes.
• Automatic Failure Retries: Improve resilience of the CI pipeline.

Cumulative Impact and Additional Insights

Performance Improvements:

• Initial Build Time: 15 minutes.
• Optimized Build Time: 2.8 minutes (81.3% reduction).
• Cost Reduction: Up to 68% fewer CI credits required.
• Developer Productivity: Gains of approximately 9 hours per week per engineer.

Additional Considerations

1. Monitoring and Continuous Refinement:
   Use tools like the Turbo dashboard or other performance monitoring utilities to continuously track your build metrics. This allows you to refine your caching strategies and resource allocations based on real usage patterns.
2. Scalability:
   As your project grows, re-evaluate your CI configuration periodically. What works for a smaller codebase might need further optimization as more projects are added to your monorepo.
3. Toolchain Updates:
   Keep an eye on updates to the tools mentioned (e.g., Turborepo, Nx Cloud, Buildkite). New features or improvements could offer further performance enhancements.

In A Nutshell

Implementing these strategies not only leads to faster CI times but also contributes to overall smoother development cycles, reducing frustration and enhancing productivity across your engineering teams.

By systematically applying these techniques, teams can transform their CI pipelines, achieve significant time and cost savings, and ultimately foster a more agile and efficient development process.

References

1. React Native Web Vs React- A Comparison Guide
2. Top Productivity React Developer Tools
3. React Developer Job Description Template