All docs/CI/CD & Testing

docs/architecture/development-workflow.md

Development Workflow Architecture

Overview

This document explains how the development environment works in this Turborepo monorepo, including how packages are built, watched, and hot-reloaded during development.

Key Technologies

ToolPurposeConfiguration
TurborepoTask orchestration & dependency graphturbo.json
tsupTypeScript bundling for packagespackages/*/tsup.config.ts
Next.jsApp framework with hot reloadapps/*/next.config.mjs
dev.tsCustom dev script with presetspackages/ci-scripts/src/dev.ts

The Dev Command Flow

Running Development

# Interactive menu
pnpm dev

# With preset
pnpm dev --all
pnpm dev --admin
pnpm dev --library

What Happens Step-by-Step

┌─────────────────────────────────────────────────────────────────┐
│ 1. pnpm dev --admin                                             │
│    └─> runs: tsx packages/ci-scripts/src/dev.ts --admin         │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 2. dev.ts constructs command:                                   │
│    turbo run dev --concurrency=32 --filter=admin^...            │
│                                   --filter=dev-assets^...       │
│                                   --filter=admin                │
│                                   --filter=dev-assets           │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 3. Turbo's task graph (automatic via dependsOn: ["^build"])     │
│    For each dev task, Turbo first runs build on dependencies:   │
│    └─> admin#dev depends on ^build → builds @repo/core, etc.    │
│    └─> Packages built in correct order via dependency graph     │
│    └─> Cached if unchanged (fast on subsequent runs)            │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│ 4. PHASE 2: Start dev servers (all parallel)                    │
│    turbo run dev --concurrency=32 ...                           │
│    │                                                            │
│    ├─> Apps run: next dev (hot reload server)                   │
│    │   • admin (port 3007)                                      │
│    │   • dev-assets (port 3010)                                 │
│    │                                                            │
│    └─> Packages run: tsup --watch                               │
│        • @repo/core, @repo/video, @repo/storage, etc.           │
│        • Watches source files, rebuilds dist/ on change         │
└─────────────────────────────────────────────────────────────────┘

Turbo.json Configuration

Task Definitions

{
  "tasks": {
    "build": {
      "dependsOn": ["^build"],           // Build dependencies first
      "outputs": ["dist/**", ".next/**"] // Cache these outputs
    },
    "dev": {
      "dependsOn": ["^build"],           // Build dependencies before starting dev
      "persistent": true,                 // Long-running (never exits)
      "cache": false                      // Don't cache dev servers
    }
  }
}

Key Concepts

ConceptSyntaxMeaning
Dependency build^buildBuild my dependencies' build task first
Persistentpersistent: trueTask runs forever (dev servers)
No cachecache: falseAlways run, never use cached result

Turbo Filter Syntax

SyntaxMeaningExample
--filter=appJust this package--filter=admin
--filter=app...Package + all dependencies--filter=admin...
--filter=app^...All dependencies (NOT the package itself)--filter=admin^...
--filter=@repo/*All packages matching scopeAll @repo/* packages

How dependsOn: ["^build"] works:

When turbo runs admin#dev:

  1. It sees dependsOn: ["^build"] in turbo.json
  2. It finds all packages admin depends on (via package.json)
  3. It runs build on those packages first (respecting their own ^build dependencies)
  4. Only then starts admin#dev

This is more efficient than manually building because turbo only builds what's needed!

Why ^build but not ^dev?

// ❌ WRONG - causes Turborepo error
"dev": {
  "dependsOn": ["^dev"],    // Can't depend on persistent tasks!
  "persistent": true
}

// ✅ CORRECT - depend on ^build (non-persistent), not ^dev (persistent)
"dev": {
  "dependsOn": ["^build"],  // Build dependencies first, then start dev
  "persistent": true,
  "cache": false
}

Rule: Persistent tasks cannot depend on OTHER persistent tasks (they never complete).

But: Persistent tasks CAN depend on non-persistent tasks like build.

What happens:

  1. admin#dev has dependsOn: ["^build"]
  2. Turbo runs build on all packages admin depends on
  3. Once builds complete, admin#dev starts (next dev)
  4. Meanwhile, packages' dev tasks also start (tsup --watch)

Package Build Strategy

tsup Configuration

Packages use tsup with bundling enabled (default):

// packages/*/tsup.config.ts
export default defineConfig({
  entry: ["src/index.ts", "src/server/index.ts"],  // Explicit entries
  format: ["esm"],
  dts: false,                                       // Types from source
  outDir: "dist"
  // Bundling enabled by default - handles internal relative imports
});

Node.js Apps use noExternal to force bundling of @repo/* packages:

// apps/*/tsup.config.ts (video-processor, sync-worker)
export default defineConfig({
  entry: ["src/index.ts"],
  format: ["esm"],
  bundle: true,
  noExternal: [/^@repo\/.*/],  // CRITICAL: Forces bundling of all @repo/*
  external: ["express", "cors", "multer", "dotenv"],
});

See tsup-configuration-patterns.md for details on why noExternal is required.

Build vs Dev Scripts

// packages/*/package.json
{
  "scripts": {
    "build": "tsup",              // One-time build to dist/
    "dev": "tsup --watch"         // Watch mode, rebuilds on change
  }
}

Watch Mode Flow

┌─────────────────────┐    ┌─────────────────────┐    ┌─────────────────────┐
│  Source File        │    │  tsup --watch       │    │  Next.js App        │
│  packages/core/     │───▶│  Detects change     │───▶│  Hot reloads        │
│  src/utils.ts       │    │  Rebuilds dist/     │    │  Reflects change    │
└─────────────────────┘    └─────────────────────┘    └─────────────────────┘

Package Exports Pattern

How Apps Import Packages

// In apps/admin/...
import { something } from "@repo/core";

Package.json Exports

{
  "name": "@repo/core",
  "exports": {
    ".": {
      "types": "./src/index.ts",      // TypeScript reads from source
      "import": "./dist/index.js"     // Runtime reads from dist/
    }
  }
}

Why This Works

  1. Types: TypeScript resolves types from src/*.ts (no build needed for typecheck)
  2. Runtime: Node.js loads from dist/*.js (requires build first)
  3. Watch: tsup rebuilds dist/ on change, Next.js hot reloads

Dev Presets

Available Presets

const PRESETS = {
  all: ["library", "branded-video-flow", "video-processor", "dev-assets", "hello", "demo"],
  core: ["library", "branded-video-flow", "video-processor", "dev-assets"],
  library: ["library", "video-processor", "dev-assets"],
  admin: ["admin", "dev-assets"],
  // ... more presets
};

Port Allocation

AppPortDescription
library3000Main web app
branded-video-flow3001Video flow app
video-processor3002Express API
sync-worker3004Sync worker
links3005Links app
hello3006Hello app
admin3007Admin app
dev-assets3010Local asset server
kanban3011Kanban board
segment-evaluation3012Evaluation app
demo3013Demo app

Troubleshooting

"ERR_MODULE_NOT_FOUND"

Cause: Package dist/ not built, or tsup misconfigured.

Fix:

# Rebuild all packages
pnpm turbo build --filter=@repo/*

# Or clean and rebuild
pnpm turbo clean && pnpm turbo build

"dist/ is stale"

Cause: Watch mode not running, or changed files not detected.

Fix:

# Delete dist folders and restart dev
rm -rf packages/*/dist
pnpm dev --admin

Changes Not Hot Reloading

Cause: Next.js not watching the package, or transpilePackages missing.

Fix: Ensure the package is in transpilePackages in next.config.mjs:

// apps/admin/next.config.mjs
const nextConfig = {
  transpilePackages: [
    "@repo/core",
    "@repo/app-admin-library",
    // ... add missing packages
  ],
};

Persistent Task Dependency Error

Cause: turbo.json has ^dev in dependsOn for a persistent task.

Fix: Persistent tasks cannot depend on other persistent tasks. Remove ^dev:

// ❌ Wrong - persistent tasks can't be dependencies
"dev": { "dependsOn": ["^dev"], "persistent": true }

// ✅ Correct - no dependsOn for persistent tasks
"dev": { "persistent": true, "cache": false }

The dev.ts script handles the build phase separately before starting dev servers.


Architecture Diagram

                    ┌─────────────────────────────────────────────────┐
                    │                 pnpm dev --admin                │
                    └────────────────────────┬────────────────────────┘
                                             │
                    ┌────────────────────────▼────────────────────────┐
                    │              dev.ts (CI Scripts)                │
                    │  • Parses preset (--admin, --all, etc.)         │
                    │  • Runs: turbo run dev --filter=admin^...       │
                    │  • Manages cleanup on exit                      │
                    └────────────────────────┬────────────────────────┘
                                             │
                    ┌────────────────────────▼────────────────────────┐
                    │              Turbo Task Graph                   │
                    │  turbo.json: dev.dependsOn = ["^build"]         │
                    │  ┌─────────────────────────────────────────┐    │
                    │  │ 1. Build dependencies first (^build)   │    │
                    │  │    @repo/core, @repo/design-system...  │    │
                    │  │    (cached if unchanged)               │    │
                    │  └─────────────────┬───────────────────────┘    │
                    │                    │                            │
                    │  ┌─────────────────▼───────────────────────┐    │
                    │  │ 2. Start dev tasks (parallel)          │    │
                    │  │    admin#dev, dev-assets#dev, etc.     │    │
                    │  └─────────────────────────────────────────┘    │
                    └────────────────────────┬────────────────────────┘
                                             │
            ┌────────────────────────────────┼────────────────────────────────┐
            │                                │                                │
            ▼                                ▼                                ▼
    ┌───────────────┐                ┌───────────────┐                ┌───────────────┐
    │   Apps        │                │   Packages    │                │   Assets      │
    │               │                │               │                │               │
    │  next dev     │◀──────────────▶│  tsup --watch │                │  Express      │
    │  Port 3007    │   Hot Reload   │  Rebuild dist │                │  Port 3010    │
    │               │                │               │                │               │
    │  admin        │                │  @repo/core   │                │  dev-assets   │
    └───────────────┘                │  @repo/video  │                └───────────────┘
                                     │  @repo/...    │
                                     └───────────────┘

Related Documentation