┌─────────────────┐
│   API Route     │  ← HTTP requests trigger workflows
│  /api/workflow  │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  Workflow       │  ← Workflow definitions and registry
│  Registry       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│  OpenWorkflow   │  ← Core workflow engine
│  Instance       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│   PostgreSQL    │  ← Workflow state storage
│   Backend       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│     Worker      │  ← Executes workflow steps
│   (20 concurrent)│
└─────────────────┘
```typescript

## Components

### File Structure

```files
workflows/
├── registry.ts
├── workflows.ts
├── instance.ts
├── worker.ts
└── schema.sql
apps/
└── dashboard/
    └── app/
        └── api/
            └── workflow/
                └── [workflowName]/
                    └── route.ts

const workflow = await getWorkflow("send-email-received-ein");
```typescript

### 3. Workflow Definitions (`workflows/workflows.ts`)

Contains all workflow definitions using the `defineWorkflows()` function.

**Structure:**

- Each workflow is defined with `ow.defineWorkflow()`
- Workflows consist of multiple steps
- Steps can be retried automatically on failure

### 4. OpenWorkflow Instance (`workflows/instance.ts`)

Singleton instance of the OpenWorkflow engine connected to PostgreSQL.

**Features:**

- Shared connection pool
- Automatic reconnection handling
- Thread-safe singleton pattern

### 5. Worker (`workflows/worker.ts`)

Long-running process that executes workflow steps.

**Configuration:**

- Concurrency: 20 concurrent workflow executions
- Graceful shutdown on SIGTERM/SIGINT
- Automatic step retry on failure

### 6. Database Schema (`workflows/schema.sql`)

PostgreSQL schema for storing workflow state.

**Tables:**

- `workflow_runs`: Tracks workflow executions
- `step_attempts`: Tracks individual step executions
- `openworkflow_migrations`: Schema version tracking

## Data Flow

### Workflow Execution Flow

1. **API Request** → Client sends POST to `/api/workflow/{name}`
2. **Route Handler** → Validates request and maps parameters
3. **Registry Lookup** → Finds workflow by name
4. **Workflow Run** → Creates workflow run in database
5. **Worker Picks Up** → Worker polls for available workflows
6. **Step Execution** → Worker executes each step sequentially
7. **State Updates** → Each step updates database state
8. **Completion** → Workflow status updated to completed/failed

### Step Execution

Each step in a workflow:

- Is wrapped in `step.run()` for tracking
- Can be retried automatically on failure
- Updates database with execution status
- Can access previous step outputs

## Concurrency Model

### Worker Concurrency

The worker runs with `concurrency: 20`, meaning:

- Up to 20 workflows can execute simultaneously
- Each workflow runs steps sequentially
- Steps within different workflows run in parallel

### Database Transactions

- Each step runs in its own transaction context
- Failed steps roll back their changes
- Workflow state is persisted after each step

## Error Handling

### Step Failures

When a step fails:

1. Error is logged to database
2. Step status set to `failed`
3. Workflow can retry the step (if configured)
4. After max retries, workflow status set to `failed`

### Workflow Failures

When a workflow fails:

- All completed steps remain persisted
- Failed step outputs are stored
- Workflow can be manually retried or cancelled

## State Management

### Workflow States

- `pending`: Workflow created, waiting for worker
- `running`: Currently executing
- `completed`: Successfully finished
- `failed`: Failed after retries
- `cancelled`: Manually cancelled

### Step States

- `pending`: Step queued for execution
- `running`: Currently executing
- `completed`: Successfully finished
- `failed`: Execution failed

## Scalability

### Horizontal Scaling

Multiple workers can run simultaneously:

- Each worker polls for available workflows
- Database ensures only one worker picks up each workflow
- No coordination needed between workers

### Performance Considerations

- Database indexes optimize workflow queries
- Worker concurrency can be adjusted per environment
- Steps should be idempotent for reliability

## Security

### API Security

- API routes should implement authentication/authorization
- Validate all input parameters
- Sanitize user-provided data

### Database Security

- Use connection pooling
- Parameterized queries prevent SQL injection
- Row-level security for multi-tenant scenarios

## Monitoring

### Workflow Metrics

Track in database:

- Workflow execution times
- Step success/failure rates
- Worker utilization

### Logging

- Worker logs step executions
- API logs workflow triggers
- Database stores execution history

## Best Practices

1. **Idempotent Steps**: Steps should be safe to retry
2. **Small Steps**: Break complex logic into smaller steps
3. **Error Messages**: Provide clear error messages
4. **Timeouts**: Set appropriate timeouts for long-running steps
5. **Monitoring**: Monitor workflow success rates and performance