Core Concepts

Worlds Engine uses a world-based architecture for distributed workflow orchestration.

Architecture Overview

┌─────────────────────────────────────────────────┐
│                    World                         │
│  ┌───────────┐  ┌───────────┐  ┌─────────────┐ │
│  │  Worker 1 │  │  Worker 2 │  │  Worker N   │ │
│  └─────┬─────┘  └─────┬─────┘  └──────┬──────┘ │
│        │              │                │        │
│  ┌─────▼──────────────▼────────────────▼─────┐  │
│  │            Task Queue                      │  │
│  └────────────────────────────────────────────┘  │
│  ┌────────────────────────────────────────────┐  │
│  │            Storage Layer                   │  │
│  │  (Memory / File / Hybrid)                  │  │
│  └────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────┘

World

The World is the orchestrator that manages:

• Worker lifecycle (spawning, monitoring, scaling)
• Task routing and queue management
• Workflow execution coordination
• Metrics collection and monitoring
• Storage persistence

const world = new World({
  minWorkers: 2,
  maxWorkers: 10,
  persistence: 'hybrid'
});

Workers

Workers are execution units that:

• Poll the task queue for work
• Execute workflows and activities
• Send heartbeat signals
• Handle graceful shutdown
• Report completion status

Each worker maintains:

• Unique identifier
• Status (idle, busy, stopping, stopped)
• Current task reference
• Last heartbeat timestamp
• Task completion counter
• Optional queue assignment

Workflows

Workflows orchestrate business logic through deterministic execution:

const workflow = workflow('name', async (ctx, input) => {
  'use workflow';  // Directive for compile-time transformation
  
  // Execute activities
  const result = await ctx.run(activity, input);
  
  // Spawn child workflows
  const child = await ctx.executeChild('child', data);
  
  // Add compensations
  ctx.addCompensation(async () => {
    await ctx.run(rollback, result);
  });
  
  return result;
});

Workflow Context

The workflow context provides:

• ctx.run() - Execute activities
• ctx.executeChild() - Spawn child workflows
• ctx.addCompensation() - Register compensation
• ctx.sleep() - Deterministic delays
• ctx.isCancelled() - Check cancellation
• ctx.fetch() - HTTP requests with retry
• ctx.createHook() - Create event hooks
• ctx.createWebhook() - Create webhooks
• ctx.getMetadata() - Access workflow metadata

Activities

Activities perform the actual business logic:

const activity = activity('name', async (ctx, input) => {
  'use step';  // Directive for side-effect tracking
  
  // Perform business logic
  const result = await database.query(input);
  
  // Send heartbeats for long operations
  ctx.heartbeat('processing...');
  
  return result;
});

Activity Context

• ctx.heartbeat() - Signal liveness
• ctx.isCancelled() - Check cancellation
• ctx.activityId - Unique activity ID
• ctx.workflowId - Parent workflow ID
• ctx.attempt - Current attempt number

Event Sourcing

Every workflow execution is recorded as ordered events:

• workflow_started
• activity_scheduled
• activity_started
• activity_completed
• activity_failed
• activity_retry
• compensation_added
• compensation_executed
• workflow_completed
• workflow_failed

This enables:

• Deterministic replay after crashes
• Complete audit trail of execution
• Time-travel debugging
• State reconstruction

Task Queue

The priority queue manages workflow and activity execution:

interface Task {
  id: string;
  type: 'workflow' | 'activity' | 'compensation';
  workflowId: string;
  name: string;
  input: any;
  priority?: number;
  scheduledAt: number;
  taskQueue?: string;
}

Tasks are:

1. Sorted by priority (descending)
2. Then by scheduled time (ascending)
3. Dequeued by available workers
4. Executed with retry logic

Storage Modes

Memory Store

Fast, no I/O overhead, loses state on restart:

const world = new World({ persistence: 'memory' });

File Store

Durable across restarts, JSON files:

const world = new World({
  persistence: 'file',
  persistencePath: '.worlds-engine'
});

Directory structure:

.worlds-engine/
├── workflows/
│   └── {workflowId}.json
├── schedules/
│   └── {scheduleId}.json
├── logs/
│   └── {date}.log
└── queue.json

Hybrid Store

Best of both - fast memory with periodic disk sync:

const world = new World({ persistence: 'hybrid' });

Syncs every 5 seconds to disk.

Directives

Compile-time transformations enable advanced features:

'use workflow'

Marks function as workflow orchestrator:

const myWorkflow = async (ctx, input) => {
  'use workflow';
  // Enables deterministic execution and replay
};

'use step'

Marks function as side-effecting operation:

const myStep = async (ctx, input) => {
  'use step';
  // Enables side-effect tracking and stable IDs
};

Failure Strategies

Handle workflow failures in different ways:

• retry - Restart from beginning with backoff
• compensate - Execute saga compensations
• cascade - Propagate to parent and children
• ignore - Mark failed, no action
• quarantine - Isolate for debugging

const workflow = workflow('name', handler, {
  failureStrategy: 'compensate'
});

Next Steps

• Dive into Workflows
• Learn about Activities
• Understand Retry Patterns