Execution Engine
Workflow execution, state management, and node processing diagrams
Workflow Execution Engine
This page covers the workflow execution engine including node processing, state management, and error handling.
Execution Flow Overview
flowchart TD
TRIGGER[Trigger Received] --> CREATE[Create Execution Record]
CREATE --> INIT[Initialize State]
INIT --> SORT[Topological Sort Nodes]
SORT --> LOOP{More Nodes?}
LOOP -->|Yes| CHECK[Check Dependencies]
CHECK --> DEPS_MET{Dependencies Met?}
DEPS_MET -->|Yes| EXECUTE[Execute Node]
DEPS_MET -->|No| SKIP[Skip - Wait for Dependencies]
EXECUTE --> RESULT{Result}
RESULT -->|Success| UPDATE[Update State]
RESULT -->|Failure| ERROR[Handle Error]
RESULT -->|Waiting| PAUSE[Pause Execution]
UPDATE --> LOOP
SKIP --> LOOP
ERROR --> ERROR_STRATEGY{Strategy}
ERROR_STRATEGY -->|Stop| FAIL[Mark Failed]
ERROR_STRATEGY -->|Continue| UPDATE
LOOP -->|No| COMPLETE[Mark Completed]
PAUSE --> WAIT[Wait for Input/Resume]
WAIT --> EXECUTE
style COMPLETE fill:#e8f5e9
style FAIL fill:#ffebeeExecution Sequence
sequenceDiagram
participant Trigger as Trigger Source
participant Service as Workflow Service
participant Engine as Execution Engine
participant DB as Database
participant Executor as Node Executor
participant State as State Storage
Trigger->>Service: triggerWorkflow(workflowId, input)
Service->>DB: Get workflow definition
DB-->>Service: Workflow data
Service->>DB: Create WorkflowExecution (PENDING)
DB-->>Service: Execution ID
Service->>Engine: execute(workflow, input)
Engine->>Engine: Topological sort nodes
Engine->>State: Initialize execution state
loop For each node in order
Engine->>Engine: Check node dependencies
alt Dependencies satisfied
Engine->>Executor: executeNode(node, context)
Executor->>Executor: Run node logic
Executor-->>Engine: Node result
alt Success
Engine->>State: Update node state (completed)
Engine->>State: Store output
else Failure
Engine->>State: Update node state (failed)
Engine->>Engine: Check error handling strategy
end
else Dependencies not met
Engine->>Engine: Skip node (process later)
end
end
Engine->>DB: Update execution (COMPLETED/FAILED)
Engine->>Service: Execution result
Service-->>Trigger: ResultNode State Machine
stateDiagram-v2
[*] --> PENDING: Node in workflow
PENDING --> RUNNING: Start execution
RUNNING --> COMPLETED: Success
RUNNING --> FAILED: Error
RUNNING --> WAITING: Input required
RUNNING --> SKIPPED: Condition not met
WAITING --> RUNNING: Input provided
WAITING --> CANCELLED: Timeout/Cancel
FAILED --> PENDING: Retry
SKIPPED --> [*]
COMPLETED --> [*]
CANCELLED --> [*]State Management
flowchart TD
subgraph "Execution State"
EXEC_ID[Execution ID]
STATUS[Status]
PROGRESS[Progress 0-100]
VARIABLES[Variables]
LOGS[Execution Logs]
end
subgraph "Node States"
NODE_MAP[Node State Map]
NODE_STATUS[Node Status]
NODE_OUTPUT[Node Output]
NODE_ERROR[Node Error]
NODE_TIMING[Start/End Time]
end
subgraph "Edge Tracking"
TRAVERSED[Traversed Edges]
PENDING_EDGES[Pending Edges]
end
EXEC_ID --> NODE_MAP
NODE_MAP --> NODE_STATUS
NODE_MAP --> NODE_OUTPUT
NODE_MAP --> NODE_ERROR
NODE_MAP --> NODE_TIMING
STATUS --> PROGRESS
TRAVERSED --> PENDING_EDGESDependency Resolution
flowchart TD
subgraph "Workflow Graph"
N1[Node A] --> N2[Node B]
N1 --> N3[Node C]
N2 --> N4[Node D]
N3 --> N4
N4 --> N5[Node E]
end
subgraph "Topological Order"
O1[1. Node A]
O2[2. Node B]
O3[3. Node C]
O4[4. Node D]
O5[5. Node E]
end
subgraph "Execution"
E1[Execute A] --> E2[Execute B & C parallel]
E2 --> E3[Wait for both]
E3 --> E4[Execute D]
E4 --> E5[Execute E]
endContext Passing
sequenceDiagram
participant Node_A as Node A
participant Engine as Engine
participant Node_B as Node B
participant Node_C as Node C
Note over Engine: Start execution
Engine->>Node_A: execute(initialContext)
Node_A-->>Engine: {output: "result_a"}
Engine->>Engine: Update context with Node A output
par Parallel execution
Engine->>Node_B: execute(context + node_a_output)
Node_B-->>Engine: {output: "result_b"}
and
Engine->>Node_C: execute(context + node_a_output)
Node_C-->>Engine: {output: "result_c"}
end
Note over Engine: Context now has all outputsError Handling Strategies
flowchart TD
ERROR[Node Error] --> STRATEGY{Error Strategy}
STRATEGY -->|stop| STOP[Stop Execution]
STRATEGY -->|continue| CONTINUE[Continue to Next]
STRATEGY -->|retry| RETRY[Retry Node]
STOP --> FAIL[Mark Workflow Failed]
FAIL --> NOTIFY[Send Failure Notification]
CONTINUE --> MARK[Mark Node as Failed]
MARK --> NEXT[Process Next Node]
RETRY --> COUNT{Retry Count?}
COUNT -->|< Max| WAIT[Wait with Backoff]
WAIT --> EXECUTE[Re-execute Node]
EXECUTE --> CHECK{Success?}
CHECK -->|Yes| SUCCESS[Continue Workflow]
CHECK -->|No| COUNT
COUNT -->|>= Max| FAIL_RETRY[Mark Failed After Retries]
FAIL_RETRY --> STRATEGY_CHECK{Continue on Fail?}
STRATEGY_CHECK -->|Yes| CONTINUE
STRATEGY_CHECK -->|No| STOP
style FAIL fill:#ffebee
style SUCCESS fill:#e8f5e9Breakpoint & Debugging
sequenceDiagram
participant User
participant UI as Debug UI
participant Engine as Execution Engine
participant State as State Storage
User->>UI: Start with breakpoints
UI->>Engine: execute(workflow, {mode: "step", breakpoints: [nodeB]})
Engine->>Engine: Execute Node A
Engine->>State: Update state
Engine->>Engine: Check breakpoint at Node B
Engine->>State: Update status (PAUSED)
Engine-->>UI: Paused at Node B
UI->>User: Show current state
User->>UI: Inspect variables
UI->>State: Get current context
State-->>UI: Context data
UI-->>User: Display context
User->>UI: Step to next
UI->>Engine: step(executionId)
Engine->>Engine: Execute Node B
Engine-->>UI: Node B complete
User->>UI: Continue execution
UI->>Engine: resume(executionId)
Engine->>Engine: Execute remaining nodes
Engine-->>UI: Execution completeUser Input Node
sequenceDiagram
participant Engine as Execution Engine
participant Node as User Input Node
participant State as State Storage
participant UI as User Interface
participant User
Engine->>Node: execute()
Node->>State: Create pending input request
Node->>State: Update node status (WAITING)
Node-->>Engine: {status: "waiting", inputId: "xyz"}
Engine->>State: Update execution (PAUSED)
State->>UI: Notify input required
UI->>User: Show input form
User->>UI: Provide input
UI->>State: submitInput(inputId, data)
State->>Engine: Resume with input
Engine->>Node: continue(inputData)
Node-->>Engine: {output: processedInput}
Engine->>State: Update node (COMPLETED)
Engine->>Engine: Continue executionExecution Metrics
flowchart LR
subgraph "Captured Metrics"
DURATION[Execution Duration]
NODE_TIME[Node Execution Times]
TOKENS[AI Token Usage]
COST[Execution Cost]
end
subgraph "Aggregations"
AVG_DURATION[Avg Duration]
SUCCESS_RATE[Success Rate]
TOTAL_COST[Total Cost]
end
subgraph "Analysis"
BOTTLENECK[Bottleneck Analysis]
COST_OPT[Cost Optimization]
RELIABILITY[Reliability Metrics]
end
DURATION --> AVG_DURATION
NODE_TIME --> BOTTLENECK
TOKENS --> TOTAL_COST
COST --> COST_OPT