Maestro.
Orchestrate pipelines with Maestro: the canvas, step types, parallel and series groups, retry policies and running a Maestro.
6. Maestro (Orchestration)
6.1 What is Maestro
Maestro is Odara's orchestration layer. A Maestro combines multiple pipelines into a single workflow, coordinating their execution with series, parallel, and conditional logic.
Where a pipeline processes data, a maestro processes pipelines — think of it as a DAG of pipeline runs.
6.2 Canvas
The Maestro canvas looks like the pipeline canvas but with different node types. Each step is a pipeline call, a parallel group, or a conditional. Connect steps with edges to indicate dependencies.
6.3 Step types
- Pipeline Call — runs one pipeline by name or ID. Optionally with variable overrides.
- Parallel Group — runs multiple child steps concurrently, with configurable
max_concurrency. - Conditional — evaluates a SQL-like expression; runs
then_stepsorelse_stepsaccordingly.
A Pipeline Call can carry its own variables (passed to the child pipeline), pass the Maestro's variables to it, and branch with on success / on failure steps. With stop on failure, nothing runs after a failed step.
6.4 Parallel vs. series execution
Connected linearly = series (A runs, then B, then C). Wrap steps in a Parallel Group = concurrent. Mix both for DAGs: parallel fan-out followed by a join (series) step.
6.5 Retry policies
Each Pipeline Call can have an optional retry policy: max attempts, backoff strategy (fixed / exponential), initial delay, max delay. Failures trigger retries up to the configured limit before the step is marked failed.
6.6 Built-in variables
Maestros expose several built-in variables to their pipeline calls:
@maestro_run_id— unique ID of the current maestro execution.@maestro_name— name of the maestro.@step_name— name of the current step.@previous_output— output of the directly preceding step.
6.7 Running and the Monitor
Click Execute on the maestro canvas. Progress streams live, with each step changing color as it progresses. Failed steps abort downstream dependents unless marked continue_on_failure. A Maestro also runs from a schedule or from the API (POST /api/v1/maestros/:id/run).
Every child pipeline runs in the Maestro run's environment — and therefore where that environment runs (Local, Docker, AWS ECS Fargate or Google Cloud Run). Abort stops the Maestro and its running children, remote runners included.
The Monitor records every Maestro run the same way, whatever started it: one row per step, the run's tally (pipelines completed and failed), and each step linked to the child run it started — open a step to see that child's per-node rows, and go back to the Maestro.