The route

Run After: wait for the other branches

Download the exercise files

Until now every pipeline was a single line of nodes. As soon as a pipeline branches, the branches run independently — and "independently" includes in any order. This level builds a pipeline where one step needs two others to be finished first, watches it fail without a guarantee, and fixes it with Run After.

Goal

A pipeline that splits the orders into two files and, only after both are written, lists the output folder to confirm they're there.

You'll need

  • orders.csv from World 3 (level 3.2's download).
  • About 15 minutes.

Steps

1. Split one source into two files

Create a pipeline w5-1-run-after:

  1. A CSV Source orders.
  2. Two SQL nodes fed by it: paid (WHERE status = 'paid') and not_paid (WHERE status <> 'paid').
  3. A CSV Write after each: paid_csv writing split/paid.csv and not_paid_csv writing split/not_paid.csv, both in the same folder.

One source feeding two nodes is fine: each branch gets all 3,000 rows.

2. Try to check the result — too early

Add a File List (INGEST ▸ File System) named check_files, with Directory set to that split folder and Pattern *.csv, and a Console after it. Leave the File List unconnected to the rest and Execute — the first time, before the folder exists.

Directory not found: …/split

Nothing told Odara that check_files depends on the two writes, so it ran whenever it was ready — immediately, since it has no input. The folder didn't exist yet.

3. Add the barrier

From ORCHESTRATE ▸ Flow Control, drop a Run After named both_written. Connect both CSV Write nodes into it, and connect it into check_files.

Two branches meeting at a Run After before the File List

Run After passes no data. It's a barrier: whatever comes after it starts only when every node connected into it has finished. That's what lets a source — which normally runs first — wait for targets.

4. Execute

Delete the split folder, then Execute and expand the console output.

The File List, run after both files were written

Expected result

  • The run completes. The log shows paid: 2682 rows, not_paid: 318 rows, then both_written, then check_files: 2 rows.
  • The console lists paid.csv and not_paid.csv, each with its size in bytes.
  • Run it again: the files are replaced, and the result is the same — a good sign the pipeline doesn't depend on leftovers from an earlier run.

Common pitfalls

It worked once without Run After, then failed. Order between independent branches isn't guaranteed. If it works without a barrier, that's luck — usually a folder left over from the previous run.

Only one branch goes into the Run After. Then it waits for that one only. Every branch the next step depends on must be connected into it.

Expecting data out of Run After. It outputs nothing. Anything after it has to read its own input — a source, like the File List here.

What you built

Explicit ordering between branches. Level 5.5 uses the same barrier for something with higher stakes: deleting a file only after it has been safely copied.

Check yourself

One question, drawn from a larger bank. Get it right to pass.

  1. How many rows does the `not_paid` branch write?