pipeline.yaml names the two ends

What pipeline.yaml is

pipeline.yaml is the wiring diagram for one job. It selects one identifier from sources.yaml and one from destinations.yaml, establishing the direction from read to write.

It is not the transformation plan and it contains no connection details. Those responsibilities stay in their own files; this manifest only joins their names.

What it contains

• The input end — from, an identifier declared under sources.
• The output end — to, an identifier declared under destinations.
• Optional context — the conventional version line, which the job runner does not use.

Names connect files

The file does not move rows. It resolves two names. After both endpoints resolve, the engine reads from the source, applies transformations.yaml when present, and writes to the destination.

Run the complete link

This manifest connects the source and destination already declared by the tutorial project:

version: "1.0"
pipeline:
  from: src_orders
  to: dest_big_orders

hdrctl run examples/tutorial/01-first-job -p examples/tutorial/01-first-job/pipeline.yaml

  ✅ Pipeline completed successfully in 7.8s
  Rows read   : 30
  Rows written: 6

The relevant command syntax, copied from hdrctl run --help:

hdrctl run [PATH]
  -p, --pipeline FILE   explicit path to pipeline.yaml
  --dry-run             validate configuration without writing data
  -v, -vv, -vvv         more detail, up to full logs

Close by

• sources.yaml — declaring the input end
• destinations.yaml — declaring the output end
• transformations.yaml — what happens between them
• from and to — the two-key reference
• Lesson 1 — run the complete link