Hydra ETL
Build your first job
Hydra DSL5 · TransformationsCombine
Grammar element · operations.combine

Combine brings in a second declared source

Three valid union forms. Join, merge, and union all resolve right before the operation runs.

transformations.yaml4 steps

      
What the runner prepares

A second source is resolved and loaded before the transformation engine combines it.

✓ 30 + 4 → 34 rows

What operations.combine is

Combine operations bring a second source into the current transformation frame.

The transformation engine does not open that source itself. The runner resolves right, loads its rows, and injects them before join, merge, or union executes.

What it contains

  • join — Matches rows with inner, left, right, or outer semantics.
  • merge — Reconciles rows by a required key, optionally deleting unmatched left rows.
  • union — Stacks compatible rows and optionally removes duplicates.

Right is prepared before the operation

A two-frame operation still has one transformation input. The runner turns right into ready rows first, so the engine receives two frames rather than a connector configuration.

Combine adds a prepared right frame Thirty current rows are combined with four prepared right rows to produce 34 rows. 30 current rowsleft frame union + rightrunner prepared 34 rowsstacked frame The runner resolves right; the operation combines data frames.
Connector work finishes before combination begins.

Run the complete example

Copy this manifest from the verified documentation project, then run it from the repository root.

version: "1.0"
transformations:
  steps:
    - union:
        right: src_orders_extra
        distinct: false
hdrctl run examples/tutorial/dsl-elements -t combine.yaml
  ✅ Pipeline completed successfully in 2.7s
  Rows read   : 30
  Rows written: 34

The relevant command syntax, copied from hdrctl run --help:

hdrctl run [PATH]
  -s, --sources FILE          explicit path to sources.yaml
  -d, --destinations FILE     explicit path to destinations.yaml
  -t, --transformations FILE  explicit path to transformations.yaml
  -P, --param KEY=VALUE       override a parameter, repeatable

Close by

0 / 0 on this page