Hydra ETL
Build your first job
Hydra DSL5 · TransformationsReshape
Grammar element · operations.reshape

Reshape changes where values live

Unpivot, pivot, and transpose alter layout without inventing a second data source.

transformations.yaml4 steps

      
What changes

Rows become columns, columns become rows, or repeated value columns become observations.

✓ 5 wide rows → 20 long rows

What operations.reshape is

Reshape operations change the arrangement of values across rows and columns.

They do not join another source or summarize groups by themselves. Pivot may aggregate collisions, but its primary responsibility is layout.

What it contains

  • pivot — Turns values from one column into output columns.
  • unpivot — Turns several value columns into repeated rows.
  • transpose — Exchanges the row and column axes.

Name the identity before moving values

A reshape is understandable when the stable identity is explicit. Read index, id_vars, or index_col first; then follow the values that move around it.

Unpivot turns repeated columns into observations Five regional rows with four quarter columns become twenty region-quarter observations. 5 × 4 quarterswide values unpivotquarter becomes data 20 observationslong values The same values remain; their coordinates change.
Reshape changes layout, not the source of the values.

Run the complete example

Copy this manifest from the verified documentation project, then run it from the repository root.

version: "1.0"
transformations:
  steps:
    - unpivot:
        id_vars: [region]
        value_vars: [q1, q2, q3, q4]
        var_name: quarter
        value_name: revenue
hdrctl run examples/tutorial/dsl-elements -s sources-revenue.yaml -t reshape.yaml
  ✅ Pipeline completed successfully in 3.7s
  Rows read   : 5
  Rows written: 20

The relevant command syntax, copied from hdrctl run --help:

hdrctl run [PATH]
  -s, --sources FILE          explicit path to sources.yaml
  -d, --destinations FILE     explicit path to destinations.yaml
  -t, --transformations FILE  explicit path to transformations.yaml
  -P, --param KEY=VALUE       override a parameter, repeatable

Close by

0 / 0 on this page