Hydra ETL
Build your first job
Hydra DSL5 · TransformationsCompute
Grammar element · operations.compute

Compute creates values without hiding the contract

Calculate handles expressions; script handles declared custom outputs. Both name what they add.

transformations.yaml4 steps

      
What the step produces

Named output columns derived from values already present in the current frame.

✓ amount_with_tax calculated

What operations.compute is

Compute operations create new values from columns in the current transformation frame.

They do not reach external systems. calculate evaluates one pandas expression; script executes declared Python code in a restricted environment.

What it contains

  • calculate — One output column and one expression.
  • script inputs — An optional list of columns made available to code.
  • script outputs and code — A typed output mapping, execution mode, and required code block.

Outputs are part of the declaration

Computation is easier to trust when every produced column is named up front. Calculate names one output; script declares an output mapping before any code runs.

Compute maps current values to named outputs The amount column is multiplied by 1.2 and stored in amount_with_tax. amountcurrent column amount * 1.2declared computation amount_with_taxnamed output The output name is visible in YAML before the expression executes.
A computation is a contract from inputs to named outputs.

Run the complete example

Copy this manifest from the verified documentation project, then run it from the repository root.

version: "1.0"
transformations:
  steps:
    - cast:
        mapping: {amount: float}
    - calculate:
        column: amount_with_tax
        expr: "amount * 1.2"
hdrctl run examples/tutorial/dsl-elements -t compute.yaml
  ✅ Pipeline completed successfully in 2.9s
  Rows read   : 30
  Rows written: 30

The relevant command syntax, copied from hdrctl run --help:

hdrctl run [PATH]
  -s, --sources FILE          explicit path to sources.yaml
  -d, --destinations FILE     explicit path to destinations.yaml
  -t, --transformations FILE  explicit path to transformations.yaml
  -P, --param KEY=VALUE       override a parameter, repeatable

Close by

0 / 0 on this page