filter.expr decides which rows continue

What filter.expr is

filter.expr is a boolean expression evaluated once across the current table to decide which rows continue.

It is not validated against runtime column types by hdrctl validate. Earlier steps must prepare both names and types before the filter runs.

What it contains

• Current column names — Resolved by pandas query at this step.
• Literals and comparisons — Numbers, quoted strings, and comparison operators.
• Boolean composition — Conditions combined with and, or, and parentheses.

The expression sees the current frame

A filter never reaches back to source columns. Rename or cast first, and the expression sees only those resulting names and values.

Run the complete example

Copy this manifest from the verified documentation project, then run it from the repository root.

version: "1.0"
transformations:
  steps:
    - cast:
        mapping:
          amount: float
    - filter:
        expr: "amount > 50"

hdrctl run examples/tutorial/dsl-elements

  ✅ Pipeline completed successfully in 2.6s
  Rows read   : 30
  Rows written: 6

The relevant command syntax, copied from hdrctl run --help:

hdrctl run [PATH]
  -s, --sources FILE          explicit path to sources.yaml
  -d, --destinations FILE     explicit path to destinations.yaml
  -t, --transformations FILE  explicit path to transformations.yaml
  -P, --param KEY=VALUE       override a parameter, repeatable

Close by

• Rows — filter and related operations
• Columns — prepare names and types
• calculate.expr — producing values
• steps — expression order
• Reference — filter contract