Hydra ETL
Build your first job
GuideWorkflowswhen
Workshop 36 · 20 minutes

Guarding a workflow step with an expression

A practical workshop for a costly export should run only in production. Build the smallest manifest, validate it, run it and read the result.

Context — a costly export should run only in production

The task is currently manual and its assumptions are not recorded. Hydra turns it into files that can be reviewed and rerun.

Where things stand

  • The current result is fragile. Its assumptions are split between tools, clicks and memory.
  • Reruns are uncertain. The write mode or orchestration rule is not visible beside the data.
  • Evidence is missing. A colleague cannot compare a declared rule with a concrete before and after state.

The question

How do you make when explicit in workflow.yaml and verify the resulting execution states?

  • Keep jobs independently runnable
  • Give every workflow step a unique name
  • Validate the graph before execution
  • Read every step status in the summary

The solution in one line

A small workflow.yaml, one validation command and one run with observable step states.

workspace tree5 steps
workflow.yaml
jobs/
  sales/
  inventory/
  publish/
What you do

Name the work the workflow will coordinate.

Step 1 · workspace ready

Steps

1. prepare the jobs and boundary

  1. Name the work the workflow will coordinate.
  2. Compare the file with the explanation in the workbench.
  3. Record the shown check before moving to the next step.
workflow.yaml
jobs/
  sales/
  inventory/
  publish/
Check — job boundaries identified

2. declare when

  1. Write the trigger, steps and policy.
  2. Compare the file with the explanation in the workbench.
  3. Record the shown check before moving to the next step.
workflow:
  version: "1.0"
  name: when_workshop
  description: "Guarding a workflow step with an expression"
  trigger:
    type: manual
  steps:
    - name: publish
      type: job
      job: ./jobs/publish
      when: "params.environment == 'production'"
Check — workflow contract declared
Trap — A false guard is a controlled skip, not a failure.

3. validate the graph

  1. Catch missing jobs, duplicate names and invalid dependencies.
  2. Compare the file with the explanation in the workbench.
  3. Record the shown check before moving to the next step.
$ hdrctl workflow validate workflow.yaml

✅ Workflow valid: when_workshop
Steps: 1
Check — workflow and dependency graph valid

4. run the workflow

  1. Execute and follow step states.
  2. Compare the file with the explanation in the workbench.
  3. Record the shown check before moving to the next step.
$ hdrctl workflow run workflow.yaml

environment=staging    publish skipped
environment=production publish succeeded
Check — terminal states recorded

5. read the orchestration result

  1. Distinguish success, skip, retry and continuation.
  2. Compare the file with the explanation in the workbench.
  3. Record the shown check before moving to the next step.
environment=staging    publish skipped
environment=production publish succeeded
Check — result matches the declared orchestration rule

Expected result

  • The workflow validates before execution.
  • environment=staging publish skipped
  • The run summary explains every terminal state.

Reading the results

Read the summary as a graph, not a flat log. A false guard is a controlled skip, not a failure.

What the counters do—and do not—prove

They prove how many records entered and left this run. They do not replace checking the target schema, the business meaning of values or the reason a workflow step was skipped.

The rule to carry forward

A false guard is a controlled skip, not a failure.

Did we answer the question?

objectiveresultwhere
Configuration explicityesthe relevant YAML block
Safe rerunyesthe destination or workflow policy
Observable resultyesthe command output and counters
Hidden manual ruleremovedthe rule now lives in versioned text

Before and after

  • Before — a costly export should run only in production requires a person to remember the order, options and checks.
  • After — one reviewed manifest and one command produce the same observable result.
  • What is really gained — The durable gain is the contract: a colleague can read the configuration, reproduce the run and challenge the assumptions.

Close by

0 / 0 on this page