Hydra ETL
Build your first job
Get startedHydra CLILesson 9
Lesson 9 of 11 · Build a workflow

Compose your jobs into a workflow

Wrap the cumulative job in a manual workflow, add a completion action, and validate the manifest and dependency graph before running either step.

hdrctl workflow validateabout 12 minutesproject inventoried

1. Objective

Goal

Create workflow.yaml beside first-job. Declare one job step and one dependent log action, then use hdrctl workflow validate workflow.yaml to prove the model and graph are valid.

Course project

Keep working from the cli-course folder. The job is named first-job, and the workflow lessons add workflow.yaml beside it; every command and terminal excerpt in this lesson was run against that same cumulative project.

Reading a result. Durations and process identifiers vary between machines — ignore them. What proves a run is the stable evidence: named files, resolved identifiers, row counters, step states, HTTP status. If one of those differs from the lesson, fix the previous step before continuing.

Prerequisites

  • Lesson 8 completed and checked.
  • The validated first-job CSV project.
  • A terminal opened in the cli-course folder.

2. Steps

01Declare the workflow identity

Objective

Create the top-level workflow mapping and manual trigger.

Actions

  1. Create workflow.yaml in cli-course.
  2. Set the name to first-job-workflow.
  3. Keep the trigger manual for this guided run.
workflow:
  name: "first-job-workflow"
  description: "Run the guided CSV job, then record completion."
  trigger:
    type: manual
What you should see

The document has the required top-level key, a unique name, and a supported trigger type.

workflow:
  name: "first-job-workflow"
  trigger:
    type: manual

02Add the job step

Objective

Reference the existing cumulative job from the workflow directory.

Actions

  1. Add run_first_job under steps.
  2. Set type to job and job to ./first-job.
  3. Leave depends_on empty.
steps:
  - name: "run_first_job"
    type: job
    job: "./first-job"
    depends_on: []
    on_failure: fail
What you should see

The first step points at a configured job and has no upstream dependency.

- name: "run_first_job"
  type: job
  job: "./first-job"
  depends_on: []

03Add the completion action

Objective

Create a second step that can run only after the job succeeds.

Actions

  1. Add an action step named record_completion.
  2. Use the implemented log action.
  3. Depend on run_first_job.
- name: "record_completion"
  type: action
  action: log
  params:
    message: "first-job-workflow completed successfully."
  depends_on: ["run_first_job"]
  on_failure: fail
What you should see

The dependency forms a simple job-to-action sequence.

run_first_job → record_completion

04Validate the workflow graph

Objective

Check the workflow model and dependency ordering without executing it.

Actions

  1. Run the workflow validator against the new file.
  2. Do not use workflow run until this verdict is clean.
hdrctl workflow validate workflow.yaml
What you should see

The observed validator output is a single successful verdict.

✅ Workflow valid — no errors detected.

3. Checklist

VerificationExpected result
Top-level keyworkflow
Job path./first-job
Dependencyrecord_completion depends on run_first_job
ValidationWorkflow valid — no errors detected.

4. Troubleshooting

SymptomCauseFix
The top-level key is rejectedThe document starts directly with name or stepsNest the definition below workflow:
The dependency is unknownThe referenced name differs by spellingUse the exact earlier step name run_first_job
A cycle is reportedSteps depend on each other in a loopKeep the first dependency empty and the second dependent only on the first

5. Next lesson

The workflow manifest and DAG are valid, but neither step has executed yet. The next lesson runs the workflow and reads one terminal state for the job, one for the completion action, and the final two-of-two summary.

Close by

0 / 0 on this page