Hydra ETL
Build your first job
Hydra DSL›1 · Manifest›workflow.yaml
Grammar element · workflow.yaml

workflow.yaml turns jobs into a graph

Three correct graphs. Walk them, then see how dependencies become an execution plan.

workflow.yaml4 steps

      

    

    

      
      
      
    

    

      
      
    

  


  

    

      
      
What the runner schedules
Dependencies form execution groups; independent steps in a group may run together.

    

    

    

    

      ✓ 2 of 2 steps completed
      
    

  



 

  
What workflow.yaml is

  
workflow.yaml is an orchestration graph above individual Hydra jobs. Its steps can run job directories or actions, while dependencies determine which nodes must finish first.

  
It is not another job manifest. A job moves rows from one source to one destination; a workflow coordinates several jobs and side effects as one named run.


  
What it contains

  
    
    
  • Identity and entry — name, optional description and version, plus a manual, schedule, or webhook trigger.
    • 
    
  • Work nodes — job steps point to job directories; action steps name an action and its parameters.
    • 
    
  • Control edges — depends_on, when, enabled, failure policy, and retry policy govern scheduling.
    • 
  


  
Dependencies are the execution plan

  
List position makes the file readable; depends_on makes the graph. The runner groups every step whose dependencies are complete. One ready step runs alone; several ready steps run in parallel.


  

    
      Dependencies create sequential and parallel execution groups
      The extract job runs first. Notify and audit both depend on it, so they form the next parallel group.
      group 1group 2 · parallel
      
      extract_csvjob · no dependency
      
      
      
      notifydepends_on extract_csv
      
      auditdepends_on extract_csv
      
      complete3 steps
      The two outgoing edges make notify and audit ready at the same time.
      
    
    
The graph, not the vertical order, determines what can run together.

  


  
Run a complete workflow

  
The repository carries this two-step workflow:

  
workflow:
  name: "demo_pipeline"
  description: "Workflow de démonstration — extraction + log"
  trigger:
    type: manual
  steps:
    - name: "extract_csv"
      type: job
      job: "../../examples/minimal_csv"
      depends_on: []
      on_failure: fail
    - name: "notify"
      type: action
      action: log
      params:
        message: "Pipeline terminé avec succès."
      depends_on: ["extract_csv"]
      on_failure: skip

  
hdrctl workflow run examples/workflow_demo/workflow.yaml

  
  ✓  Step 'extract_csv' OK (3.2s)
  ✓  Step 'notify' OK (0.0s)

  ✅ Workflow 'demo_pipeline' completed in 3.2s — 2/2 steps OK

  
The command syntax, copied from hdrctl workflow run --help:

  
hdrctl workflow run PATH
  PATH          path to workflow.yaml
  --lang LANG   interface language
  -h, --help    show command help


  
Close by

  


 
  
 
 
      
  0 / 0 on this page