Hydra ETL
Build your first job
Get started with Hydra

From raw data to a repeatable pipeline

Hydra turns a data task into a small set of readable YAML files. Describe what enters, what changes and where the result goes—then validate and run the same contract anywhere.

Your first 30 minutes

A short path from reading to running

  1. 1Run a complete exampleDownload, integrate and run a complete Studio job.
  2. 2Change one decisionExplore valid variants without installing.
  3. 3Read the exact grammarUnderstand every manifest block.
01

Overview

The Hydra emblem: seven dragon heads growing from a single body, inside a circle.
HYDRA One body, many heads

What is Hydra?

Hydra is an open-source declarative ETL platform. A job connects one source to one destination through an ordered list of transformations. Larger processes combine those jobs in a workflow.

The manifests are the contract. They are readable in a code review, testable before execution and reusable from the CLI, API or Studio.

How a Hydra job moves data CSV, database and API sources enter a declarative Hydra job. Sources, transformations, pipeline and destinations manifests describe execution, which produces a result and an observable run. INPUTS CSV / JSON DATABASE WEB API HYDRA JOB sources.yaml transformations.yaml destinations.yaml pipeline.yaml EXECUTION CLI / API WORKFLOW RESULT declare once · validate · run · observe
02

The core idea

Everything is YAML

YAML is the main language of Hydra: sources, transformations, destinations and workflows are all declared in it. Version them, review them and move them between environments.

Declare the outcome

Choose a and compose proven operations. Hydra handles batching, execution order and delivery while the business rule stays visible.

One path from check to run

the manifest first, execute it through the same and read row counts and step states from the run.

Jobs compose into workflows

A workflow is the pipeline of your jobs. Keep each job independently runnable, then add dependencies, schedules, actions and retry policies only where orchestration needs them.

Install HydraGet the CLI, the engine and the Studio — every installation path.
03

Start where you are

CLIHydra CLI tutorialsCreate, validate, run and automate from the terminal. UIHydra Studio tutorialsDesign, validate and follow jobs on the visual canvas.
Data engineersBuild the first file-to-file job, then replace either end with the connector you need. Platform and operations teamsOrchestrate existing jobs, schedule them and follow every terminal state.
Three commands to a first run 
$ hdrctl init my_job
$ hdrctl test my_job
$ hdrctl run my_job -vv

Requires Python 3.9+. The repository install uses pip install -e .

Build the guided example Explore the language
0 / 0 on this page