Hydra ETL
Build your first job
Lesson 7 of 14 · Add a destination

Close the job with a destination

A short lesson, because you already know most of it: a destination is configured exactly like a source. One rule differs, and it is worth the five minutes.

Destinationsabout 8 minutesa transformation in place

1. Objective

Goal

Add the node that ends the job, recognise that its dialogs are the ones you already filled in for sources, and learn the single rule that sets destinations apart: a job writes to exactly one place.

Course project

Keep one cumulative Studio project named studio-course, one manual workflow named first-workflow, and one job named first-job. Each lesson builds on the state saved by the previous one.

Prerequisites

  • Hydra API started with hdrctl serve.
  • Studio started from Hydra/studio with npm run dev.
  • A job with a source and at least one transformation.

2. Steps

01Read the six destinations

Objective

See what Hydra can write to, and notice what is missing.

Actions

  1. Expand Destinations in the job palette.
  2. Compare the list with Sources just above it.
  3. Count the difference.
Jobs configuration → Destinations (6)

Six connectors, and you have met all six already — three file formats and three databases. The seventh source, Web API, has no counterpart here: Hydra reads from an HTTP endpoint but never writes to one. To send data out over HTTP, orchestrate a Webhook action in the workflow scene instead.

What you should see

Seven entries on the way in, six on the way out. The difference is one, and it is deliberate.

Sources        CSV  JSON  Parquet  MySQL  PostgreSQL  MongoDB  Web API
Destinations   CSV  JSON  Parquet  MySQL  PostgreSQL  MongoDB
The job palette with Destinations expanded, listing CSV, JSON, Parquet, MySQL, PostgreSQL and MongoDB.
The same six connectors as the sources, minus Web API.

02Configure it — you already know how

Objective

Fill in a destination, and recognise the dialog.

Actions

  1. Drag CSV from Destinations onto the canvas.
  2. Connect the last transformation to it.
  3. Double-click it and set the output path.
Destinations → CSV → connect → File path → Save

The dialogs are the ones from lesson 5, unchanged. A file destination asks for a path; a database destination asks for host, port, database, user and password — and the same rule applies to the password: write ${ENV:MY_SECRET} and keep the value under Environments, never in the manifest.

DestinationWhat it asks for
CSV · JSON · ParquetAn output path. Browse writes one that exists.
MySQL · PostgreSQL · MongoDBA connection and a target table or collection.
What you should see

The red badge clears as soon as the required field is filled, exactly as it did for the source.

CSV  DESTINATION   Writes a CSV file
  STEP NAME   dest_csv_1
  FILE PATH * ./output/result.csv

03One destination, and one only

Objective

Learn the rule that separates destinations from every other node.

Actions

  1. Look at the shape of the finished job, from left to right.
  2. Count the sources, then the destinations.
  3. Read the Pipeline tab of Hydra DSL.
Hydra DSL → Pipeline

A job may read from several sources and chain several transformations. It writes to exactly one destination. Branching is allowed upstream, where streams converge; it is not allowed downstream, where the job would have to split.

NodeHow manyMay branch
Sourcesone or moreYes — several may converge on one transformation
Transformationsnone, one or manyYes, as a chain
Destinationexactly oneNo

That is why pipeline.yaml holds a single from and a single to. If you need the same data in two places, write two jobs and orchestrate them in the workflow — which is what the next lessons build towards.

What you should see

One from, one to, and identifiers rather than paths.

pipeline:
  from: source_csv_8
  to:   dest_json_10
A finished job canvas with two sources converging on a Join transformation, which feeds a single JSON destination, marked Valid.
Two sources converge, one destination closes. The counter reads 4 nodes · 3 edges, and the job is Valid.
Tips and traps

A destination is configured exactly like a source. Same dialogs, opposite direction.

A job writes to one destination. Two outputs means two jobs, joined in the workflow.

Branching is allowed upstream only. Sources converge; destinations never split.

There is no HTTP destination. To push over HTTP, use the Webhook action instead.

A password belongs in Environments, written as ${ENV:NAME} — in a destination as in a source.

Nothing can follow a destination. It has no output handle, and that is why the edge refuses.

3. Checklist

VerificationExpected result
PaletteSix destinations, no Web API
DialogThe same fields as the matching source
CountExactly one destination on the canvas
Pipeline tabOne from, one to
StatusThe job reads Valid

4. Troubleshooting

SymptomCauseFix
No HTTP destination in the listWeb API is a source onlyUse the Webhook action in the workflow scene
A second destination is refusedA job writes to one placeCreate a second job and orchestrate both
Nothing connects after the destinationIt has no output handleExpected — the destination ends the job
The badge stays after savingThe output path or table is emptyReopen the node and fill the field marked with a red asterisk

5. Next lesson

The job is complete: it reads, reshapes and writes. The next lesson opens the manifests Studio has been writing all along, and checks that they say what you meant.

Close by

0 / 0 on this page