type chooses the connector contract
Three runnable local declarations. Change the type and the meaning of connection and extract changes with it.
A connector family, then the contract for the two blocks below it.
What sources.<id>.type is
type is the discriminator that tells Hydra which connector must interpret a source declaration.
It is not a file format hint applied after reading. Connector construction happens first; that connector owns the accepted connection settings and extraction behavior.
What it contains
- One normalized string — Whitespace is trimmed and the value is lower-cased.
- A registered connector name — CSV and JSON are built in; database and optional connectors use the same slot.
- A downstream contract — The selected connector interprets
connectionandextract.
The type changes the contract
The outer YAML stays stable while the inner meaning changes. Choose type first, then fill the connection and extraction blocks that belong to that connector.
Run the complete example
Copy this manifest from the verified documentation project, then run it from the repository root.
version: "1.0"
sources:
src_orders:
type: csv
connection:
base_path: "data/canonical"
extract:
table: orders.csv
batch_size: 10000
schema:
mode: manual
fields:
- name: order_id
type: string
required: true
- name: amount
type: float
drift_policy: warn
validation_policy: best_effort
sample_size: 100
src_orders_extra:
type: csv
connection:
base_path: "data/canonical"
extract:
table: orders_extra.csv
batch_size: 10000
hdrctl run examples/tutorial/dsl-elements
✅ Pipeline completed successfully in 2.6s Rows read : 30 Rows written: 6
The relevant command syntax, copied from hdrctl run --help:
hdrctl run [PATH] -s, --sources FILE explicit path to sources.yaml -d, --destinations FILE explicit path to destinations.yaml -t, --transformations FILE explicit path to transformations.yaml -P, --param KEY=VALUE override a parameter, repeatable