Prove the connection exists
Prove that the declared source, destination, and transformations are ready while keeping the job free of output writes.
1. Objective
Goal
Use hdrctl test first-job after strict manifest validation. Read each section separately so you know whether source access, destination configuration, transformations, and environment lookup are ready.
Keep working from the cli-course folder. The job is named first-job; every command and terminal excerpt in this lesson was run against that same project.
Prerequisites
- Lesson 3 completed and its checklist confirmed.
- The repository clone and hdrctl entry point from lesson 1.
- A terminal opened in the parent folder that contains
first-job.
2. Steps
01Read the test options
Objective
Confirm the command can test the whole job or one concern at a time.
Actions
- Run the subcommand help.
- Locate
--only-sources,--only-destinations, and--only-transform.
hdrctl test --help
The help exposes the three focused test flags and explicit manifest paths.
Usage: hdrctl test [OPTIONS] [PATH] Options: --only-sources --only-destinations --only-transform -s, --sources FILE -d, --destinations FILE
02Test the complete job
Objective
Exercise every non-writing readiness check with one command.
Actions
- Run hdrctl test on
first-job. - Wait for the source, destination, transformation, and environment sections.
hdrctl test first-job
Hydra recognizes the local CSV source, replace-mode destination, and all three transformations.
🔍 Test job first-job Sources ok sources.yaml — DSL valid ok src_csv — csv (local file) Destinations ok destinations.yaml — DSL valid ok dest_csv — mode replace Transformations ok transformations.yaml — 3 step(s) valid Operations: select, cast, filter
03Read the environment result
Objective
Confirm that this file-only job does not depend on hidden credentials.
Actions
- Read the environment section in the same report.
- Treat the missing
.envmessage as informational because no manifest contains an environment placeholder.
hdrctl test first-job
Hydra reports no local environment file, then still marks every test successful.
Environment variables
.env not found — ${ENV:...} variables will not be resolved
✅ All tests pass — ready to execute.
04Confirm the test did not write
Objective
Verify that readiness checking leaves destination data untouched.
Actions
- List the job data directory.
- Confirm the source remains available and no new output is required from this command.
Get-ChildItem first-job\data
The input remains present; hdrctl test does not execute the pipeline.
input.csv
Tips and traps
Keep the command beside its result. A verdict alone proves nothing a week later.
The Valid badge means the manifest parses. Only a run proves the job works.
Hydra resolves relative paths from the job folder, not from where you typed.
Trust the exit code, not the wording. A reassuring message can follow a failure.
When a command writes a file, open the file. Output can be stale or partial.
Durations and row counts change between runs. Never quote them as thresholds.
3. Checklist
| Verification | Expected result |
|---|---|
| Source result | src_csv — csv (local file) |
| Destination result | dest_csv — mode replace |
| Transformation count | 3 step(s) valid |
| Final result | All tests pass — ready to execute. |
4. Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| The source file is unavailable | The table path is relative to the job | Keep data/input.csv inside first-job |
| A transformation count differs | The cast edit was not saved | Restore the ordered select, cast, and filter steps |
| An environment variable is unresolved | A manifest now uses an environment placeholder | Create a local .env with the required key; never commit its secret value |
5. Next lesson
The job is now valid and connector-ready without having written a destination. The next lesson performs the first real run, reads the observed counters, and opens the CSV Hydra produced.