Run the job and read the evidence
Execute the validated pipeline, use its counters as evidence, and inspect the destination file created by the run.
1. Objective
Goal
Run hdrctl run first-job for the first time. Confirm that Hydra reads all scaffolded input rows, filters the negative value, writes the remaining rows, and creates the configured CSV destination.
Keep working from the cli-course folder. The job is named first-job; every command and terminal excerpt in this lesson was run against that same project.
Prerequisites
- Lesson 4 completed and its checklist confirmed.
- The repository clone and hdrctl entry point from lesson 1.
- A terminal opened in the parent folder that contains
first-job.
2. Steps
01Reconfirm the destination path
Objective
Know which file the run is expected to create before execution.
Actions
- Open
first-job/destinations.yaml. - Read the
load.tableandmodevalues.
Get-Content first-job\destinations.yaml
The generated destination points to data/output.csv in replace mode.
destinations:
dest_csv:
type: csv
load:
table: data/output.csv
mode: replace
02Execute the pipeline
Objective
Read, transform, and write the cumulative CSV job.
Actions
- Run hdrctl run against
first-job. - Wait for the final status and both row counters.
hdrctl run first-job
The real execution succeeds and reports the observed input and output counts.
▶ Pipeline first-job — first-job ✅ Pipeline completed successfully in 2.0s Rows read : 3 Rows written: 2
03Open the generated CSV
Objective
Verify the exact rows and values written by the destination connector.
Actions
- Read
first-job/data/output.csv. - Compare it with the three-row input and identify the removed negative row.
Get-Content first-job\data\output.csv
Alice and Bob remain, and the cast is visible in their decimal values.
id,name,value 1,Alice,100.0 2,Bob,200.0
04Relate counters to transformations
Objective
Explain the two written rows from the declared sequence rather than from guesswork.
Actions
- Read the transformations in order.
- Confirm that
castchanges type andfilterremoves Charlie's negative value.
Get-Content first-job\transformations.yaml
The ordered list accounts for three rows read and two rows written.
- cast:
mapping:
value: float
- filter:
expr: "value > 0"
Tips and traps
Keep the command beside its result. A verdict alone proves nothing a week later.
The Valid badge means the manifest parses. Only a run proves the job works.
Hydra resolves relative paths from the job folder, not from where you typed.
Trust the exit code, not the wording. A reassuring message can follow a failure.
When a command writes a file, open the file. Output can be stale or partial.
Durations and row counts change between runs. Never quote them as thresholds.
3. Checklist
| Verification | Expected result |
|---|---|
| Terminal status | Pipeline completed successfully |
| Rows read | 3 |
| Rows written | 2 |
| Destination file | Contains Alice and Bob with decimal value fields |
4. Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| The filter raises a type error | value was not cast before comparison | Complete lesson 2 and place the float cast before the filter |
| The input file is not found | The job folder or relative table path changed | Run from the parent folder and keep data/input.csv under the job |
| The output has three rows | The filter expression was removed or changed | Restore expr: "value > 0" and run again |
5. Next lesson
The first execution is proven by status, counters, and destination contents. The next lesson repeats the same job with -vv to expose configuration, environment, and timing metrics without changing the pipeline.