Runs and lineage
Runs and lineage explain what happened when Data Pipelines processed an input. A Run executes one Pipeline version. Lineage connects that Run to the inputs and outputs its evidence can establish. Read the Run to learn its result, then trace its evidence to understand the path.
Ownership
A Run belongs to its Pipeline and Workspace. The Run keeps the version, mode, trigger, actor, status, row counts, errors, and artifact references. Raw rows live in artifacts, not in the Run's status fields. Read a Run for those facts. An Activity entry cannot replace the Run's result or lineage.
States
A Run moves through queued, validating, and running before it settles. The terminal status describes execution. The mode tells you whether that execution could promote Records. A completed dry run therefore proves a finished preview, not a write.
| Terminal status | Meaning for readback |
|---|---|
completed | The Run finished; inspect counts and artifacts for its result. |
partial_failure | Some work succeeded; inspect the failed part before recovery. |
failed | Execution failed; read the error summary. |
cancelled | A decision to stop settled; promotion is denied. |
Request identity
A Run request key makes an uncertain start recoverable. Choose the key before sending the start request. The start contract owns its exact fields and durable identity. A matching replay returns the original Run handle. Changing the request under the same key is refused.
If a start response is lost, replay the start with the same key and request. Then read by runId, not by request key. Without a request key, each start is new work. Do not use a list of recent Runs as proof that an uncertain request was accepted.
Boundaries
Run lineage derives links from retained Sources, snapshots, artifacts, the Pipeline version, and declared targets. The graph can show several bound Sources. A snapshot appears only when the evidence can establish it. An absent link stays absent.
A Run's target link identifies an Entity or Relationship kind. It does not list every Record written to that kind. Use Records & Lists to inspect the resulting Records and their history.
Failures
A Run can fail while reading input, shaping rows, or promoting output. Row errors name a row and cause. A system abort records pipeline_run_aborted. Counts and artifacts may still explain work done before failure. Neither a failed status nor an empty preview alone proves that nothing changed.
Lineage can also be partial when an evidence read fails. Missing lineage is a limit on what the graph can explain, not a substitute result for the Run.
Recovery
A Run needs readback before retry. For a settled failure, inspect its mode, counts, error summary, and artifacts, then repair the named cause. Promote and troubleshoot separates replay of an uncertain request from a retry of failed work.
For lineage, open the Run's Open in explorer link in the Console. Restore any refused evidence read before using that graph to explain the missing path.
Related
- Pipelines explains the version and target rules a Run uses.
- Sources and media explains the input and review evidence behind a Run.
- The API guide explains how to read a Run when a caller needs status outside the Console.