Data Pipelines
Data Pipelines brings outside data into the Company Schema. Sources name the inputs. Pipelines map them. Runs keep what was read and written.
Boundary
Data Pipelines owns input, mapping, validation, and promotion. The Company Schema owns the definitions that its targets must match. A Source connection does not prove that a mapping ran. A completed preview does not prove that Records were written. Standing quality checks assess evidence separately from a Pipeline's checks during execution.
Services
Data Pipelines connects reusable inputs to versioned mappings and readable execution evidence. Choose the service that owns the fact you need.
A Source feeds a Pipeline. A Pipeline owns its Runs. A Run writes Records into the Company Schema and keeps its artifacts.
| Service | Use it to understand |
|---|---|
| Sources and media | Connections, discovered fields, and candidate review before a write. |
| Pipelines | The saved mapping, validation, identity, and write policy. |
| Runs and lineage | Execution status, request identity, artifacts, and the evidence path. |
Start here
Use Connect and map a Source when Data Pipelines needs a checked input and mapping. Use Promote and troubleshoot when a preview is ready for a write or an execution needs repair. For a lost start response, follow request identity before starting different work.
The Console holds Sources, Pipelines, Runs, Media, and Lineage. The start contract owns admission fields and request keys. MCP can start a Run. Run readback is available through the API and Console. MCP does not expose it. Use the relevant native Surface guide to check whether it is available there.
Next steps
- Run a data pipeline connects the Console to the pages that own a first input, its execution, and its readback.
- Records & Lists owns the Records a Run promoted and the Lists that collect them.
- Quality & Measurement supplies standing assessments when the loaded evidence needs a rule checked.