Skip to content
Merulan

Sources and media

Sources and media keep the input and review evidence used before a Pipeline writes Records.

Sources and media hold the input and review evidence for Data Pipelines. A Source names where data comes from. Media holds artifacts and proposed values for review. Neither a connection nor a review decision proves that a Pipeline wrote Records.

Ownership

A Source belongs to a Workspace. It keeps its connection, scope, status, and discovered schema. The source nodes in a Pipeline version choose which inputs feed its mapping. One Source can feed several Pipelines. One Pipeline can read several Sources. Media keeps the evidence behind proposed values. A reviewer can inspect that input before accepting a candidate.

Lifecycle

A Source can be active, archived, or unavailable. Its schema snapshot records what discovery found for an object at a given time. Connection health, schema discovery, and a finished Pipeline Run answer different questions. A healthy connection alone says nothing about the mapping or its output.

Media candidates receive an assignment and a versioned decision: accept, edit, reject, or abstain. Only an accepted decision enables promotion. The write shows an effect reference when it succeeds. A pending decision or pending promotion is still work in progress.

Boundaries

A Source identity names a reusable input, not one request to process it. A Run request key names one attempt to start work over chosen inputs. Runs and lineage explains how request keys recover an uncertain start. Retain the returned Run handle for Run readback.

Media review is not a Pipeline Run. Accepting a candidate records a review decision. Promotion applies the accepted value. Use the effect reference to distinguish those results. The Source's discovery snapshot likewise describes input fields without proving that any Record was written.

Failures

A Source can exist even when its connection cannot run. The Console shows the connection's available actions and the reason an action is unavailable. A listed connection type is not proof that live reads are supported. Missing schema or sample evidence stays visible as missing.

A media decision can conflict with another reviewer's change. The queue restores the current decision and version. The refused edit is not applied, so the reviewer must inspect that version before deciding again.

Recovery

A Source failure needs the repair named by its state. Restore connection access, correct the scope, or run supported discovery. Recheck the Source before using it in a Pipeline. A missing snapshot calls for discovery. A failed Pipeline Run calls for Run recovery.

For media, inspect the current candidate and its evidence after a review conflict. Decide against that version, then check promotion's effect reference. Do not treat an accepted review as a completed write.