Semogram Docs
Data PipelinesReferenceNode reference

Ingress

Read records from a workspace source endpoint

A source endpoint identifies the external dataset and installed read capability. Ingress invokes that contract and produces records for downstream tasks. It does not create the external dataset or grant access.

Before using it

Create or select the real resources described above in the pipeline’s project/workspace, inspect the upstream data shape and ensure your actor can perform this operation. A configuration fragment cannot create those resources. Read First pipeline for a complete graph and fixture.

Fields and bindings

FieldMeaning
Data endpoint / dataEndpointIdSelect a workspace source endpoint; use its actual UUID for execution
Load mode / modeFull load or Incremental, subject to connector support
Extraction strategy / strategyparser, llm or hybrid; a prompt/model is needed for AI extraction where required
outputName of the intermediate record output
flowSupported checkpoint, batch and retry configuration
concurrency / anomalySupported read parallelism and row-count-drop threshold

Configure

Assistant prompt
Prepare the ingress node for the selected test resources. Use the operation and fields shown in this example. Show actual resource bindings, upstream/downstream ports and any write/model effects before applying the draft.

Select Data endpoint and Load mode in the ingress inspector. Inspect the loaded endpoint contract and extraction configuration. Parser is appropriate for the structured two-row SQL fixture; an unstructured-source task may require AI or hybrid extraction and a task-specific prompt.

Put this section under ingress on a full node of kind ingress. It is not a standalone API or MCP request. Replace resource placeholders, and provide the full node ports/policy and graph edges.

ingress section
{
  "dataEndpointId": "<SOURCE_ENDPOINT_UUID>",
  "mode": "full",
  "output": "orders",
  "strategy": "parser"
}

Verify a bounded run

The two-row orders fixture should retain IDs 1 and 2, their regions and totals. Inspect cursor/snapshot metadata and actual row output. A downstream step must receive the declared output port and correct binding.

Validate, save a version and run a small known fixture. Inspect the actual node result and downstream consumer, not only the graph preview.

Limits and failure behavior

Incremental, schema discovery and pagination are connector capabilities, not guaranteed by the node fields. Preflight does not read a sample. Keep database/API credentials on the installation.