News/Web

Use @craven/news-web to read RSS feeds, sitemaps, web pages, generic news APIs, article URLs, and source lists into normalized article records.

Install

Before you start

You need a Semogram account with access to the workspace and permission to manage plugin installations. You also need access to the external system; installing a plugin does not grant credentials or provision it.

Choose a source mode and a reachable feed or URL. Start with a single RSS feed or article. Add API authentication only when the provider requires it and define crawl and request limits before expanding the scope.

Configure the installation

1. Open Plugins in your workspace and select Explore.
2. Find News/Web, inspect its publisher, version and capabilities, and choose the matching capability.
3. Give the installation a useful name and fill in its connection configuration.
4. Save it and run the connection check where the capability provides one. Read the returned error before proceeding.

This is an illustrative configuration. Replace every example value; never paste real credentials into an assistant prompt or public document.

Install the plugin described on this page in this workspace. Discover its catalog entry, select the matching capability and propose the installation using the connection settings shown here. Ask me to enter credentials in protected installation fields. Show the selected plugin/version, capability and non-secret settings before saving.

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "plugin_installation_create",
    "arguments": {
      "capabilityClass": "<MATCHING_CAPABILITY_CLASS>",
      "discoveryId": "<DISCOVERY_ID_FROM_CATALOG>",
      "name": "<INSTALLATION_NAME>",
      "config": {
        "source": {
          "mode": "rss_feed",
          "feedUrl": "https://example.com/news/feed.xml"
        }
      },
      "idempotencyKey": "<UNIQUE_KEY_FOR_THIS_INSTALLATION>"
    }
  }
}

Configuration fields

View the complete published contracts for nested settings and endpoint selectors. Inspect the installed version’s contract before configuring optional fields; catalog availability and versions can vary by deployment.

Make your first read

Follow the data endpoint scenario for target setup and verification.

Open Data Endpoints, choose the installed capability and configure a concrete target. For this plugin, an illustrative target is:

Create the endpoint described on this page using these settings:
name: <ENDPOINT_NAME_FROM_THIS_EXAMPLE>
namespace: <ENDPOINT_NAMESPACE_FROM_THIS_EXAMPLE>
role: source
contractKind: source_stream
target / type: articles
pluginCapabilityInstallationId: <INSTALLED_CAPABILITY_UUID>
Use the actual installed capability and the endpoint name/namespace selected in this example. Show the proposed direction, connection and target before saving. Keep credentials on the installation.

curl --request POST "https://platform.semogram.com/api/v1/data-endpoints" \
  --header "Authorization: Bearer ${SEMOGRAM_API_KEY}" \
  --header "Idempotency-Key: <UNIQUE_KEY_FOR_THIS_ENDPOINT>" \
  --header "Content-Type: application/json" \
  --data-binary @- <<'JSON'
{
  "name": "<ENDPOINT_NAME_FROM_THIS_EXAMPLE>",
  "namespace": "<ENDPOINT_NAMESPACE_FROM_THIS_EXAMPLE>",
  "role": "source",
  "contractKind": "source_stream",
  "target": {
    "type": "articles"
  },
  "pluginCapabilityInstallationId": "<INSTALLED_CAPABILITY_UUID>"
}
JSON

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "source_create",
    "arguments": {
      "name": "<ENDPOINT_NAME_FROM_THIS_EXAMPLE>",
      "namespace": "<ENDPOINT_NAMESPACE_FROM_THIS_EXAMPLE>",
      "role": "source",
      "contractKind": "source_stream",
      "target": {
        "type": "articles"
      },
      "pluginCapabilityInstallationId": "<INSTALLED_CAPABILITY_UUID>",
      "idempotencyKey": "<UNIQUE_KEY_FOR_THIS_ENDPOINT>"
    }
  }
}

Read a small sample and inspect article URL, title, body, publication time and source information. Compare an article with the actual source page; some sites provide only summaries or prevent automated fetching.

Installation is not ingestion. Use the saved endpoint in a supported query or a small pipeline, validate the pipeline, execute it and inspect its run. Add a schedule only after the bounded run succeeds.

Manage the connection

Supported source modes are rss_feed, article_url, sitemap, web_page, api and source_list. API mode can use itemsPath and field mappings. Web-page crawling uses crawlDepth and maxLinks; keep limits explicit.

Change connection settings on the installation and target selectors on the endpoint. Recheck the connection after credential rotation. Review consuming endpoints and pipelines before replacing a capability or removing its installation.

For assistant-driven setup, use catalog discovery, installation creation and installation checks. Use real IDs returned by discovery, not the example name as an ID.

FAQ

Why does the check or first run fail?

Check URL reachability, provider authentication, rate limits and parsing. For API mode, inspect the response array and mapping. Missing article text may reflect the provider’s content rather than a connection error. This is a read connector, not a general-purpose write destination.

Does installing this plugin start a sync?

No. A check verifies a supported connection probe. Queries and pipeline runs perform reads; recurring ingestion needs a schedule. Inspect run status, records and evidence to confirm actual work.

Can an assistant use every operation after installation?

Only operations supported by the capability and permitted for the caller. The MCP tool reference marks plugin and connector requirements; declaring a write capability does not grant every source-write or maintenance operation.

What should I verify before using production data?

Check a known small input, the expected result, permissions, supported read/write behavior and failure handling. These guides describe the implemented contracts; they are not a claim that your external system has already passed a live connection test.