Edge Delta Google Cloud Pub/Sub Source

Overview

The Pub/Sub Source node allows the Edge Delta agent to read data from Google Cloud pub/sub topics. This node is essential for ingesting log data published on pub/sub and processing it within the Edge Delta ecosystem.

• outgoing_data_types: log

Example Configuration 1

This configuration retrieves messages using a directly provided authentication key.

nodes:
- name: google_pubsub_with_key
  type: pubsub_input
  project_id: "projectID1"
  sub_id: "subID1"
  key: '{"pubsub_key": "key_123"}'

Example Configuration 2

This configuration pulls messages using a file path to an authentication key.

nodes:
- name: google_pubsub_with_key_path
  type: pubsub_input
  project_id: "projectID2"
  sub_id: "subID2"
  key_path: "pubsub_key_path"

Required Parameters

name

A descriptive name for the node. This is the name that will appear in Visual Pipelines and you can reference this node in the YAML using the name. It must be unique across all nodes. It is a YAML list element so it begins with a - and a space followed by the string. It is a required parameter for all nodes.

nodes:
  - name: <node name>
    type: <node type>

type: pubsub_input

The type parameter specifies the type of node being configured. It is specified as a string from a closed list of node types. It is a required parameter.

nodes:
  - name: <node name>
    type: <node type>

project_id

The project_id parameter specifies the Project ID of the topic/subscription in Google Cloud. It is a string and is required.

nodes:
- name: <node name>
  type: pubsub_input
  project_id: <your project ID>

sub_id

The sub_id parameter indicates the Pub/Sub subscription from which messages will be pulled. It is a string and is required

nodes:
- name: <node name>
  type: pubsub_input
  sub_id: <your subscription ID>

Optional Parameters

key

The key parameter is a password value required for authentication if key_path is not provided. This key will be used by agents to authenticate calls to Pub/Sub.

nodes:
- name: <node name>
  type: pubsub_input
  key: '{"pubsub_key": "your_key"}'

key_path

The key_path parameter specifies the file path for the key used in the authentication process. It is an alternative to the key parameter.

nodes:
- name: <node name>
  type: pubsub_input
  key_path: "/path/to/keyfile"