Skip to main content
POST
/
v1
/
projects
/
{projectId}
/
streaming-pipelines
Create a streaming pipeline definition
curl --request POST \
  --url https://eu1.api.matillion.com/dpc/v1/projects/{projectId}/streaming-pipelines \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "agentId": "12345678-1234-1234-1234-123456789012",
  "name": "A Streaming Pipeline",
  "streamingSource": {
    "connection": {
      "host": "127.0.0.1",
      "password": {
        "secretName": "the-secret-name",
        "secretType": "AWS_SECRETS_MANAGER",
        "secretKey": "the-secret-key",
        "secretLocation": "the-secret-location"
      },
      "port": 8471,
      "username": "db2_streaming_user",
      "jdbcProperties": {
        "date format": "ymd"
      }
    },
    "tables": [
      {
        "schema": "SCHEMA_A",
        "table": "TABLE_A"
      }
    ],
    "type": "db2ibmi"
  },
  "streamingTarget": {
    "accountKey": {
      "secretName": "the-secret-name",
      "secretType": "AWS_SECRETS_MANAGER",
      "secretKey": "the-secret-key",
      "secretLocation": "the-secret-location"
    },
    "accountName": "abs-account-name",
    "container": "streaming-container",
    "type": "abs",
    "decimalMapping": "LEGACY",
    "prefix": "streaming-prefix"
  },
  "advancedProperties": {
    "max.batch.size": 10000
  }
}
'
{
  "advancedProperties": {
    "max.batch.size": 10000
  },
  "agentId": "12345678-1234-1234-1234-123456789012",
  "name": "A Streaming Pipeline",
  "projectId": "12345678-1234-1234-1234-123456789012",
  "streamingPipelineId": "12345678-1234-1234-1234-123456789012",
  "streamingSource": {
    "connection": {
      "host": "127.0.0.1",
      "password": {
        "secretName": "the-secret-name",
        "secretType": "AWS_SECRETS_MANAGER",
        "secretKey": "the-secret-key",
        "secretLocation": "the-secret-location"
      },
      "port": 8471,
      "username": "db2_streaming_user",
      "jdbcProperties": {
        "date format": "ymd"
      }
    },
    "tables": [
      {
        "schema": "SCHEMA_A",
        "table": "TABLE_A"
      }
    ],
    "type": "db2ibmi"
  },
  "streamingTarget": {
    "accountKey": {
      "secretName": "the-secret-name",
      "secretType": "AWS_SECRETS_MANAGER",
      "secretKey": "the-secret-key",
      "secretLocation": "the-secret-location"
    },
    "accountName": "abs-account-name",
    "container": "streaming-container",
    "type": "abs",
    "decimalMapping": "LEGACY",
    "prefix": "streaming-prefix"
  }
}

Authorizations

​
Authorization
string
header
required

a valid bearer token

Path Parameters

​
projectId
string<uuid>
required

The ID of the project in which to create the streaming pipeline definition

Body

application/json

The details of the streaming pipeline definition to create

​
agentId
string<uuid>
required

The ID of the agent that will run the streaming pipeline

Example:

"12345678-1234-1234-1234-123456789012"

​
name
string
required

The name of the streaming pipeline

Minimum string length: 1
Example:

"A Streaming Pipeline"

​
streamingSource
object
required

The details of the streaming source

​
streamingTarget
object
required

The details of the streaming target

​
advancedProperties
object

Any additional properties for the streaming pipeline

Example:
{ "max.batch.size": 10000 }

Response

Successfully created the streaming pipeline definition

​
advancedProperties
object

Any additional properties for the streaming pipeline

Example:
{ "max.batch.size": 10000 }
​
agentId
string<uuid>

The ID of the agent that will run the streaming pipeline

Example:

"12345678-1234-1234-1234-123456789012"

​
name
string

The name of the streaming pipeline

Example:

"A Streaming Pipeline"

​
projectId
string<uuid>

The ID of the project that contains the streaming pipeline definition

Example:

"12345678-1234-1234-1234-123456789012"

​
streamingPipelineId
string<uuid>

The ID of the streaming pipeline definition

Example:

"12345678-1234-1234-1234-123456789012"

​
streamingSource
object

The details of the streaming source

​
streamingTarget
object

The details of the streaming target