Skip to main content
In this guide, you will learn how to use Matillion’s APIs to create and list secret references that point to the name and location of a secret stored in your cloud provider’s secrets manager. Secret references can represent any sensitive data, such as passwords, access tokens, OAuth credentials, and cloud credentials. You can then use these secret references to access databases or connectors when configuring a pipeline. For more information about secret references, read Secrets overview.

​
Prerequisites

Before you begin, make sure you have the following:

​
Create a secret reference

Use this API endpoint to create a new secret reference in a specified project. Currently, the only supported secretReferenceType is PASSWORD. If you use a Hybrid SaaS setup, you can create secret references that point to your cloud provider’s secret manager. If you use a Full SaaS setup, you can create secret references hosted by Matillion. To create a secret reference in any project, you first need to find and note your project ID and ID.

​
Find your project ID

Retrieve the list of projects using the Project API and copy the id value–this is your project ID. Base URL: GET /v1/projects Example response: 
{
  "page": 0,
  "results": [
    {
      "description": "project-1",
      "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
      "name": "Test-project"
    }
  ],
  "size": 0,
  "total": 0
}

​
Find your ID

Click the menu button in the top left of any screen, then click Manage → Agents. If you use a Hybrid SaaS setup (customer-hosted s), in the row for the , click the three dots … → Agent details. Scroll down to Agent environment variables and copy the AGENT_ID value. Alternatively, if you use a Full SaaS setup (Matillion-hosted s), hover over the and click Copy to clipboard next to the ID.

​
Create a secret reference for an AWS customer-hosted project

To create a secret reference for a project connected to the AWS Secrets Manager, you will need your project ID, ID, and AWS Secrets Manager secret key and secret name. Base URL: POST /v1/projects/{projectId}/secret-references/{secretReferenceName} Configure the following variables:
  • projectId: Enter the target project ID.
  • secretReferenceName: Enter a name for the new secret reference.
Example request body: 
{
  "agentId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "agentType": "AWS",
  "description": "My secret reference",
  "secretReferenceType": "PASSWORD",
  "secretKey": "aws-secret-key",
  "secretName": "aws-secret-name"
}

​
Create a secret reference for an Azure customer-hosted project

To create a secret reference for a project connected to an Azure Key Vault, you will need your project ID, ID, and Azure Key Vault secret key and secret name. Base URL: POST /v1/projects/{projectId}/secret-references/{secretReferenceName} Configure the following variables:
  • projectId: Enter the target project ID.
  • secretReferenceName: Enter a name for the new secret reference.
Example request body: 
{
  "agentId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "agentType": "Azure",
  "description": "My secret reference",
  "secretReferenceType": "PASSWORD",
  "secretName": "azure-secret-name",
  "vaultName": "azure-vault-name"
}

​
Create a secret reference for a Matillion-hosted project

To create a secret reference that is hosted and managed by Matillion, you will need your project ID and the ID of your Matillion-hosted . Base URL: POST /v1/projects/{projectId}/secret-references/{secretReferenceName} Configure the following variables:
  • projectId: Enter the target project ID.
  • secretReferenceName: Enter a name for the new secret reference.
Example request body: 
{
  "agentId": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "agentType": "Matillion hosted",
  "description": "My secret reference",
  "secretReferenceType": "PASSWORD",
  "secretValue": {"password":"password-plain-text-value"}
}

​
List secret references

Use this API endpoint to retrieve a list of secret references within a specific project. It supports pagination for large result sets and optional filtering by secret reference type and providers. If no value is provided for secretReferenceTypes, only PASSWORD types will be returned. To retrieve a list of secret references, you will need your project ID—see Find your project ID for how to find this—and any secret reference types that you want to use to filter the results.

​
List secret references for an AWS or Azure customer-hosted project

To list secret references for an AWS or Azure customer-hosted project, you will need your project ID. The response body will vary based on where the used in the project is hosted, e.g. AWS or Azure. Base URL: GET /v1/projects/{projectId}/secret-reference Configure the following variables:
  • projectId: Enter the target project ID.
The example response for an AWS customer-hosted project is shown below. For an Azure customer-hosted project, the values for vaultSecretKey and vaultSecretName will be your Azure secret key and secret name. The vaultType value will be Azure. 
{
"page": 1073741824,
  "results": [
    {
      "description": "My secret reference",
      "name": "Data-Productivity-Cloud-secret-name",
      "vaultSecretKey": "aws-secret-key",
      "vaultSecretName": "aws-secret-name",
      "type": "PASSWORD",
      "vaultType": "AWS"
    }
  ],
  "size": 1073741824,
  "total": 9007199254740991
}

​
List secret references for a Matillion-hosted project

To list secret references for a Matillion-hosted project, you will need your project ID. Base URL: GET /v1/projects/{projectId}/secret-reference Configure the following variables:
  • projectId: Enter the target project ID.
Example response: 
{
  "page": 1073741824,
  "results": [
    {
      "name": "my-secret",
      "secretReferenceType": "PASSWORD",
      "vaultType": "Matillion hosted"
    }
  ],
  "size": 1073741824,
  "total": 9007199254740991
}

​
Copy secret references from one project to another

Use this API endpoint to list the secret references in one project (the source project). A post-response import script will then build the request body to create these secret references in another project (the destination project). Before you begin, make sure you have the following:
  • Postman to enter environment variables and the post-response import script.
  • The project IDs for the source project and destination project—see Find your project ID for how to find these project IDs.
  • The ID for the destination project–see Find your ID for how to find this.
To copy secret references from one project to another, follow these steps:
  1. Make a request using the following base URL, entering the destination project ID as the projectId variable: GET /v1/projects/{projectId}/secret-references.
  2. Add the post-response import script shown below in the Scripts → Post-response section and configure the following variables:
    • agentId: Enter the ID for the destination project.
    • projectId: Enter the destination project ID.
    • secretsData: Leave this empty—this will be populated by the post-response import script.
  3. Click Send.
Post-response import script: 
// Extract secrets from API response
let secrets = pm.response.json().results || [];

// Format secrets and store in an environment variable
pm.environment.set("secretsData", JSON.stringify(secrets.map(secret => ({
    secretReferenceName: secret.name,
    secretReferenceType: secret.secretReferenceType,
    secretKey: secret.vaultSecretKey,
    secretName: secret.vaultSecretName,
    agentId: pm.environment.get('agentId'),
    agentType: secret.vaultType
}))));

// Send create requests
JSON.parse(pm.environment.get("secretsData")).forEach(secret => {
    let { secretReferenceName, ...body } = secret;

    let requestUrl = `${pm.environment.get('public_api_host')}/dpc/v1/projects/${pm.environment.get('projectId')}/secret-references/${secretReferenceName}`;

    pm.sendRequest({
        url: requestUrl,
        method: "POST",
        header: {
            "Content-Type": "application/json",
            "Authorization": `Bearer ${pm.environment.get('jwt')}`
        },
        body: { mode: "raw", raw: JSON.stringify(body) }
    }, (err, res) => console.log(err ? "Request failed:" : "Request successful:", err || res));
});

​
Delete a secret reference

Use this API endpoint to delete an existing secret reference. To delete a secret reference, you will need the ID of the project containing the secret reference and the name of the secret reference. Base URL: DELETE /v1/projects/{projectId}/secret-references/{secretReferenceName} Configure the following variables:
  • projectId: Enter the target project ID.
  • secretReferenceName: Enter the name of the secret reference to delete.