Skip to main content
Agent Type:
Hybrid
Agent Platform:
Snowflake ✅
An agent can be installed directly into your Snowflake account’s Snowpark Container Services, using a Snowflake Native App. The compute pool for the agent will consume Snowflake credits. For more details of credit consumption, read Compute pool cost in the Snowflake documentation. Snowflake is used to store secrets used by . For details of how to configure Matillion agent for Snowflake to use these secrets, read Secrets in Matillion agent for Snowflake.

​
Prerequisites

To deploy the agent to your Snowflake account, the Snowflake environment must be prepared. This requires the following actions in your Snowflake account. A user with the ACCOUNTADMIN system role will be required to perform these steps.

​
Create a new role

This is the role that the Container will run as, and is the role needed to perform all subsequent deployment and configuration for the agent.
  1. Create the role, using the following commands. 
    USE ROLE ACCOUNTADMIN;
CREATE ROLE <native_app_role>;
    Where <native_app_role> is the name you want the role to have.
  2. Grant permissions to the role. 
    GRANT ROLE <native_app_role> TO USER <user_name>;
GRANT CREATE INTEGRATION ON ACCOUNT TO ROLE <native_app_role>;
GRANT CREATE DATABASE ON ACCOUNT TO ROLE <native_app_role>;
GRANT CREATE APPLICATION ON ACCOUNT TO ROLE <native_app_role>;
GRANT CREATE COMPUTE POOL ON ACCOUNT TO ROLE <native_app_role> WITH GRANT OPTION;
    Where <native_app_role> is the name you gave to the role in step 1, and <user_name> is the user you want to grant the role to.

​
Set up a warehouse

A warehouse is required when activating the application and creating the agent, and will be used by the agent when completing tasks such as running pipelines. You may use an existing warehouse if a suitable one exists, or create a new warehouse for the agent to use.

​
Use an existing warehouse

To use an existing warehouse, you must give the new role USAGE permissions for the warehouse, as follows: 
GRANT USAGE ON WAREHOUSE <warehouse_name> TO ROLE <native_app_role>;
Where <warehouse_name> is the warehouse you wish to use, and <native_app_role> is the role you created previously.
As this warehouse will be used by the agent, it should have the appropriate suspend configuration and sizing.

​
Create and use a new warehouse

If you choose to create a new warehouse, it must be created by the role created earlier. The role will require the following additional permission to allow this: 
GRANT CREATE WAREHOUSE ON ACCOUNT TO ROLE <native_app_role>;
Where <native_app_role> is the role you created previously. You can then create the new warehouse as follows: 
CREATE OR REPLACE WAREHOUSE <native_app_wh> WITH
  WAREHOUSE_SIZE = 'X-SMALL'
  AUTO_SUSPEND = 180
  AUTO_RESUME = true
  INITIALLY_SUSPENDED = false;
Where <native_app_wh> is the name of the warehouse.
Warehouses consume Snowflake credits, so you should size the warehouse appropriately. For guidance, read CREATE WAREHOUSE in the Snowflake documentation.

​
Install the application

Once you have configured the appropriate prerequisites in your Snowflake environment, you can proceed to install and configure the Matillion agent for Snowflake.

​
Create the Data Productivity Cloud agent

  1. Create an agent in as described in Create an agent in your infrastructure.
  2. In the Cloud Provider drop-down, select Snowflake.
  3. The Deployment drop-down will display Native App. This can’t be changed.
  4. On clicking Create agent, you will see the Agent details screen that shows you the parameters you will need for installing into Snowflake:
    • ACCOUNT_ID
    • AGENT_ID
    • MATILLION_REGION
    • Client ID
    • Client Secret

​
Install into Snowflake

  1. Ensure you are using the role created previously.
  2. Open the Snowflake Marketplace.
  3. Click Private Sharing in the left-hand menu.
  4. Locate Matillion in the list of apps, and click to select it.
    If the app is not listed, contact your Matillion representative for advice.
  5. Click Request.
  6. In the Request this Data Product dialog, enter your Name, Title and Company Name.
  7. Click Request to submit the request. The button status will change to Requested.
  8. A Matillion representative will contact you to confirm the request. Once access is granted, return to this page and click Get to access the listing.
  9. If required, expand the Options panel in order to change:
    • The application name. This is optional, and you can keep the default name if you wish.
    • The warehouse to use for the installation. The warehouse should be the one created or selected previously.
  10. Click Get.
  11. Click Open to view the app, or Done to finish.
If the role you’re using to install and set up the application does not have the appropriate permissions (OWNERSHIP or USAGE) to the selected warehouse, the agent will not start.

​
Configure the application

  1. Open the application.
  2. You will be prompted to grant access to allow the application to create a compute pool and a network rule allowing all access to ports 80 and 443. You must grant this access. Once the permissions have been granted, you will be able to activate the application.
    These are the minimum ports required by the agents. If you need to open additional ports for any reason, follow the instructions in How to open more ports.
  3. Under Privileges to objects, add the Snowflake warehouse that the agent will use to run queries within the account.
  4. Click Launch App. On the initial configuration screen, enter the details you previously obtained from the Agent details screen in :
    • Matillion Region
    • Account ID
    • Agent ID
    • Client ID
    • Client Secret
  5. Leave the Environment property unchanged.
  6. You can leave the Default Secrets Schema unchanged (recommended), or change it to another schema of your choice. Read Secrets in Matillion agent for Snowflake for further details.
    If you uninstall the application, any secrets created within the listed schema will be deleted.
  7. Click Submit. This will start the agent.
The agent status will initially show PENDING. This should eventually move to a READY state, meaning the agent is connected and available to run pipelines. If the agent doesn’t change to a READY state, follow the guidance in the Troubleshooting document. When the agent is in a READY state, you can begin creating projects or environments that use the new agent.

​
Connecting an environment

When creating a environment using Matillion agent for Snowflake, you must use the Snowflake “Format 1” account identifier as the environment’s Account property. To get the correct Format 1 identifier, run the following query within the account: 
SELECT CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME();
For further details of Format 1, read Format 1 (preferred): Account name in your organization in the Snowflake documentation.

​
Installing multiple agents

You can run multiple agents, which may be a requirement for the level of performance and concurrency you need. You can do this by installing multiple Native App instances, with each instance running a single agent instance. For a given Snowflake account, up to 10 Native App instances can be installed. To increase the number of instances, open the installed app and click Add instance. Configure the new instance as described in the sections above. You can use the same role and warehouse for each app instance, or you can create new roles and warehouses for each.

​
Uninstalling the application

You can uninstall the application from the Apps area within Snowflake. From your Snowflake Home screen, click Data Products → Apps. You must be using the role that originally installed the application. Following an uninstall, some artefacts are left behind and must be manually removed. These are:
  • The <APPLICATION_NAME>_APP_DATA database.
  • The <APPLICATION_NAME>_ALL_EXTERNAL_ACCESS_EXTERNAL_ACCESS external integration.
To remove these, run the following commands: 
DROP DATABASE <APPLICATION_NAME>_APP_DATA CASCADE;
DROP INTEGRATION <APPLICATION_NAME>_ALL_EXTERNAL_ACCESS_EXTERNAL_ACCESS;
Where <APPLICATION_NAME> is the name of the application set during installation. By default, this is MATILLION_DATA_PRODUCTIVITY_CLOUD.

​
Updating agents

Matillion provides regular updates to both the Native App and the for Snowflake. Updates are released on the second Tuesday of every month, starting at 12:00 UTC and ending at 11:59 UTC on the second Wednesday. During this update window, Snowflake updates the agent. Any pipelines running during this time may fail if the update is in progress. The Matillion agent for Snowflake is only available on the Stable version track, and is updated once per month on the second Tuesday of the month. New agent releases may include new features, bug fixes, and security patches.