> ## Documentation Index > Fetch the complete documentation index at: https://docs.maia.ai/llms.txt > Use this file to discover all available pages before exploring further. # Streaming runner installation on Azure export const maia = "Maia"; export const s_runner = "Streaming runner"; export const RunnerMetadata = ({runnerType, platforms = []}) => { return
Runner type {runnerType}
Runner platform
{platforms.map((platform, i) => {platform} ✅ )}
; }; This topic details how to install a {s_runner} in your Azure infrastructure using an ACI template provided by Matillion. It's possible to install multiple agents into a single resource group. Multiple agent installations require you to provide a different name for each container instance you launch a template into. Otherwise, the process is identical to that described here. For convenience, you might want to [download and amend the template](/docs/streaming/amend-azure-arm-streaming-template) so that the agents you install will have consistent properties. For troubleshooting and frequently asked questions encountered in this process, read [Azure agent troubleshooting](/docs/streaming/azure-streaming-agent-troubleshooting). *** ## Resources deployed The agent installation process will deploy the following resources in your Azure environment: * [Resource group](#create-a-resource-group): A new or existing resource group to contain all deployed resources. * [Virtual network and subnet](#create-a-virtual-network): A virtual network with a specified subnet size (/23 or larger). * [Key vault](#create-a-key-vault) (optional): A key vault for storing secrets. If you prefer, the agent can use an existing key vault. * Managed identity: A managed identity to access the key vault and other resources securely. * Container instance: The container instance that runs the Matillion agent. * Log analytics workspace: A workspace for collecting and analyzing logs from the deployed resources. * Role assignments: Various role assignments to ensure appropriate permissions for the deployed resources. * Other associated resources: Additional resources required for the deployment, such as storage accounts and container registries, depending on the configuration specified in the template. *** ## Prerequisites To get started, you'll need: * A [{maia}](https://app.matillion.com/hub/register) account. To register, read [Registration](/docs/administration/registration). * An Azure subscription with appropriate permissions to provision cloud resources in the Azure environment and manage access control, specifically for managing resource groups, virtual networks, key vaults, and container instances. Read [ARM template permissions](/docs/streaming/arm-streaming-template-permissions) for details. * A suitable resource group and virtual network already defined in your Azure environment. Instructions for creating these—if they don't already exist—are included below. * Optionally, a suitable key vault already defined in your environment. This isn't a mandatory requirement, as the installation process will create a new key vault by default. We recommend that you read the documentation and prerequisites before beginning this process. You may require input from your organization's cloud administrator for access and permissions. *** ## Create a resource group You need an Azure resource group that will contain all of the Azure resources for the agent. The agent deployment template requests the name of the resource group to deploy into and will create all resources there. If you don't have a suitable existing resource group, create a new one as follows: 1. Log in to the [Azure Portal](https://portal.azure.com/#home). 2. Use the search bar to search for "resource groups", and click the **Resource groups** result. 3. On the **Resource groups** screen, click **+ Create**. 4. On the **Create resource group** screen, select the **Subscription** you want to use for the agent from the drop-down. 5. Enter a new name for the **Resource group**. 6. Select an Azure **Region** for your resource group. 7. If your organization has a policy that mandates resource group tags, click **Tags** and assign tags as required. 8. Click **Review + create**, and then click **Create** if you are satisfied with your resource group configuration. For more information, read [Manage Azure resource groups by using the Azure portal](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) in the Microsoft developer documentation. *** ## Create a virtual network You need a virtual network to secure access between the agent and other resources, reducing the scope of possible access to those resources. If you don't have a suitable existing virtual network, create a new one as follows. Read the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-network/quick-create-portal) for more details. 1. Log in to the [Azure Portal](https://portal.azure.com/#home). 2. Use the search bar to search for "virtual networks", and click the **Virtual networks** result. 3. On the **Virtual networks** screen, click **+ Create**. 4. On the **Create virtual network** screen, select the [resource group](#create-a-resource-group) you created previously. 5. Enter a new **Virtual network name**. 6. Select an Azure **Region** for your virtual network. By default, this should be the same region as your resource group. 7. Click the **IP addresses** tab and edit the default subnet, or create a new subnet. 1. The virtual network's IP range (CIDR) should be compatible with any other networks that it may be peered with (either other virtual networks or on-premises networks via site-to-site tunnel). If the virtual network is new and doesn't require connectivity to other networks, the default CIDR range can be used. 2. The subnet must be sized appropriately. For example, if you have only three IP addresses (subnet IP range `/29`), use subnet IP range `/28` or above, so you can have at least one IP buffer per container group. This avoids having containers in states such as stuck or not able to start. For more information about working with subnets, read [Add, change, or delete a virtual network subnet](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-manage-subnet) in the Azure documentation. 3. Select a **NAT gateway**, or create a new one. Read the [Azure documentation](https://learn.microsoft.com/en-us/azure/virtual-network/nat-gateway/manage-nat-gateway?tabs=manage-nat-portal#create-a-nat-gateway-and-associate-it-with-an-existing-subnet) for detailed instructions on creating a NAT gateway. 4. Configure the NAT gateway with the allowed IPs required for your specific use case. Refer to the IP allow list [documentation](/docs/security/network-access-and-ip-allowlist-requirements#data-loader-cdc) for information about configuring allowed IPs. 5. Select the required **Service Endpoints**. These are required if the subnet doesn't have access to the Internet (via Internet gateway or NAT gateway). As a minimum, the following service endpoints will be required: * `Microsoft.KeyVault` * `Microsoft.Storage` 6. Configure **Subnet Delegation** with the following permission: * `Microsoft.ContainerInstance/containerGroups` This delegation allows the deployment of container groups within the subnet. For more details, read the [Azure documentation on subnet delegation](https://learn.microsoft.com/en-us/azure/virtual-network/manage-subnet-delegation?tabs=manage-subnet-delegation-portal). 7. Click **Save** to save the subnet configuration. 8. If your organization has a policy that mandates virtual network tags, click the **Tags** tab and assign tags as required. 9. Click **Review + create**, then click **Create** if you are satisfied with your virtual network configuration. When deploying the template, you need to provide the following extra parameters: * **networkType:** A string parameter that defines the agent network deployment type. The allowed values are "private" and "public". The default value is "public", however the agent doesn't require public access, and public access isn't recommended. * **vNet:** A string parameter that specifies the target virtual network for the agent deployment. This parameter is only required if the private subnet is being used. * **subnetName:** A string parameter that specifies the target subnet name for the agent deployment. This parameter is only required if the private subnet is being used. *** ## Create a key vault The agent requires access to an Azure key vault. However, you don't need to create a key vault yourself; the deployment template will automatically create a key vault in the same resource group as the agent. If you prefer, the agent can reference a different key vault you have previously created. The existing key vault may be in the same or a different resource group. You might want to reference a key vault in a different resource group so that you can have multiple agents referencing the same secrets. If you are using an existing key vault, you will need to complete the configuration steps described in [Configuring a key vault for Azure agent](/docs/streaming/azure-arm-streaming-agent-key-vault). *** ## Launch the ARM template in Azure Ensure that the prerequisites of [resource group](#create-a-resource-group), [virtual network](#create-a-virtual-network), and [key vault](#create-a-key-vault) (if required) have all been created, and that you are logged in to an Azure account with sufficient [permissions](/docs/streaming/arm-streaming-template-permissions) to deploy the template in a container instance. 1. In {maia}, follow the steps to [create a new agent](/docs/streaming/create-streaming-agent), with **Streaming** as the agent type and **Azure** as the cloud provider. 2. On the **Runner details** screen, click **Reveal credentials** in the **Credentials** section and note the **client\_id** and **client\_secret**. 3. Click the **Launch** button in the **Install using ARM** section. This will open the Azure portal at the **Custom deployment** page in a new browser tab. 4. Complete the following details on the **Custom deployment** page. | Property | Description | | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Subscription | Select the subscription you will deploy the agent into. | | Resource group | Select the [resource group you created previously](#create-a-resource-group). | | Region | Select the Azure region you will deploy the agent to, or leave the default value to deploy the agent in the same region as the selected resource group. | | Resource Prefix | The prefix used for all resources deployed by this process. | | Location | This is set automatically by the template and shouldn't be changed. | | Account ID | This is set automatically by the template and shouldn't be changed. It should be the same as the **ACCOUNT\_ID** parameter shown on the **Runner details** screen. | | Agent ID | This is set automatically by the template and shouldn't be changed. It should be the same as the **AGENT\_ID** parameter shown on the **Runner details** screen. | | Matillion Cloud Region | This is set automatically by the template and shouldn't be changed. It should be the same as the **MATILLION\_REGION** parameter shown on the **Runner details** screen. | | CPU Core | This is set automatically by the template and shouldn't be changed. | | Memory Size | This is set automatically by the template and shouldn't be changed. | | Existing Key Vault URI | To connect to an already existing Key Vault, enter the URI of the vault. If you leave this blank, the template will create a new Key Vault. | | Client ID | Enter the **client\_id** credential you obtained from the **Runner details** screen. | | Client Secret | Enter the **client\_secret** credential you obtained from the **Runner details** screen. | | Existing Managed Identity Name | The name of an existing Managed Identity. Leave this empty if you want the template to create a new Managed Identity. | | Existing Managed Identity Resource Group | The resource group where the existing Managed Identity is located. Leave this empty if the template is creating a new Managed Identity. | | Network Type | The network deployment type to be used by the agent. Select **Public** or **Private**. | | Virtual Network Name | Enter the name of the [virtual network you created previously](#create-a-virtual-network). Leave this blank if the **Network Type** is **Public**. | | Virtual Network Resource Group | The resource group where the Virtual Network is located. Leave this blank if the **Network Type** is **Public**. | | Subnet Name | Enter the name of the [subnet you created previously](#create-a-virtual-network). Leave this blank if the **Network Type** is **Public**. | | Tags | Optionally, add any tags that your organization's tagging policy requires. Tags should be added as a JSON string, for example: `{"Category": "Development","Business Unit": "Data Pipelines","Owner": "owner@organization.com"}`. | 5. Click **Review & Create**. *** ## Environment variables Environment variables are properties that are passed to the {s_runner} to configure how it operates. All necessary environment variables are automatically set by the template during agent deployment, with no further action required on your part. However, if you choose to deploy the agent manually, you will need to set these environment variables yourself. The following table lists the required environment variables and where to find their values. | Key | Value | | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | ACCOUNT\_ID | The **ACCOUNT\_ID** value from the **Parameters** tab of the **Runner details** page. | | AGENT\_ID | The **AGENT\_ID** value from the **Parameters** tab of the **Runner details** page. | | MATILLION\_REGION | The **MATILLION\_REGION** value from the **Parameters** tab of the **Runner details** page. | | OAUTH\_CLIENT\_ID | The **Client ID** value from the **Credentials** tab of the **Runner details** page. Click **Reveal** to make the value visible before copying. | | OAUTH\_CLIENT\_SECRET | The **Client Secret** value from the **Credentials** tab of the **Runner details** page. Click **Reveal** to make the value visible before copying. | | SECRET\_PROVIDERS | Always set to `azure-key-vault:1`. | | AZURE\_SECRET\_KEY\_VAULT\_URL | The URL of your Azure key vault. This is typically in the following format: `https://.vault.azure.net`. |