Skip to main content
This topic explains how to create a Maia runner in your own infrastructure. For a discussion of why you might want to deploy in your own infrastructure rather than use a Matillion Full SaaS solution, read Maia runner overview. We currently support Maia runners running in Snowflake, AWS, Azure, or Google Cloud environments. There are two different types of Maia runner: This article covers Maia runners only. For Streaming runners, read Create a Streaming runner. The installation process is similar for both types of Maia runner, but you must select the correct Maia runner type for the type of pipeline you want to run. You can install both Maia runners in the same infrastructure if you intend to run both types of pipeline. Read Scaling best practices to understand how to size your Maia runner for the level of performance you need.

​
Prerequisites

  • A Maia account. To register, read Registration. Once you have signed up, log in to the Maia.
  • An account in Snowflake, AWS, Azure, or Google Cloud to host the Maia runner.

​
Create a Maia runner

  1. In the left navigation, click . Then, select Runners from the menu. You will see a list of all Maia runners currently created.
  2. Click Add runner.
  3. Click Maia.
  4. Complete the following properties:
    • Runner name: A unique name for your new Maia runner. Maximum 30 characters. Accepts both uppercase and lowercase A-z, 0-9, whitespace (not the first character), hyphens and underscores.
    • Description: Optionally enter a brief description of the Maia runner.
    • Version track: Select the version track that this Maia runner will use: Current or Stable. Read Version tracks for details of this option.
    • Cloud provider: The cloud platform that the Maia runner will be deployed to. Currently, Snowflake, AWS, Azure, and Google Cloud are supported.
    • Deployment: The supported deployment method for the given cloud provider. Currently, Native App for Snowflake, Fargate (ECS) or EKS for AWS, Container App or AKS for Azure, and GKE for Google Cloud are supported.
    • Auto update: Select whether you want your Maia runner to automatically update or not. Read Auto update for details of this option.
  5. Click Create runner.

​
Set up the Maia runner in your cloud infrastructure

When created with the above process, the Maia runner needs to be set up in your cloud infrastructure. There are several different ways of doing this, and you can use whichever method suits you: To complete these processes, you will require certain details of the created Maia runner. To obtain these details, locate the Maia runner in the Runners list of Maia runners, and click the three dots … next to it, then click Runner details. The parameters and values in the sections Runner image URI, Runner environment variables, and Credentials are required when configuring your Maia runner in your cloud infrastructure.

​
Check Maia runner status

After deploying the Maia runner in your cloud infrastructure, you should return to Maia to verify that it’s correctly connected and running.
  1. Click the menu button in the top left of any Maia page, then click Manage runners.
  2. Locate the Maia runner in the list and check the status:
    • Running: The Maia runner is connected and ready for development and pipeline tasks.
    • Unknown: The Maia runner is in an unknown state.
    • Pending: The Maia runner has been created but has not yet connected to Maia.
    • Stopped: The Maia runner has been stopped.
  3. When the Maia runner status shows Running, it’s ready to use.

​
Restarting your Maia runner

There are a number of reasons why you may need to restart your Maia runner, such as receiving Maia runner updates. To restart a Maia runner, follow the steps in Restart a Maia runner. In some cases, you may need or choose to pause the Maia runner before restarting.

​
Deleting a Maia runner

To delete a Maia runner from Maia, locate the Maia runner in the Runners list, then click the three dots … → Remove runner. Deleting the Maia runner from the Runners list doesn’t remove the underlying Snowflake, AWS, Azure, or Google Cloud resources. You should go into the relevant cloud console and clean up any resources that you no longer require.
  • Snowflake: Use Snowpark Container Services to manage and clean up any Snowflake resources associated with the Maia runner.
  • AWS: Use the AWS Console to identify and delete any resources associated with the Maia runner, such as EC2 instances and IAM roles.
  • Azure: Use the Azure Portal to locate and remove any related resources, such as virtual machines and resource groups.
  • Google Cloud: Use the Google Cloud Console to locate and remove any related resources, such as the GKE cluster, VPC, GCS bucket, Secret Manager secrets, and service accounts.
Deleting a Maia runner that is currently running may interrupt scheduled pipelines or pipelines that are currently running. Therefore, you should always stop the Maia runner service before deleting it.

​
Using the Maia runner to develop and run pipelines

To use a Maia runner for developing and then running pipelines, you need to select the Maia runner when you create an environment. See below for details of how to make Maia runners available to specific environments. You can change which Maia runner an environment uses at any time by editing the environment and selecting a different Maia runner. This is safe to do, as the change will only affect pipelines that are run after the change. So:
  • Your open Designer session won’t change.
  • Any pipelines that are currently running will continue to run using the old Maia runner.
  • Any existing schedules will continue to run using the old Maia runner. To use the new Maia runner, you would have to change the schedule.

​
Restricting a Maia runner

You can restrict which Maia runners are available to a specific project or environment. This is useful if you have multiple Maia runners and want to ensure that a project only uses specific ones. An unrestricted Maia runner (the default state when a Maia runner is created) can always be used by any project or environment. A restricted Maia runner can only be used by the projects and environments you specify. If you restrict a Maia runner but don’t specify any projects, the Maia runner won’t be accessible to any project. For details of how to restrict a Maia runner, read Restricting Maia runners.

​
Video