> ## Documentation Index > Fetch the complete documentation index at: https://docs.maia.ai/llms.txt > Use this file to discover all available pages before exploring further. # AWS Fargate manual setup for Maia runners export const m_runner = "Maia runner"; export const maia = "Maia"; export const RunnerMetadata = ({runnerType, platforms = []}) => { return
Runner type {runnerType}
Runner platform
{platforms.map((platform, i) => {platform} ✅ )}
; }; This page is a guide to manually installing a {m_runner} into AWS Fargate. This documentation refers to the **New ECS Experience** for Elastic Container Service. Each time you click **Create** in AWS, a colored banner will appear at the top of the AWS UI. * Blue indicates the creation of a resource is in-progress. * Green indicates the creation of a resource has completed. * Red indicates the creation of a resource has failed. *** ## Prerequisites * First you must create a {m_runner} from {maia}, as described in [Create a {m_runner}](/docs/guides/create-a-runner). This will provide you with the parameters and values needed to manually complete the {m_runner} setup in AWS Fargate. * Before deploying a {m_runner} in AWS, confirm the existence of a service-linked role for Amazon ECS in your AWS account. Typically, Amazon ECS generates this role automatically. If not, follow the [manual creation documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using-service-linked-roles.html#create-slr) for setup instructions. *** ## Security groups 1. Log in to the [AWS Console](https://aws.amazon.com/console/). 2. Once logged in, type `security groups` in the search bar. 3. In the **Features** list, select **Security Groups|EC2 feature**. 4. Choose **Create security group**. 5. Complete the following fields: * **Security group name:** Add the following: `matillion-runner-` Your {m_runner} ID can be found in the **Runner environment variables**. To find these, click **Manage runners**, then click the three dots **...** → **Runner details** for the corresponding {m_runner}. * **Description:** Add the following description: `runner security group for runner ID `. Alternatively, add a description of your choice. * **VPC:** Select your VPC. * **Inbound rules:** Add any inbound rules. None are required by default. * **Outbound rules:** Add `0.0.0.0/0`. * **Tags:** You may want to add a tag where **Key** = `matillion-runner-id` and **Value** = ``. 6. Click **Create security group** to confirm creation. On successful creation of the security group, you will be redirected to the new security group's dashboard. 1.1. 7. Make a note of your **Security group ID**. *** ## Log groups (optional) This part may be omitted if you want to use the default that can be created as part of the **Task definition**. 1. In the AWS console, type `CloudWatch` in the search bar and navigate to **AWS CloudWatch**. 2. In the left-hand menu, choose **Logs** → **Log groups**. 3. Click **Create log group**. 4. Complete the following fields: * **Log group name:** Add the following: `/ecs/matillion-runner/`. * **Retention setting:** Select a retention setting from the drop-down. **1 month (30 days)** is recommended. * **KMS key ARN - optional:** Not required. * **Tags:** You may want to add a tag where **Key** = `matillion-runner-id` and **Value** = ``. 5. Click **Create** to confirm creation of the log group. On successful creation of the log group, you will be redirected to **Log groups**. *** ## Cluster 1. In the AWS console, type `Elastic Container Service` in the search bar and choose that service. 2. Click **Get started** or **Clusters** if this is your first time using this service. 3. Click **Create Cluster**. 4. Complete the following fields: * **Cluster name:** Add the following: `matillion-runner-` * **VPC:** Select your VPC. * **Subnets:** Optional. Choose subnets where your tasks will run. * **Use Container Insights:** Optional, but recommended. Toggle this to **On**. * **Tags:** You may want to add a tag where **Key** = `matillion-runner-id` and **Value** = ``. 5. Click **Create** to confirm creation of the cluster. On successful creation of the cluster, you will be redirected to **All Clusters**. *** ## Task definitions 1. In the AWS console, type `Elastic Container Service` in the search bar and choose that service. 2. In the left-hand menu, click **Task definitions**. 3. Click **Create new task definition**. 4. Complete the following fields: * **Task definition family:** Add the following: `matillion-runner-`. This must be unique—two task definitions can't share a name. * **Name:** `matillion-runner`. * **Image URI:** Copy and paste your **Runner Image URI** from **Runner details**. * **Port mappings:** Remove any existing entries. 5. In **Environment Variables** under **Add individually**, add the following environment variables using the values in your **Runner details** page in {maia}: | Key | Value type | Value | | ----------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ACCOUNT\_ID | Value | Your ACCOUNT\_ID value. | | AGENT\_ID | Value | Your AGENT\_ID value. | | MATILLION\_REGION | Value | Your MATILLION\_REGION value. | | OAUTH\_CLIENT\_ID | ValueFrom | The Amazon resource number (ARN) of the AWS Secrets Manager secret appended with the key name that contains the {m_runner} client\_id in this format: `{your-arn}:{key}::`. For example: `arn:aws:secretsmanager:eu-west-1:112233445566:secret:my_secret_name-9wxygh:client_id::` To learn more about this process, read [Add {m_runner} credentials to AWS Secrets Manager](/docs/guides/add-runner-credentials-to-aws-secrets-manager). | | OAUTH\_CLIENT\_SECRET | ValueFrom | The Amazon resource number (ARN) of the AWS Secrets Manager secret appended with the key name that contains the {m_runner} client\_secret in this format: `{your-arn}:{key}::`. For example: `arn:aws:secretsmanager:eu-west-1:112233445566:secret:my_secret_name-9wxygh:client_secret::` | | EXTENSION\_LIBRARY\_LOCATION\* (optional) | Value | (Optional) The location of any additional Python libraries you want to use with the [Python Script](/docs/components/python-script) component, specified without the protocol. For example: `runner-python-libs` or `mybucket/runner-python-libs`. Currently, Amazon S3 and Azure Blob Storage are supported. | | EXTENSION\_LIBRARY\_PROTOCOL\* (optional) | Value | (Optional) Specified in conjunction with EXTENSION\_LIBRARY\_LOCATION, for example `S3`. | For a full description of optional environment variables, read [Optional {m_runner} parameters](/docs/guides/optional-runner-parameters). Click **Next**. *** ## Configure environment, storage, monitoring, and tags 1. Complete the following configuration: * **App environment:** Choose **AWS Fargate (serverless)**. * **CPU:** Choose **1 vCPU**. * **Memory:** Choose **4 GB**. * **Task role:** Use an IAM role that has similar permissions to your role. If you're using the same IAM role, you might need to add `ecs-tasks.amazonaws.com` to the **Service** section in the **Trust Relationship** section of your IAM role to allow the role to work with the ECS service. An example trusted entity can be found below. To learn more, read [AWS IAM Roles](/docs/guides/aws-iam-roles). * **Task execution role:** Choose `escTaskExecutionRole` unless instructed otherwise. Example trusted entity: ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "ecs-tasks.amazonaws.com", "ec2.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] } ``` 2. Scroll down to **Monitoring and logging - optional**. 3. The **awslogs-group** value should have automatically suggested a log group name. If you don't wish to use another log group, leave the **aws-create-group** entry and a log group will be created with the **awslogs-group** value. 4. If the naming of components has been consistent so far, the **awslogs-group** value should equal the log group that was previously created (note the use of the `/` preceding the ID in the original **Log group** name). Otherwise, select the log group created earlier. Delete the **aws-create-group** entry by clicking **Remove**. 5. Once again, add a tag where **Key** = `matillion-runner-id` and **Value** = ``. 6. Click **Next**. Review your settings and if you're happy, click **Create**. You will receive a blue banner at the top of the page explaining that AWS is currently creating the task definition. This banner will turn green upon completion. *** ## Service 1. In your newly created task definition, click the **Deploy** drop-down button and then click **Create service**. 2. In the **Choose a cluster** field, choose the cluster you created earlier. 3. In **Deployment configuration**, complete the following fields: * **Service name:** Add the following: `matillion-runner-` or another suitable descriptor. * **Desired tasks:** Set this count to your normal operational requirements. A minimum of two tasks are required. More tasks reduce the execution time of multi-threaded pipelines. If you use only one task, you run the risk of service outages during {m_runner} upgrades. 4. In **Networking**, under **Security group name**, locate the security group you created earlier and select it. Remove any other security groups if they aren't required. 5. Ensure that **Public IP** is toggled to **Turned on**, unless you are using Network Address Translation. 6. As in previous sections, you may want to add a tag where **Key** = `matillion-runner-id` and **Value** = ``. 7. Click **Create**. Once the deployment has finished, you should see your ECS service running inside your ECS cluster. You can then return to the {m_runner} dashboard and view the {m_runner}s list. Your new {m_runner} will be on the list and will have a **Running** status. You don't need to back up the AWS Fargate containers running your {m_runner}s. {m_runner}s are stateless, so no backup procedure is required.