Third-party environment roles and permissions

This article provides an outline of various roles, permissions, and authentication methods for managing resources and access control in cloud environments for use with . Always follow the principle of least privilege and assign only the minimum permissions required for the specific task.

AWS permissions

This section details permissions related to AWS services like CloudFormation, S3 buckets, IAM roles, ECS (Amazon Elastic Container Service), and deployment.

S3 permissions

Users must have appropriate permissions defined in their IAM policies to access S3 buckets from components.

AWS Permission	Description
s3:GetObject	Grants the ability to get objects from an S3 bucket.
s3:ListBucket	Permits listing objects within an S3 bucket.
s3:PutObject	Allows to upload (put) an object into an S3 bucket.
s3:DeleteObject	Grants permission to delete an object from an S3 bucket.
s3:GetBucketLocation	Permits to retrieve the location of an S3 bucket.
s3:ListAllMyBuckets	Allows listing buckets. Omitting this does not necessarily cause pipeline errors (although may cause validation errors), but it’s essential to verify the specific requirements for each use case.

Redshift permissions

This section outlines the steps to manage users, databases, and schemas within the Redshift cluster for . It covers user creation, modification of user access, database creation, modification of database ownership, and schema creation.

Action	Description	Syntax
User creation
Create User	Create a user with a password.	`CREATE USER new_user PASSWORD 'your_password';`
Modify User Access	Grant unrestricted syslog access to the specified user.	`ALTER USER new_user SYSLOG ACCESS UNRESTRICTED;`
Database and schema management
Create Database	Create a new database with the specified name.	`CREATE DATABASE new_database;`
Modify Database Ownership	Change the owner of the specified database to a new user.	`ALTER DATABASE new_database OWNER TO new_owner;`
Create Schema	Create a new schema within the specified database and assign ownership to a user.	`CREATE SCHEMA new_schema AUTHORIZATION new_owner;`

IAM roles

Users configuring their AWS resources for use with are likely to require permission to create IAM Roles.

AWS Permission	Description
iam:CreateRole	Allows the creation of IAM roles.
iam:AttachRolePolicy	Enables attaching policies to an IAM role.
iam:PassRole	Necessary if the CloudFormation stack involves passing an IAM role to an AWS service.

Maia runner deployment permissions

Users deploying a in will require the following permission types: Coarse-grained:

AmazonECS_FullAccess
CloudWatchFullAccess
SecretsManagerReadWrite
IAMFullAccess
AmazonS3FullAccess

Fine-grained:

AWS policy	Action
IAM	iam:AddRoleToInstanceProfile, iam:PutRolePolicy, iam:PassRole, iam:GetRolePolicy, iam:CreateInstanceProfile, iam:DeleteInstanceProfile, iam:RemoveRoleFromInstanceProfile, iam:DetachRolePolicy, iam:AttachRolePolicy, iam:DeleteRolePolicy, iam:GetRole, iam:CreateRole, iam:DeleteRole, iam:TagPolicy, iam:UntagRole, iam:TagRole, iam:UntagPolicy, iam:UpdateRole, iam:UntagInstanceProfile, iam:TagInstanceProfile.
EC2	ec2:ReplaceIamInstanceProfileAssociation, ec2:AuthorizeSecurityGroupEgress, ec2:AuthorizeSecurityGroupIngress, ec2:RevokeSecurityGroupEgress, ec2:RevokeSecurityGroupIngress, ec2:DescribeSecurityGroups, ec2:DescribeVpcs, ec2:DescribeSubnets, ec2:UpdateSecurityGroupRuleDescriptionsIngress, ec2:UpdateSecurityGroupRuleDescriptionsEgress, ec2:CreateTags, ec2:ModifySecurityGroupRules, ec2:DisassociateIamInstanceProfile, ec2:DeleteTags, ec2:CreateSecurityGroup, ec2:AssociateIamInstanceProfile, ec2:DeleteSecurityGroup.
Logging	logs:PutRetentionPolicy, logs:CreateLogStream, logs:TagLogGroup, logs:TagResource, logs:CreateLogGroup, logs:DeleteLogStream, logs:UntagLogGroup, logs:DeleteLogGroup, logs:UntagResource.
ECS	secretsmanager:DeleteSecret, secretsmanager:UpdateSecretVersionStage, secretsmanager:TagResource, secretsmanager:UntagResource, secretsmanager:CreateSecret, secretsmanager:UpdateSecret, ecs:UpdateCluster, ecs:DescribeServices, ecs:DescribeTasks, ecs:DescribeClusters, ecs:UpdateContainerInstancesState, ecs:StartTask, ecs:CreateCapacityProvider, ecs:UpdateService, ecs:RegisterTaskDefinition, ecs:StopTask, ecs:DeregisterContainerInstance, ecs:DeleteTaskDefinitions, ecs:TagResource, ecs:UpdateClusterSettings, ecs:CreateCluster, ecs:DeleteService, ecs:DeleteCluster, ecs:RegisterContainerInstance, ecs:DeleteCapacityProvider, ecs:DeregisterTaskDefinition, ecs:CreateService, ecs:RunTask, ecs:UntagResource, ecs:PutClusterCapacityProviders, ecs:UpdateCapacityProvider, ecs:UpdateContainerAgent.

You can refer to the IAM roles documentation for more detailed information on task roles and task execution roles.

AWS service-linked roles

AWSServiceRoleForECS: This role is required to allow the deployment of an ECS (Amazon Elastic Container Service) agent in AWS. The ECS agent is often used within ecosystem for managing containerized workloads and orchestrating tasks.

Other service permissions

Below are permissions generally required for specific AWS services:

AWS Permission	Description
`cloudformation:CreateStack`	Allows the creation of CloudFormation stacks.
`rds:DescribeDBInstances`	Required for the RDS Query component to display available RDS instances.

Azure permissions

The Azure Blob permissions section provides guidance on managing access to Azure Blob Storage, which is crucial for controlling data access and ensuring security within Azure environments. Here’s a brief overview of the key points covered in this section. Granular control with SAS or Azure Roles helps ensure data security.

Authorization methods

Shared Access Signature (SAS):

Grants temporary access to specific blobs or containers without exposing account keys.
Allows defining specific read, write, or list permissions with start and expiry times.

Azure roles:

Assign built-in or custom roles based on the Azure Role-Based Access Control (RBAC) system.
Grants access to specific containers, blobs, or entire storage accounts.

Azure blob storage

The below are recommended permissions for accessing Azure Blob Storage with .

Permission Type	Description
Shared Access Signature (SAS)
Blob Reader	Allows reading blob data.
Blob Contributor	Enables reading, writing, and deleting blob data.
Blob Container Reader	Grants read access to all blobs within a container.
Blob Container Contributor	Provides permissions to manage all blobs within a container.
Azure roles
Storage Blob Data Reader	Allows reading blob data.
Storage Blob Data Contributor	Enables reading, writing, and deleting blob data.
Storage Account Contributor	Gives access to manage all resources within a storage account, including blobs, containers, and account settings.
Azure blob integration permissions
Queue Storage Permissions	To interact with Azure Queue Storage, consider permissions like `queue:PeekMessages`, `queue:DequeueMessages`, and `queue:SendMessage` for secure message handling.
Data Lake Storage Permissions	To integrate with Azure Data Lake Storage, permissions such as `Data Lake Storage Blob Data Owner` and `Data Lake Storage Blob Data Contributor` may be relevant for data access and management.

Snowflake

Your Snowflake environment in uses your role name to identify you. This role name will have certain privileges associated with it in Snowflake that you can adjust accordingly. For ease of use and ensuring full functionality, we recommend granting all privileges to the role being used and tying that role only to schemas for use with .

Your role name must be enclosed by double quotes if it contains a space.

Roles and privileges

Action	Syntax
Role creation
Creating a Custom Role	`CREATE ROLE "Data_Productivity_Cloud_role";`
Or, if the role name has spaces	`CREATE ROLE "Data_Productivity_Cloud role";`
Granting privileges
Granting Usage on Warehouse	`GRANT USAGE ON WAREHOUSE <warehouse-name> TO ROLE <role-name>;`
Granting Usage on Database	`GRANT USAGE ON DATABASE <database-name> TO ROLE <role-name>;`
Granting All Privileges on Schema	`GRANT ALL ON SCHEMA <schema-name> TO ROLE <role-name>;`
Granting All Privileges on Tables in Schema	`GRANT ALL ON ALL TABLES IN SCHEMA <schema-name> TO ROLE <role-name>;`
Granting All Privileges on Future Tables in Schema	`GRANT ALL ON FUTURE TABLES IN SCHEMA <schema-name> TO ROLE <role-name>;`
Granting Specific Privilege on Future Tables	`GRANT <privilege> ON FUTURE TABLES IN SCHEMA <schema-name> TO ROLE <role-name>;`
Defining role hierarchy
Assigning Roles	`GRANT Data <role-name> TO USER <username>;`
Revoking privileges
Revoking Privileges	`REVOKE <privilege> ON ALL TABLES IN SCHEMA <schema-name> FROM ROLE <role-name>;`

Authentication methods

Users can authenticate using their Snowflake username and password.
Users can authenticate using a keypair generated for their Snowflake account.
Multi-factor authentication connections are not supported for projects. It’s advisable to use dedicated Snowflake Service Account Users for projects.

​AWS permissions

​S3 permissions

​Redshift permissions

​IAM roles

​Maia runner deployment permissions

​AWS service-linked roles

​Other service permissions

​Azure permissions

​Authorization methods

​Azure blob storage

​Snowflake

​Roles and privileges

​Authentication methods

AWS permissions

S3 permissions

Redshift permissions

IAM roles

Maia runner deployment permissions

AWS service-linked roles

Other service permissions

Azure permissions

Authorization methods

Azure blob storage

Snowflake

Roles and privileges

Authentication methods