Documentation Index
Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
Use this file to discover all available pages before exploring further.
This guide lists all GCP IAM roles and permissions required to deploy and run a on Google Kubernetes Engine (GKE). It covers two distinct principals: the Terraform deployer (the human or CI identity running terraform apply) and the runtime service accounts, which are automatically provisioned by Terraform.
For the full deployment walkthrough, see the GKE deployment guide.
Required GCP APIs
Enable the following APIs in the target GCP project before running Terraform. Replace YOUR_PROJECT_ID with your actual GCP project ID.
gcloud services enable \
container.googleapis.com \
compute.googleapis.com \
secretmanager.googleapis.com \
storage.googleapis.com \
iam.googleapis.com \
cloudresourcemanager.googleapis.com \
artifactregistry.googleapis.com \
--project=YOUR_PROJECT_ID
| API | Purpose |
|---|
container.googleapis.com | GKE cluster and node pool management. |
compute.googleapis.com | VPC network, subnets, Cloud NAT, and firewall rules. |
secretmanager.googleapis.com | Secret Manager secrets for credentials. |
storage.googleapis.com | GCS bucket for staging storage. |
iam.googleapis.com | Service account creation and IAM binding (Workload Identity). |
cloudresourcemanager.googleapis.com | Project metadata read access—required by roles/browser for the GCP Project ID field in the UI. |
artifactregistry.googleapis.com | Pull container images from GCP Artifact Registry. |
The identity running terraform apply must have the following IAM roles on the target GCP project. These can be granted to a user account, a service account used by CI/CD, or a group.
| IAM role | Purpose | Scope |
|---|
roles/container.admin | Create and manage GKE clusters and node pools. | Project |
roles/compute.networkAdmin | Create VPC network, subnets, and Cloud NAT. | Project |
roles/iam.serviceAccountAdmin | Create the GKE node service account and the Workload Identity service account. | Project |
roles/iam.serviceAccountUser | Attach service accounts to GKE nodes. | Project |
roles/iam.securityAdmin | Grant IAM bindings to service accounts—covers both project-level (google_project_iam_member) and SA-level (Workload Identity binding) policies. | Project |
roles/storage.admin | Create the staging GCS bucket and set bucket-level IAM policies. | Project |
roles/secretmanager.admin | Create the Secret Manager secret and set secret-level IAM policies. | Project |
roles/artifactregistry.admin | Create and manage Artifact Registry repositories (required if creating the repository via Terraform). | Project |
roles/logging.admin | Create log buckets and sinks (required if managing Cloud Logging resources via Terraform). | Project |
roles/owner or roles/editor + roles/iam.securityAdmin will also satisfy all requirements. The granular roles above are recommended for production environments, following the principle of least privilege.
Maia runner runtime permissions
Terraform automatically creates and configures two service accounts. No manual IAM grants are required for these.
Maia runner Workload Identity service account
This service account is annotated onto the Kubernetes service account via Workload Identity, so pods authenticate to GCP APIs without any key files. The following roles are granted automatically by Terraform.
| IAM role | Scope | Why it’s needed |
|---|
roles/iam.workloadIdentityUser | GCP service account | Binds the Kubernetes service account to this GCP service account via Workload Identity Federation—enables secretless pod authentication. |
roles/storage.objectAdmin | Staging GCS bucket | Read, write, and delete objects in the ‘s own staging bucket. |
roles/secretmanager.secretAccessor | Project-wide | Read secret values from Secret Manager—required for the to retrieve credentials and for the Matillion UI to read secrets. |
roles/secretmanager.viewer | Project-wide | List and get secret metadata—required for the Matillion UI list-secrets functionality. |
roles/browser | Project-wide | Read GCP project metadata—required for the GCP Project ID field when adding secrets in the Matillion UI. |
roles/secretmanager.secretVersionManager | Secret Manager secret | Manage versions of the ‘s own Secret Manager secret. |
roles/storage.legacyBucketReader + roles/storage.objectViewer | Each additional GCS bucket (optional) | Bucket metadata and object read access—required to download file contents from user-supplied buckets (custom CA certificates, Python extension libraries, external JDBC drivers). |
roles/cloudkms.cryptoKeyDecrypter (optional) | Cloud KMS key(s) | Decrypt CMEK-encrypted secrets—only required if you use customer-managed encryption keys (CMEK) in Secret Manager. |
The optional extra GCS bucket permissions (roles/storage.legacyBucketReader and roles/storage.objectViewer) are only granted when bucket names are supplied via the runner_gcs_buckets Terraform variable. Typical use cases include custom CA certificate bundles, Python extension libraries, and external JDBC drivers.
GKE node service account
A separate, least-privilege service account is attached to the GKE nodes for cluster operations. The following roles are granted automatically by Terraform.
| IAM role | Scope | Why it’s needed |
|---|
roles/logging.logWriter | Project | Write container and node logs to Cloud Logging. |
roles/monitoring.metricWriter | Project | Write node and pod metrics to Cloud Monitoring. |
roles/monitoring.viewer | Project | Read monitoring data for node health checks and autoscaling decisions. |
Pipeline-level permissions
The following permissions are required at pipeline execution time, depending on the data warehouse the connects to. These aren’t provisioned by the GKE deployment Terraform—they must be granted separately to the Workload Identity service account, or to a delegated service account.
BigQuery
| IAM role | Scope | Why it’s needed |
|---|
roles/bigquery.jobUser | Project (required) | Submit and run BigQuery jobs—must be project-level; can’t be scoped to a dataset. |
roles/bigquery.dataViewer | Dataset or project | Read data from BigQuery datasets and tables. |
roles/bigquery.dataEditor | Dataset or project | Read and write data—only required if the pipeline writes back to BigQuery. |
roles/bigquery.readSessionUser | Project | Use the BigQuery Storage Read API for efficient large reads—recommended for production workloads. |
Dataset-level grants are preferred for roles/bigquery.dataViewer and roles/bigquery.dataEditor, to follow the principle of least privilege.
Snowflake via GCS staging
Snowflake access isn’t governed by GCP IAM—the authenticates to Snowflake using credentials stored in Secret Manager (key-pair, OAuth, or username/password). However, if Snowflake uses a GCS Storage Integration for staging, the following additional permission is needed on the Snowflake staging bucket:
| IAM role | Scope | Why it’s needed |
|---|
roles/storage.objectAdmin | Snowflake staging GCS bucket | Allows the Snowflake GCS integration service account to read and write stage files. |
Workload Identity binding
Workload Identity allows Kubernetes pods to impersonate a GCP service account without storing any key files in the cluster. Terraform configures this binding automatically. It takes the form:
serviceAccount:<PROJECT_ID>.svc.id.goog[<K8S_NAMESPACE>/<K8S_SERVICE_ACCOUNT_NAME>]
→ roles/iam.workloadIdentityUser on <RUNNER_GCP_SA>
K8S_NAMESPACE defaults to var.name. Override with the k8s_namespace Terraform variable if your Helm release is deployed into a different namespace.K8S_SERVICE_ACCOUNT_NAME defaults to <name>-sa. Override with k8s_service_account_name if your Helm release name differs from var.name.
If the Kubernetes namespace or service account name doesn’t match the Workload Identity binding, pods will silently fail to authenticate to GCP APIs. Run terraform apply a second time if you see an identity pool race condition error after initial cluster creation.
Quick reference
The following table summarizes all permissions.
| Principal | IAM role | Scope | Provisioned by |
|---|
| GKE node SA | roles/logging.logWriter | Project | Terraform |
| GKE node SA | roles/monitoring.metricWriter | Project | Terraform |
| GKE node SA | roles/monitoring.viewer | Project | Terraform |
| Workload Identity SA | roles/bigquery.dataViewer / roles/bigquery.dataEditor | Dataset or project | Manual (BigQuery pipelines) |
| Workload Identity SA | roles/bigquery.jobUser | Project | Manual (BigQuery pipelines) |
| Workload Identity SA | roles/browser | Project-wide | Terraform |
| Workload Identity SA | roles/cloudkms.cryptoKeyDecrypter | Cloud KMS key(s) | Manual (if CMEK used) |
| Workload Identity SA | roles/iam.workloadIdentityUser | GCP service account | Terraform |
| Workload Identity SA | roles/secretmanager.secretAccessor | Project-wide | Terraform |
| Workload Identity SA | roles/secretmanager.secretVersionManager | secret | Terraform |
| Workload Identity SA | roles/secretmanager.viewer | Project-wide | Terraform |
| Workload Identity SA | roles/storage.legacyBucketReader + roles/storage.objectViewer | Each additional GCS bucket | Terraform (optional) |
| Workload Identity SA | roles/storage.objectAdmin | Staging GCS bucket | Terraform |
| Terraform deployer | roles/artifactregistry.admin | Project | Manual |
| Terraform deployer | roles/compute.networkAdmin | Project | Manual |
| Terraform deployer | roles/container.admin | Project | Manual |
| Terraform deployer | roles/iam.securityAdmin | Project | Manual |
| Terraform deployer | roles/iam.serviceAccountAdmin | Project | Manual |
| Terraform deployer | roles/iam.serviceAccountUser | Project | Manual |
| Terraform deployer | roles/logging.admin | Project | Manual |
| Terraform deployer | roles/secretmanager.admin | Project | Manual |
| Terraform deployer | roles/storage.admin | Project | Manual |
