Skip to main content
This document provides an overview of Kubernetes deployment options for s. For cloud-specific architecture decisions and implementation details, see: Kubernetes deployment is one of several options for running s in your own infrastructure (Hybrid SaaS model). This approach offers:
  • Standardized deployment via Helm charts.
  • Native cloud authentication (AWS IRSA, Azure Managed Identity).
  • Horizontal pod autoscaling (available on EKS and AKS, not available in ECS/ACI alternatives).
  • Platform portability across AWS, Azure, and self-managed Kubernetes clusters.
This document is not a step-by-step guide or tutorial, but rather an overview of what a Kubernetes administrator needs to know to deploy and manage s. We assume you have a working knowledge of Kubernetes, and have existing Kubernetes infrastructure, before attempting this deployment.

​
When to choose Kubernetes

Choose Kubernetes for deployment when:
  • You have existing Kubernetes expertise and infrastructure.
  • You need autoscaling capabilities (EKS/AKS provide HPA support).
  • You want standardized deployment across multiple cloud providers.
  • You require integration with existing Kubernetes monitoring and security tooling.
Alternatives to consider:

​
Prerequisites

​
Required tools

​
Matillion account

Before deploying, create an agent in the console to obtain:
  • Account ID
  • Agent ID
  • OAuth Client ID and Secret
  • Region (us1 or eu1)
For details, read create an agent.

​
Cloud provider deployment guides

​
Amazon EKS

For AWS deployments, the EKS guide covers:
  • IAM Roles for Service Accounts (IRSA) for credential-free AWS access.
  • VPC architecture decisions (new vs. existing, public vs. private cluster).
  • Worker node sizing and autoscaling strategies.
  • CloudWatch monitoring integration.
  • EKS-specific cost optimization.
Read the EKS deployment guide for details.

​
Azure AKS

For Azure deployments, the AKS guide covers:
  • Workload Identity/Managed Identity for credential-free Azure access.
  • VNet architecture decisions (new vs. existing, public vs. private cluster).
  • Node pool sizing and autoscaling strategies.
  • Azure Monitor integration.
  • AKS-specific cost optimization.
Read the AKS deployment guide for details.

​
Self-managed Kubernetes

For self-managed Kubernetes clusters (customer-managed control plane in cloud or other environments) you need to consider: Container registry: Use AWS ECR Public (no authentication required).
  • public.ecr.aws/matillion/etl-agent:stable
  • public.ecr.aws/matillion/etl-agent:current
Authentication: OAuth credentials stored in Kubernetes Secrets. Deployment: Use the standard Helm chart from the Deployment Library. Other considerations:
  • Native cloud authentication (IRSA, Workload Identity) not available for self-managed clusters.
  • Use OAuth credentials for Matillion control plane authentication.
  • Ensure outbound HTTPS connectivity to Matillion regional endpoints.
  • Horizontal Pod Autoscaler supported (requires metrics server).
  • Cluster Autoscaler availability depends on your infrastructure provider.
For managed Kubernetes services on other cloud providers, contact Matillion for platform availability.

​
Container images

Agent images are available in cloud-specific container registries:
Cloud providerImage repositoryTags
AWS (EKS)public.ecr.aws/matillion/etl-agent:stable, :current
Azure (AKS)matillion.azurecr.io/cloud-agent:stable, :current
Self-managed Kubernetespublic.ecr.aws/matillion/etl-agent:stable, :current
Image Tag Strategy:
  • :stable - Slower release cycle, maximum stability.
  • :current - Faster release cycle, earlier access to new features.
Both tags are production-ready. Choose based on your preference for stability vs. early feature access. The tag you select here must match the version track configured for this agent in Agent Manager. Mismatching the two can result in unexpected version behavior.

​
Common Kubernetes deployment components

All Kubernetes deployments use these same core components.

​
Helm charts

Standard Helm chart deploys:
  • Agent pods (deployment with configurable replicas).
  • Kubernetes service (Prometheus metrics endpoint on port 8080).
  • ConfigMaps (agent configuration).
  • Secrets (OAuth credentials for Matillion control plane).
  • ServiceAccount (for cloud-native authentication where available).
Read the Helm charts documentation.

​
Prometheus metrics

Agents expose Prometheus-compatible metrics at /actuator/prometheus on port 8080:
  • app_version_info: Agent build version.
  • app_agent_status: Running status (1=running, 0=stopped).
  • app_active_task_count: Currently executing tasks.
  • app_active_request_count: Active HTTP requests.
  • app_open_sessions_count: Open data warehouse connections.
The Helm chart includes service discovery annotations for automatic Prometheus scraping.

​
Security standards

All Kubernetes deployments implement pod security best practices:
  • Run as non-root user (UID 65534).
  • Read-only root filesystem.
  • No privilege escalation.
  • All Linux capabilities dropped.
  • Seccomp profile applied (RuntimeDefault).

​
Pre-deployment validation

The deployment library includes automated validation scripts in agent/helm/checks/:
  • run-check.sh - orchestrator that:
    • Auto-discovers agent pods using Helm chart labels.
    • Performs cluster-level security scans.
    • Executes in-pod validation.
  • pre-deployment-check.sh - validator that checks:
    • Python 3 and Java runtime availability.
    • Filesystem permissions.
    • Environment variables.
    • Network connectivity to Matillion control plane.
    • Security agents that might interfere (Crowdstrike, Prisma Cloud).
Usage: 
./agent/helm/checks/run-check.sh --namespace matillion --release matillion-agent
The output uses color-coded results (PASS, WARN, FAIL), with exit code 0 for success and 1 for failure.

​
Next steps