> ## Documentation Index
> Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks

export const m_runner = "Maia runner";

export const maia = "Maia";

Before you run any pipelines in {maia}, you need a connection to a suitable cloud data platform account. This topic discusses the basics of connecting {maia} to **Databricks**.

***

## Full SaaS or Hybrid SaaS?

{maia} can be run in a [Full SaaS or Hybrid SaaS](/docs/guides/runner-overview#matillion-full-saas-vs-hybrid-saas) architecture.

* Databricks on AWS is compatible with both Full SaaS and Hybrid SaaS.
* Databricks on Azure is compatible with both Full SaaS and Hybrid SaaS.

***

## Compute types

{maia} supports many of the Databricks compute types. For more information, read [Compute](https://docs.databricks.com/en/compute/index.html) in the Databricks documentation.

***

## Authentication to Databricks

{maia} supports Personal Access Token (PAT) authentication as well as OAuth for service principals (OAuth M2M) when connecting to Databricks.

To use [Personal Access Token (PAT) authentication](https://docs.databricks.com/en/dev-tools/auth/pat.html) in pipeline [components](/docs/guides/components-overview), enter `token` as the username and the actual value of the token as the password.

To use [OAuth M2M authentication](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) in pipeline [components](/docs/guides/components-overview), select the OAuth Client Credentials option and enter the client ID and client secret. Note, if you are using a Hybrid SaaS project your {m_runner} version should be 10.1021.1 or greater.

***

## Catalog types

We recommend using [Unity Catalog enabled workspaces](https://docs.databricks.com/en/data-governance/unity-catalog/enable-workspaces.html). {maia} does support Hive catalogs, but many of its advanced features (such as Unity Catalog staging) and future features will be reliant on Unity Catalog workspaces.

***

## Feature support

Some features will only work with specific Databricks runtimes and configurations:

| Feature                       | Minimum Databricks runtime | Notes                                                                                                                                                       |
| ----------------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Unity Catalog Volumes staging | 13.4+                      |                                                                                                                                                             |
| Run Notebook                  | 10.4+                      | If you are using a serverless SQL or classic SQL compute, you can only run SQL notebooks using the [Run notebook](/docs/components/run-notebook) component. |

***

## S3 buckets and Azure Blob storage

If you wish to load data from, or stage via, S3 buckets or Azure Blob storage, you must create and associate AWS or Azure [cloud credentials](/docs/guides/cloud-credentials) to your environment.

You should also make sure that the instance profile attached to your Databricks compute resources also has access to the same AWS or Azure storage.