> ## Documentation Index > Fetch the complete documentation index at: https://docs.maia.ai/llms.txt > Use this file to discover all available pages before exploring further. # Snowflake Load export const component_0 = "Snowflake Load" export const database_0 = "Snowflake" export const ComponentMetadata = ({warehouses, unsupportedWarehouses = [], componentType, connectionInputs, connectionOutputs}) => { const allWarehouses = [...warehouses.map(w => ({ name: w, supported: true })), ...unsupportedWarehouses.map(w => ({ name: w, supported: false }))]; return

Project Availability	{allWarehouses.map((w, i) => {w.name} {w.supported ? '✅' : '❌'} )}
Component Type	{componentType}
Connection Inputs	{connectionInputs}
Connection Outputs	{connectionOutputs}

; }; The {component_0} component supersedes the option to use the [Database Query](/docs/components/database-query) component to retrieve data from {database_0}, and provides an incremental load option that is unavailable when using Database Query. Existing pipelines that use the Database Query component to fetch data from {database_0} will continue to work as expected. The Snowflake Load orchestration component runs SQL queries on an accessible database, and copies the results to a table via storage. You can query cloud or on-premises databases, so long as they are network-accessible. You can stage data (load data into a table) with this component, to perform further processing and transformations on it. The target table should be considered temporary, because it will either be truncated or recreated each time the component runs. You do not need to set up a Create Table component before using this component. If the component requires access to a cloud provider (AWS, Azure, or Google Cloud), it will use the [cloud credentials](/docs/guides/cloud-credentials) associated with your environment to access resources. To stage data to Azure Blob Storage, the Azure credentials associated with your environment must be assigned the `Storage Blob Data Contributor` role. For more information, read [User assigned with the Storage Blob Data Contributor role](https://learn.microsoft.com/en-us/answers/questions/1155139/user-assigned-with-storage-blob-data-contributor-r). *** ## Properties Reference material is provided below for the Connect, Configure, Destination, and Advanced Settings properties. A human-readable name for the component. ### Connect Supports key-pair authentication. {/* */} Your Snowflake account username. {/* */} Select the secret definition that denotes the value of your Snowflake private key. Read [Using Snowflake key-pair authentication](/docs/administration/snowflake-key-pair-authentication) to learn how to store the Snowflake private key using a secret. {/* */} Select the secret definition that denotes the value of your Snowflake passphrase. If your private key is passphrase protected, you will also need to add a secret to store the passphrase. Read [Using Snowflake key-pair authentication](/docs/administration/snowflake-key-pair-authentication) to learn how to store the Snowflake private key using a secret. {/* */} Enter your Snowflake account name and region. This is the string of text between `https://` and `.snowflakecomputing.com` in the URL you use to log in to Snowflake. {/* */} * **Parameter:** A JDBC parameter supported by the database driver. The available parameters are explained in the data model. Manual setup is not usually required, since sensible defaults are assumed. * **Value:** A value for the given parameter. Click the **Text Mode** toggle at the bottom of the **Connection Options** dialog to open a multi-line editor that lets you add items in a single block. For more information, read [Text mode](/docs/guides/components-overview#text-mode). {/* */} ### Configure {/* */} * **Full Load:** Select this option to load your entire dataset. * **Incremental Load:** Select this option to only load new and updated records from your dataset. {/* */} * **Basic:** This mode will build a query for you using settings from the **Schema**, **Data Source**, **Data Selection**, **Data Source Filter**, **Combine Filters**, and **Limit** parameters. In most cases, this mode will be sufficient. * **Advanced:** This mode will require you to write an SQL-like query to call data from the service you're connecting to. The available fields and their descriptions are documented in the data model. **Advanced** mode is currently not supported when **Incremental Load** is selected. There are some special pseudo columns that can form part of a query filter, but are not returned as data. This is fully described in the data model. While the query is exposed in an SQL-like language, the exact semantics can be surprising, for example, filtering on a column can return more data than not filtering on it. This is an impossible scenario with regular SQL. {/* */} This is an SQL-like SELECT query. Treat collections as table names, and fields as columns. Only available in **Advanced** mode. For more information, read the Snowflake [SELECT](https://docs.snowflake.com/en/sql-reference/sql/select) documentation. {/* */} Select the Snowflake role for this connection. {/* */} Select the Snowflake warehouse for this connection. {/* */} Select the Snowflake database for this connection. {/* */} Select the database schema to load data from. {/* */} {/* */} Select a single data source to be extracted from the source system and loaded into a table in the destination. The source system defines the data sources available. Use multiple components to load multiple data sources. {/* */} Choose one or more columns to return from the query. The columns available are dependent upon the data source selected. Move columns left-to-right to include in the query. To use [grid variables](/docs/guides/grid-variables), select the **Use Grid Variable** checkbox at the bottom of the **Data Selection** dialog. {/* */} Define one or more filter conditions that each row of data must meet to be included in the load. * **Input Column:** Select an input column. The available input columns vary depending upon the data source. * **Qualifier:** * **Is:** Compares the column to the value using the comparator. * **Not:** Reverses the effect of the comparison, so "Equals" becomes "Not equals", "Less than" becomes "Greater than or equal to", etc. * **Comparator:** Choose a method of comparing the column to the value. Possible comparators include: "Equal to", "Greater than", "Less than", "Greater than or equal to", "Less than or equal to", "Like", "Null". Not all data sources support all comparators. * **Value:** The value to be compared. Click the **Text Mode** toggle at the bottom of the dialog to open a multi-line editor. For more information, read [Text mode](/docs/guides/components-overview#component-properties). To use [grid variables](/docs/guides/grid-variables), select the **Use Grid Variable** checkbox at the bottom of the **Data Source Filter** dialog. {/* */} {/* */} The data source filters you have defined can be combined using either **And** or **Or** logic. If **And**, then all filter conditions must be satisfied to load the data row. If **Or**, then only a single filter condition must be satisfied. The default is **And**. If you have only one filter, or no filters, this parameter is essentially ignored. {/* */} {/* */} Set a numeric value to limit the number of rows that are loaded. The default is an empty field, which will load all rows. {/* */} When **Incremental Load** is selected, select a datetime field from your dataset that is always updated when your data changes, such as `dateModified`. The connector will record the maximum value of this field each time you run this pipeline. On subsequent runs when **Incremental Load** is selected, only data with a higher value in this field will be loaded. If no rows in the source have a higher value than the stored high-water mark, the task will complete with a "task is skipped" message. This is expected behavior, indicating that no new or updated data was found since the last successful run. ### Destination Select the destination for your data. This is either in Snowflake as a table or as files in cloud storage. * **Snowflake:** Load your data into a table in Snowflake. The data must first be staged via Snowflake or a cloud storage solution. * **Cloud Storage:** Load your data directly into files in your preferred cloud storage location. The format of these files can differ between source systems and will not have a file extension so we suggest inspecting the output to determine the format of the data. When **Incremental Load** is selected, **Cloud Storage** is not supported as a destination. Only warehouse destinations (Snowflake or Google BigQuery) are available for incremental loads. The Snowflake warehouse used to run the queries. The special value `[Environment Default]` uses the warehouse defined in the environment. Read [Overview of Warehouses](https://docs.snowflake.com/en/user-guide/warehouses-overview.html) to learn more. {/* */} The Snowflake database to access. The special value `[Environment Default]` uses the database defined in the environment. Read [Databases, Tables and Views - Overview](https://docs.snowflake.com/en/guides-overview-db) to learn more. {/* */} The Snowflake schema. The special value `[Environment Default]` uses the schema defined in the environment. Read [Database, Schema, and Share DDL](https://docs.snowflake.com/en/sql-reference/ddl-database.html) to learn more. {/* */} The name of the table to be created in your Snowflake database. You can use a [Table Input](/docs/components/table-input) component in a transformation pipeline to access and transform this data after it has been loaded. {/* */} Define what happens if the table name already exists in the specified Snowflake database and schema. Only available when **Full Load** is selected. * **Replace:** If the specified table name already exists, that table will be destroyed and replaced by the table created during this pipeline run. * **Truncate and Insert:** Each time the pipeline runs, two operations are performed: first, the table is truncated, meaning all existing rows are deleted. Then, your new rows are inserted. The table itself is never destroyed and recreated. * **Fail if Exists:** If the specified table name already exists, this pipeline will fail to run. * **Append:** If the specified table name already exists, then the data is inserted without altering or deleting the existing data in the table. It's appended onto the end of the existing data in the table. If the specified table name doesn't exist, then the table will be created, and your data will be inserted into the table. {/* */} Choose one or more columns to be designated as the table's primary key. When using **Incremental Load**, if you select a primary key, the loaded data will be merged with your existing data. If you don't select a primary key, the loaded data will be appended to your existing data. {/* */} * **Yes:** Staged files will be destroyed after data is loaded. This is the default setting. * **No:** Staged files are retained in the staging area after data is loaded. {/* */} Select the stage access strategy. The strategies available depend on the cloud platform you select in **Stage Platform**. * **Credentials:** Connects to the external stage (AWS, Azure) using your configured [cloud provider credentials](/docs/guides/cloud-credentials). Not available for Google Cloud Storage. * **Storage Integration:** Use a Snowflake storage integration to grant access to Snowflake to read data from and write to a cloud storage location. This will reveal the **Storage Integration** property, through which you can select any of your existing Snowflake storage integrations. {/* */} Use the drop-down menu to choose where the data is staged before being loaded into your Snowflake table. * **Amazon S3:** Stage your data on an AWS S3 bucket. * **Snowflake:** Stage your data on a Snowflake internal stage. * **Azure Storage:** Stage your data in an Azure Blob Storage container. * **Google Cloud Storage:** Stage your data in a Google Cloud Storage bucket. {/* */} Select the Snowflake internal stage type. Use the Snowflake links provided to learn more about each type of stage. * **User:** Each Snowflake user has a [user stage](https://docs.snowflake.com/en/user-guide/data-load-local-file-system-create-stage#user-stages) allocated to them by default for file storage. You may find the user stage convenient if your files will only be accessed by a single user, but need to be copied into multiple tables. * **Named:** A [named stage](https://docs.snowflake.com/en/user-guide/data-load-local-file-system-create-stage#named-stages) provides high flexibility for data loading. Users with the appropriate privileges on the stage can load data into any table. Furthermore, because the stage is a database object, any security or access rules that apply to all objects will apply to the named stage. Named stages can be altered and dropped. User stages cannot. {/* */} Select your named stage. Read [Creating a named stage](https://docs.snowflake.com/en/user-guide/data-load-local-file-system-create-stage#creating-a-named-stage) to learn how to create a new named stage. There is a known issue where named stages that include special characters or spaces are not supported. * **Append Files in Folder:** Appends files to storage folder. This is the default setting. * **Overwrite Files in Folder:** Overwrite existing files with matching structure. {/* */} The folder path for the files to be written to. Note that this path follows, but does not include, the bucket or container name. {/* */} A string of characters that precedes the name of the written files. This can be useful for organizing database objects. {/* */} A cloud storage location to load your data into files for storage. Choose either Amazon S3, Azure Storage, or Google Cloud Storage. Select the destination for your data: * **Google BigQuery:** Load your data into a table in Google BigQuery. * **Cloud Storage:** Load your data directly into files in your preferred cloud storage location. The format of these files can differ between source systems and will not have a file extension. Check the output to determine the format of the data. Select the Google BigQuery project to load data into. The special value `[Environment Default]` uses the project defined in the environment. Select the Google BigQuery dataset to load data into. The special value `[Environment Default]` uses the dataset defined in the environment. The name of the table to be created in your Google BigQuery project. You can use a [Table Input](/docs/components/table-input) component in a transformation pipeline to access and transform this data after it has been loaded. Define what happens if the table name already exists in the specified destination. Only available when **Full Load** is selected. * **Replace:** If the specified table name already exists, that table will be destroyed and replaced by the table created during this pipeline run. * **Truncate and Insert:** Each time the pipeline runs, two operations are performed: first, the table is truncated, meaning all existing rows are deleted. Then, your new rows are inserted. The table itself is never destroyed and recreated. * **Fail if Exists:** If the specified table name already exists, this pipeline will fail to run. * **Append:** If the specified table name already exists, then the data is inserted without altering or deleting the existing data in the table. It's appended onto the end of the existing data in the table. If the specified table name doesn't exist, then the table will be created, and your data will be inserted into the table. Select one or more columns to be designated as the table's primary key. When using **Incremental Load**, if you select a primary key, the loaded data will be merged with your existing data. If you don't select a primary key, the loaded data will be appended to your existing data. * **Yes:** Staged files will be destroyed after data is loaded. This is the default setting. * **No:** Staged files are retained in the staging area after data is loaded. Select the time period to use to partition the data loaded into your table, for example **Day** or **Month**. Select the Google Cloud Storage (GCS) bucket linked to your Google Cloud Platform account. Select whether to overwrite files of the same name when this pipeline runs. Default is **Yes**. * **Append Files in Folder:** Appends files to storage folder. This is the default setting. * **Overwrite Files in Folder:** Overwrite existing files with matching structure. The folder path for the files to be written to. Note that this path follows, but does not include, the bucket or container name. A string of characters that precedes the name of the written files. This can be useful for organizing database objects. A cloud storage location to load your data into files for storage. Choose either Amazon S3, Azure Storage, or Google Cloud Storage. ### Advanced Settings Set the file format used to stage extracted records from this component before loading. The default is `CSV`. `CSV` is compact and widely compatible, while `JSON Lines` preserves the difference between NULL and empty-string values. {/* */} Choose how to change the column names in your data so that they conform to the destination warehouse's identifier rules: * `None`: (Default) Do not change the source column names. * `Alphanumeric (lower case)`: Replace non-alphanumeric characters with underscores, change the name to lowercase, and add an underscore at the start of the name if it begins with a digit. * `Alphanumeric (upper case)`: Replace non-alphanumeric characters with underscores, change the name to uppercase, and add an underscore at the start of the name if it begins with a digit. If two source column names sanitize to the same result, the load fails with an error rather than dropping data. {/* */} Converts common strings that represent null into a null value. This is case-sensitive and works with the following strings: "", "NULL", "NUL", "Null", "null". The default is **No**. Currently, this property is only applicable when using Snowflake as your destination. {/* */} When **Yes**, remove leading and trailing characters from a string column. The default is **No**.