Documentation Index
Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
Use this file to discover all available pages before exploring further.
The Table Output transformation component lets you write an input dataset out to an existing table in your cloud data warehouse. Table Output will be the last component in your pipeline, as you write the dataset out after all other processing is complete.
The target table must already exist, and should have suitable columns to map to the columns in your dataset. The component will validate if the input columns don’t map to suitable output columns (for example, if you map a date column to an integer column), but you may encounter unexpected results when the pipeline runs. You can choose whether the new data will be appended to the existing table (Append option), or whether the existing table rows will be deleted before the new data is written out (Truncate option).
This component is equivalent to using the SQL INSERT INTO statement, optionally preceded by a DELETE FROM statement.
There are other components that can write data to a table and may better suit your use case: Use case
The Table Output component supports two ways of writing data out to a table:
- Making incremental data loads to an existing table. For example, appending today’s transactions to a sales table.
- Overwriting existing data with fresh data. For example, replacing a staging table’s data with the latest extracted file.
Properties
Snowflake
Databricks
Amazon Redshift
A human-readable name for the component.
The Snowflake warehouse used to run the queries. The special value [Environment Default] uses the warehouse defined in the environment. Read Overview of Warehouses to learn more. The Snowflake schema. The special value [Environment Default] uses the schema defined in the environment. Read Database, Schema, and Share DDL to learn more. Select the output table. The available tables depend on the selected environment.
Select whether to cast types in the event of data type mismatches.
- Yes: If the source column type does not match the target table type, attempt to CAST the value to the required target type.
- No: Do not cast types (do not fix data type mismatches).
- Input Column: The source field from the input flow.
- Output Column: The target table’s output column to insert data into.
Click the Add all button to automatically populate the Output Column with all the columns in the table selected in the Target Table property. The Input column matches what is in the Output Column, unless there is no column in the source table that match that column name in output table, in which case it is blank. You can then selectively edit the list if required, but if you click Add all again it will return the list to the default fully populated state, removing any changes you have made.
- Column: Select the output column(s) to sort by.
- Sort Order: Set the corresponding column to be ordered ascending (default) or descending.
- Append: Adds the records to the end of the table.
- Truncate: Empty existing rows in the target table before loading the incoming input rows. This operation may use either a TRUNCATE or DELETE FROM query, depending on whether the current flow is in a transaction or not. TRUNCATE is faster, and therefore preferred, but is not transaction-safe and is not used if a transaction is in progress.
A human-readable name for the component.
Select a Databricks Unity Catalog. The special value [Environment Default] uses the catalog defined in the environment. Selecting a catalog will determine which databases are available in the next parameter. The Databricks schema. The special value [Environment Default] uses the schema defined in the environment. Read Create and manage schemas to learn more. Select the output table. The available tables depend on the selected environment.
Select whether to cast types in the event of data type mismatches.
- Yes: If the source column type does not match the target table type, attempt to CAST the value to the required target type.
- No: Do not cast types (do not fix data type mismatches).
- Input Column: The source field from the input flow.
- Output Column: The target table’s output column to insert data into.
Click the Add all button to automatically populate the Output Column with all the columns in the table selected in the Target Table property. The Input column matches what is in the Output Column, unless there is no column in the source table that matches that column name in the output table, in which case it is blank. You can then selectively edit the list if required, but if you click Add all again it will return the list to the default fully populated state, removing any changes you have made.
- Append: Adds the records to the end of the table.
- Truncate: Empty existing rows in the target table before loading the incoming input rows. This operation may use either a TRUNCATE or DELETE FROM query, depending on whether the current flow is in a transaction or not. TRUNCATE is faster, and therefore preferred, but is not transaction-safe and is not used if a transaction is in progress.
A human-readable name for the component.
Select the table schema. The special value [Environment Default] uses the schema defined in the environment. For more information on using multiple schemas, read Schemas. Select the output table. The available tables depend on the selected environment.
Select whether to cast types in the event of data type mismatches.
- Yes: If the source column type does not match the target table type, attempt to CAST the value to the required target type.
- No: Do not cast types (do not fix data type mismatches).
- Input Column: The source field from the input flow.
- Output Column: The target table’s output column to insert data into.
Click the Add all button to automatically populate the Output Column with all the columns in the table selected in the Target Table property. The Input column matches what is in the Output Column, unless there is no column in the source table that match that column name in output table, in which case it is blank. You can then selectively edit the list if required, but if you click Add all again it will return the list to the default fully populated state, removing any changes you have made.
- Append: Adds the records to the end of the table.
- Truncate: Empty existing rows in the target table before loading the incoming input rows. This operation may use either a TRUNCATE or DELETE FROM query, depending on whether the current flow is in a transaction or not. TRUNCATE is faster, and therefore preferred, but is not transaction-safe and is not used if a transaction is in progress.
