Skip to main content
For Snowflake projects, the Excel Load connector supersedes the Excel Query connector, which is no longer available for new pipelines. Existing pipelines that use the Excel Query connector will continue to work as expected.Databricks and Amazon Redshift projects should continue to use the Excel Query connector.
Excel Load connector loads data stored in an Office Open XML Excel sheet into a table. Each time the Excel Load connector runs, the target table is recreated, dropping any existing table of the same name. You do not need to use the Create Table component when using this component. By default, data types are guessed by looking at the cell formatting, not cell contents. This is controlled using the Connection Option “type detection scheme”, which can be set to ColumnFormat (the default, which examines the cell formatting), RowScan (which will scan 15 rows of data and guess the data type based on the data values), or None (treat everything as text). None is often a sensible choice if you intend to parse the values later, or the types in a single column are mixed. The Connection Option “row scan depth” controls how many rows to scan when RowScan is selected. If the component requires access to a cloud provider (AWS, Azure, or Google Cloud), it will use credentials as follows:
  • If using Matillion Full SaaS: The component will use the cloud credentials associated with your environment to access resources.
  • If using Hybrid SaaS: By default the component will inherit the agent’s execution role (service account role). However, if there are cloud credentials associated with your environment, these will overwrite the role.
You need valid AWS or Azure credentials to access the data. The bucket may be defined as public, but is secured by the credentials.
This component is potentially destructive. If the target table undergoes a change in structure, it will be recreated. Otherwise, the target table is truncated. Setting the load option Recreate Target Table to Off will prevent both recreation and truncation. Do not modify the target table structure manually.

​
Properties

Reference material is provided below for the Connect, Configure, Destination, and Advanced Settings properties.
​
string
required
A human-readable name for the component.

​
Connect

​
drop-down
required
Select the cloud storage location that hosts your Excel file. Currently, the following storage types are supported:
  • Amazon S3 Storage
  • Azure Blob Storage
  • Google Cloud Storage
​
string
required
The URL to the location of your .xlsx source file. Only Office Open XML (.xlsx) files are supported. Storage containers are explorable here if your credentials include access to resources of the selected Storage Type.Clicking this property will open the Storage URL dialog. This displays a list of all existing storage accounts. Select a storage account, then a container, and then a subfolder if required. This constructs a URL with the following format:
You can also type the URL directly into the Storage Accounts path field, instead of selecting listed elements. This is particularly useful when using variables in the URL, for example:
Special characters used in this field must be URL-safe.
​
column editor
Connection Options = column editor
  • Parameter: A JDBC parameter supported by the database driver. The available parameters are explained in the data model. Manual setup is not usually required, since sensible defaults are assumed.
  • Value: A value for the given parameter.
Click the Text Mode toggle at the bottom of the Connection Options dialog to open a multi-line editor that lets you add items in a single block. For more information, read Text mode.To use grid variables, select the Use Grid Variable checkbox at the bottom of the Connection Options dialog.

​
Configure

​
drop-down
required
  • Full Load: Select this option to load your entire dataset.
  • Incremental Load: Select this option to only load new and updated records from your dataset.
​
drop-down
required
  • Basic: This mode will build a query for you using settings from the Schema, Data Source, Data Selection, Data Source Filter, Combine Filters, and Row Limit parameters. In most cases, this mode will be sufficient.
  • Advanced: This mode will require you to write an SQL-like query to call data from the service you’re connecting to. The available fields and their descriptions are documented in the data model.
    Advanced mode is currently not supported when Incremental Load is selected.
There are some special pseudo columns that can form part of a query filter, but are not returned as data. This is fully described in the data model.
While the query is exposed in an SQL-like language, the exact semantics can be surprising, for example, filtering on a column can return more data than not filtering on it. This is an impossible scenario with regular SQL.
​
code editor
required
Enter an SQL-like SELECT query to retrieve the data you want. This query must be written in the SQL accepted by your cloud data warehouse. Treat collections as table names, and fields as columns. Only available in Advanced mode.For more information, read the SELECT query documentation for your cloud data warehouse:
​
boolean
  • Yes: The first row of data is the column names.
  • No: The first row of data is just data. Columns will be named A, B, C…
​
string
By default the whole worksheet is considered. However, you may optionally specify a range of cells instead. For example, A5:E100 would only consider rows 5-100 in columns A-E. Wildcards () are also supported, for example A5:E would consider columns A-E and rows 5 onwards.
​
drop-down
required
Select a single data source to be extracted from the source system and loaded into a table in the destination. The source system defines the data sources available. Use multiple components to load multiple data sources.
​
object selector
required
Select the columns to fetch from the selected data source. Only available in Basic mode.To use grid variables, toggle Use Grid Variable on. For more information, read Grid variables.
​
column editor
Define one or more filter conditions that each row of data must meet to be included in the load.
  • Input Column: Select an input column. The available input columns vary depending upon the data source.
  • Qualifier:
    • Is: Compares the column to the value using the comparator.
    • Not: Reverses the effect of the comparison, so “Equals” becomes “Not equals”, “Less than” becomes “Greater than or equal to”, etc.
  • Comparator: Choose a method of comparing the column to the value. Possible comparators include: “Equal to”, “Greater than”, “Less than”, “Greater than or equal to”, “Less than or equal to”, “Like”, “Null”. Not all data sources support all comparators.
  • Value: The value to be compared.
Toggle Text mode on at the bottom of the dialog to open a multi-line editor that lets you add items in a single block. For more information, read Text mode.To use grid variables, toggle Use Grid Variable on. For more information, read Grid variables.
​
drop-down
The data source filters you have defined can be combined using either And or Or logic. If And, then all filter conditions must be satisfied to load the data row. If Or, then only a single filter condition must be satisfied. The default is And.If you have only one filter, or no filters, this parameter is essentially ignored.
​
integer
Set a numeric value to limit the number of rows that are loaded. The default is an empty field, which will load all rows.
​
boolean
When enabled, Excel data is cached to a temporary directory on disk rather than being held entirely in memory. This significantly reduces memory usage and is recommended when processing large Excel files. The cache is automatically cleared once the query completes.

​
Destination

​
drop-down
required
Select the destination for your data. This is either in Snowflake as a table or as files in cloud storage.
  • Snowflake: Load your data into a table in Snowflake. The data must first be staged via Snowflake or a cloud storage solution.
  • Cloud Storage: Load your data directly into files in your preferred cloud storage location. The format of these files can differ between source systems and will not have a file extension so we suggest inspecting the output to determine the format of the data.
When Incremental Load is selected, Cloud Storage is not supported as a destination. Only warehouse destinations (Snowflake or Google BigQuery) are available for incremental loads.
​
drop-down
required
The Snowflake warehouse used to run the queries. The special value [Environment Default] uses the warehouse defined in the environment. Read Overview of Warehouses to learn more.
​
drop-down
required
The Snowflake database to access. The special value [Environment Default] uses the database defined in the environment. Read Databases, Tables and Views - Overview to learn more.
​
drop-down
required
The Snowflake schema. The special value [Environment Default] uses the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
​
string
required
The name of the table to be created in your Snowflake database. You can use a Table Input component in a transformation pipeline to access and transform this data after it has been loaded.
​
drop-down
required
Define what happens if the table name already exists in the specified destination. Only available when Full Load is selected.
  • Replace: If the specified table name already exists, that table will be destroyed and replaced by the table created during this pipeline run.
  • Truncate and Insert: Each time the pipeline runs, two operations are performed: first, the table is truncated, meaning all existing rows are deleted. Then, your new rows are inserted. The table itself is never destroyed and recreated.
  • Fail if Exists: If the specified table name already exists, this pipeline will fail to run.
  • Append: If the specified table name already exists, then the data is inserted without altering or deleting the existing data in the table. It’s appended onto the end of the existing data in the table. If the specified table name doesn’t exist, then the table will be created, and your data will be inserted into the table.
​
object selector
Select one or more columns to use as the table’s primary key. If you select multiple columns, the component creates a composite primary key using the combination of the selected columns.Leave this blank to create the table without a primary key.When using Incremental Load, if you select a primary key, the loaded data will be merged with your existing data. If you don’t select a primary key, the loaded data will be appended to your existing data.
​
boolean
required
  • Yes: Staged files will be destroyed after data is loaded. This is the default setting.
  • No: Staged files are retained in the staging area after data is loaded.
​
drop-down
required
Use the drop-down menu to choose where the data is staged before being loaded into your Snowflake table.
  • Amazon S3: Stage your data on an AWS S3 bucket.
  • Snowflake: Stage your data on a Snowflake internal stage.
  • Azure Storage: Stage your data in an Azure Blob Storage container.
  • Google Cloud Storage: Stage your data in a Google Cloud Storage bucket.
​
drop-down
required
Select the Snowflake internal stage type. Use the Snowflake links provided to learn more about each type of stage.
  • User: Each Snowflake user has a user stage allocated to them by default for file storage. You may find the user stage convenient if your files will only be accessed by a single user, but need to be copied into multiple tables.
  • Named: A named stage provides high flexibility for data loading. Users with the appropriate privileges on the stage can load data into any table. Furthermore, because the stage is a database object, any security or access rules that apply to all objects will apply to the named stage.
Named stages can be altered and dropped. User stages cannot.
​
drop-down
required
Select your named stage. Read Creating a named stage to learn how to create a new named stage.
You can’t select a named stage if its name includes special characters or spaces.

​
Advanced Settings

​
boolean
required
Choose whether to automatically log debug information about your load. These logs can be found in the task history and should be included in support requests concerning the component. This property is set to No by default. Turning this on will override any debugging Connection Options you may have set.
​
drop-down
required
The level of detail you want to include in your debug logs. Select a level between 1 and 4:
  1. Will log the query, the number of rows returned by it, the start of execution, the time taken, and any errors.
  2. Will log everything included in Level 1, plus cache queries and additional information about the request, if applicable.
  3. Will log everything included in Levels 1 and 2, and additionally log the body of the request and the response. This is the default logging level when debug logging is activated.
  4. Will log everything included in Levels 1, 2, and 3, and additionally log transport-level communication with the data source. This includes SSL negotiation.
Levels above 1 can log huge amounts of data and result in slower query execution.

​
Data model

The JDBC driver for this component models Excel APIs as relational databases and stored procedures, which are documented in the data model. You’ll also find API limitations and requirements. This connector also allows you to query system tables in Advanced mode. To see the available system tables in the data model, read the System Tables section of the data model. For more information about using system tables, read our System tables guide.