Skip to main content
For Snowflake projects, the Google Sheets connector supersedes the Google Sheets Query connector, which is no longer available for new pipelines. Existing pipelines that use the Google Sheets Query connector will continue to work as expected.Databricks and Amazon Redshift projects should continue to use the Google Sheets Query connector.
The Load component lets you connect to your account and configure a query to load data from the API. You can select the data source, choose the fields to include in your query, and apply filters to refine your results. You can also specify a row limit to control the number of records returned. The results of your query can be loaded into your destination as a new table or can replace an existing table. You can also configure incremental load settings to load only new or updated records since the last load. Your destination can be your connected cloud data warehouse or a cloud storage location. You can also configure advanced settings to control the output encoding, key sanitization strategy, and other options. If the component requires access to a cloud provider (AWS, Azure, or Google Cloud), it will use the cloud credentials associated with your environment to access resources. To stage data to Azure Blob Storage, the Azure credentials associated with your environment must be assigned the Storage Blob Data Contributor role. For more information, read User assigned with the Storage Blob Data Contributor role.

​
Properties

Reference material is provided below for the Connect, Configure, Destination, and Advanced Settings properties.
​
string
required
A human-readable name for the component.

​
Connect

​
drop-down
required
Select how to authenticate this component:
  • Select Service account to use a JSON structured service account key.
  • Select OAuth 2.0 Authorization Code to use an OAuth connection.
Choose your OAuth connection from the drop-down menu. Click Manage to navigate to the OAuth connections list to review OAuth connections and to add new connections. Read OAuth to learn how to create an OAuth connection.
​
drop-down
required
Choose the secret definition that represents your credentials for this connector.If you have not already saved your credentials for this connector as a secret definition, click Add secret to create a secret definition representing these credentials. Read Secrets and secret definitions for details about creating a secret definition.
​
column editor
  • Parameter: A JDBC parameter supported by the database driver. The available parameters are explained in the data model. Manual setup is not usually required, since sensible defaults are assumed.
  • Value: A value for the given parameter.
Click the Text Mode toggle at the bottom of the Connection Options dialog to open a multi-line editor that lets you add items in a single block. For more information, read Text mode.To use grid variables, select the Use Grid Variable checkbox at the bottom of the Connection Options dialog.

​
Configure

​
string
required
Enter the name of the spreadsheet. This must be accessible to the account used to authenticate (but that account doesn’t need to own the spreadsheet).
The spreadsheet ID is the long string of characters in the middle of the spreadsheet URL. For example, if the spreadsheet URL is docs.google.com/spreadsheets/d/1aBCD2efg3HiJk4lMNopQ5rs6Tu7-vwxyZ8/edit then the spreadsheet ID is 1aBCD2efg3HiJk4lMNopQ5rs6Tu7-vwxyZ8.
​
boolean
required
  • Yes: The first row of data (row 1) is used to derive column names. Spaces and special characters are removed. This is the default setting.
  • No: Data is returned using columns A, B, C, and so on. All rows are treated as data.
​
drop-down
required
  • Basic: Configure data selection using the Data Source, Data Selection, Data Source Filter, Combine Filters, and Row Limit properties.
  • Advanced: Write your own SQL query using the SQL Query property.
​
string
Optionally specify a cell range. For example, A1:C500. If not specified, the entire sheet is read. If specified, then you must set Contains Header Row to No. The specified cell range should only include data and not column headings.The wildcard character * is supported, for example A5:E* would consider columns A-E and rows 5 onwards.
​
code editor
required
Enter an SQL-like SELECT query to retrieve the data you want. This query must be written in the SQL accepted by your cloud data warehouse. Treat collections as table names, and fields as columns. Only available in Advanced mode.For more information, read the SELECT query documentation for your cloud data warehouse:
​
drop-down
required
Select the data source to query from. Available when Mode is set to Basic.
​
object selector
required
Select the columns to fetch from the selected data source. Only available in Basic mode.To use grid variables, toggle Use Grid Variable on. For more information, read Grid variables.
​
grid
Filter the rows returned from the data source. Available when Mode is set to Basic.
​
drop-down
required
Choose how to combine multiple filters. Select AND to return rows matching all filters, or OR to return rows matching any filter. Available when Mode is set to Basic.
​
integer
The maximum number of rows to return from the data source. Leave blank to return all rows. Available when Mode is set to Basic.

​
Destination

Select your cloud data warehouse.
​
drop-down
required
Select the destination for your data. This is either in Snowflake as a table or as files in cloud storage.
  • Snowflake: Load your data into a table in Snowflake. The data must first be staged via Snowflake or a cloud storage solution.
  • Cloud Storage: Load your data directly into files in your preferred cloud storage location. The format of these files can differ between source systems and will not have a file extension so we suggest inspecting the output to determine the format of the data.
​
drop-down
required
The Snowflake warehouse used to run the queries. The special value [Environment Default] uses the warehouse defined in the environment. Read Overview of Warehouses to learn more.
​
drop-down
required
The Snowflake database to access. The special value [Environment Default] uses the database defined in the environment. Read Databases, Tables and Views - Overview to learn more.
​
drop-down
required
The Snowflake schema. The special value [Environment Default] uses the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
​
string
required
The name of the table to be created in your Snowflake database. You can use a Table Input component in a transformation pipeline to access and transform this data after it has been loaded.
​
drop-down
required
Define what happens if the table name already exists in the specified Snowflake database and schema.
  • Replace: If the specified table name already exists, that table will be destroyed and replaced by the table created during this pipeline run.
  • Truncate and Insert: Each time the pipeline runs, two operations are performed: first, the table is truncated, meaning all existing rows are deleted. Then, your new rows are inserted. The table itself is never destroyed and recreated.
  • Fail if Exists: If the specified table name already exists, this pipeline will fail to run.
  • Append: If the specified table name already exists, then the data is inserted without altering or deleting the existing data in the table. It’s appended onto the end of the existing data in the table. If the specified table name doesn’t exist, then the table will be created, and your data will be inserted into the table.
​
object selector
Select one or more columns to use as the table’s primary key. If you select multiple columns, the component creates a composite primary key using the combination of the selected columns.Leave this blank to create the table without a primary key.
​
boolean
required
  • Yes: Staged files will be destroyed after data is loaded. This is the default setting.
  • No: Staged files are retained in the staging area after data is loaded.
​
drop-down
Select the stage access strategy. The strategies available depend on the cloud platform you select in Stage Platform.
  • Credentials: Connects to the external stage (AWS, Azure) using your configured cloud provider credentials. Not available for Google Cloud Storage.
  • Storage Integration: Use a Snowflake storage integration to grant access to Snowflake to read data from and write to a cloud storage location. This will reveal the Storage Integration property, through which you can select any of your existing Snowflake storage integrations.
​
drop-down
required
Use the drop-down menu to choose where the data is staged before being loaded into your Snowflake table.
  • Amazon S3: Stage your data on an AWS S3 bucket.
  • Snowflake: Stage your data on a Snowflake internal stage.
  • Azure Storage: Stage your data in an Azure Blob Storage container.
  • Google Cloud Storage: Stage your data in a Google Cloud Storage bucket.
​
drop-down
required
Select the Snowflake internal stage type. Use the Snowflake links provided to learn more about each type of stage.
  • User: Each Snowflake user has a user stage allocated to them by default for file storage. You may find the user stage convenient if your files will only be accessed by a single user, but need to be copied into multiple tables.
  • Named: A named stage provides high flexibility for data loading. Users with the appropriate privileges on the stage can load data into any table. Furthermore, because the stage is a database object, any security or access rules that apply to all objects will apply to the named stage.
Named stages can be altered and dropped. User stages cannot.
​
drop-down
required
Select your named stage. Read Creating a named stage to learn how to create a new named stage.
You can’t select a named stage if its name includes special characters or spaces.

​
Advanced Settings

​
drop-down
Set the file format used to stage extracted records from this component before loading. The default is CSV. CSV is compact and widely compatible, while JSON Lines preserves the difference between NULL and empty-string values.
​
drop-down
Choose how to change the column names in your data so that they conform to the destination warehouse’s identifier rules:
  • None: (Default) Do not change the source column names.
  • Alphanumeric (lower case): Replace non-alphanumeric characters with underscores, change the name to lowercase, and add an underscore at the start of the name if it begins with a digit.
  • Alphanumeric (upper case): Replace non-alphanumeric characters with underscores, change the name to uppercase, and add an underscore at the start of the name if it begins with a digit.
If two source column names sanitize to the same result, the load fails with an error rather than dropping data.
​
drop-down
required
Select whether to automatically log debug information about your load. You can find these logs in the task history; include them in support requests concerning the component. Turning this on overrides any debugging connection options.
​
drop-down
required
The verbosity level for logging debug information. Levels above 1 can log large amounts of data and result in slower execution. You can find these logs in the Message field of the task details after pipeline execution; include them in support requests concerning the component.
  1. Logs the query, the number of rows returned by it, the start of execution and the time taken, and any errors.
  2. Logs everything in level 1, plus cache queries and additional information about the request, if applicable.
  3. Also logs the body of the request and the response.
  4. Also logs transport-level communication with the data source, including SSL negotiation.
  5. Also logs communication with the data source and additional troubleshooting details, including interface commands.
​
boolean
required
Converts common strings that represent null into a null value. This is case-sensitive and works with the following strings: "", “NULL”, “NUL”, “Null”, “null”. The default is No.
This parameter is only applicable when using Snowflake as your destination.
​
boolean
required
When Yes, remove leading and trailing characters from a string column. The default is No.
This parameter is only applicable when using Snowflake as your destination.

​
Data model

The JDBC driver for this component models Google Sheets APIs as relational tables, views, and stored procedures, which are documented in the data model. You’ll also find API limitations and requirements. This connector also allows you to query system tables in Advanced mode. To see the available system tables in the data model, read the System Tables section of the data model. For more information about using system tables, read our System tables guide.