Skip to main content
The component supersedes the option to use the Database Query component to retrieve data from , and provides an incremental load option that is unavailable when using Database Query.Existing pipelines that use the Database Query component to fetch data from will continue to work as expected.
The IBM Db2 for i Load connector uses the Connect and Configure parameters to create a table of IBM Db2 for i data, which is then stored in your cloud data warehouse. You do not need to use the Create Table component when using this connector, as the IBM Db2 for i Load component will create a new table or replace an existing table for you using the Destination parameters you define. If you use a Full SaaS setup, the IBM Db2 for i Load connector supports IBM i version 7.1 and above. If you use a Hybrid SaaS setup, you can upload an external driver—this may change the supported IBM i versions. If the component requires access to a cloud provider (AWS, Azure, or Google Cloud), it will use the cloud credentials associated with your environment to access resources. To stage data to Azure Blob Storage, the Azure credentials associated with your environment must be assigned the Storage Blob Data Contributor role. For more information, read User assigned with the Storage Blob Data Contributor role.

​
Properties

Reference material is provided below for the Connect, Configure, Destination, and Advanced Settings properties.
​
Name
string
required
A human-readable name for the component.

​
Connect

​
Authentication Method
drop-down
required
Currently supports Username & Password.
​
Username
string
required
Your IBM Db2 for i username to use for authentication.
​
Password
drop-down
required
Select the secret definition that represents your IBM Db2 for i password.Choose the secret definition that represents your credentials for this connector.If you have not already saved your credentials for this connector as a secret definition, click Add secret to create a secret definition representing these credentials. Read Secrets and secret definitions for details about creating a secret definition.
​
Connection URL
string
required
The URL for your chosen IBM Db2 for i database. The general pattern of the URL is jdbc:as400://<host>/<database>. Make appropriate substitutions for the <host> and <database> parameters in this URL.Although many parameters and options can be added to the end of the URL, it is generally easier to add them in the Connection Options property.
​
Connection Options
column editor
  • Parameter: A JDBC parameter supported by the database driver. The available parameters are explained in data model. Manual setup is not usually required, since sensible defaults are assumed.
  • Value: A value for the given parameter.
Click the Text Mode toggle at the bottom of the Connection Options dialog to open a multi-line editor that lets you add items in a single block. For more information, read Text mode.To use grid variables, select the Use Grid Variable checkbox at the bottom of the Connection Options dialog.
​
SSH Tunnel
drop-down
Select an SSH tunnel to use for connecting to your IBM Db2 for i database. Read SSH tunnels to learn how to create an SSH tunnel.
If selected, the Connection URL will be the data source that your secure tunnel connects to.

​
Configure

​
Load Type
drop-down
required
  • Full Load: Load all data from the data source on every pipeline run.
  • Incremental Load: Load only new or updated data since the last successful pipeline run using a high-water mark column.
​
Mode
drop-down
required
  • Basic: Configure data selection using the Data Source, Data Selection, Data Source Filter, Combine Filters, and Row Limit properties.
  • Advanced: Write your own SQL query using the SQL Query property.
  • Incremental: Use incremental data loading with a high-water mark column. Available when Load Type is set to Incremental Load.
​
SQL Query
code editor
Write an SQL query to retrieve your data. Only available for Snowflake destinations when Mode is set to Advanced.
​
Schema
drop-down
required
Select the schema to use when browsing available data sources.
​
Data Source
drop-down
required
Select the data source to query from. Available when Mode is set to Basic.
​
Data Selection
dual listbox
required
Select the columns to include from the data source. Available when Mode is set to Basic.
​
Data Source Filter
grid
Filter the rows returned from the data source. Available when Mode is set to Basic.
​
Combine Filters
drop-down
required
Choose how to combine multiple filters. Select AND to return rows matching all filters, or OR to return rows matching any filter. Available when Mode is set to Basic.
​
Row Limit
integer
The maximum number of rows to return from the data source. Leave blank to return all rows. Available when Mode is set to Basic.
​
High Water Mark Selection
drop-down
required
Select the column to use as the high-water mark for incremental loading. Available when Load Type is set to Incremental Load.

​
Destination

​
Destination
drop-down
required
Select the destination for your data. This is either in Snowflake as a table or as files in cloud storage.
  • Snowflake: Load your data into a table in Snowflake. The data must first be staged via Snowflake or a cloud storage solution.
  • Cloud Storage: Load your data directly into files in your preferred cloud storage location. The format of these files can differ between source systems and will not have a file extension so we suggest inspecting the output to determine the format of the data.
When Incremental Load is selected, Cloud Storage is not supported as a destination. Only warehouse destinations (Snowflake or Google BigQuery) are available for incremental loads.
​
Warehouse
drop-down
required
The Snowflake warehouse used to run the queries. The special value [Environment Default] uses the warehouse defined in the environment. Read Overview of Warehouses to learn more.
​
Database
drop-down
required
The Snowflake database to access. The special value [Environment Default] uses the database defined in the environment. Read Databases, Tables and Views - Overview to learn more.
​
Schema
drop-down
required
The Snowflake schema. The special value [Environment Default] uses the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
​
Table Name
string
required
The name of the table to be created in your Snowflake database. You can use a Table Input component in a transformation pipeline to access and transform this data after it has been loaded.
​
Load Strategy
drop-down
required
Define what happens if the table name already exists in the specified Snowflake database and schema. Only available when Full Load is selected.
  • Replace: If the specified table name already exists, that table will be destroyed and replaced by the table created during this pipeline run.
  • Truncate and Insert: Each time the pipeline runs, two operations are performed: first, the table is truncated, meaning all existing rows are deleted. Then, your new rows are inserted. The table itself is never destroyed and recreated.
  • Fail if Exists: If the specified table name already exists, this pipeline will fail to run.
  • Append: If the specified table name already exists, then the data is inserted without altering or deleting the existing data in the table. It’s appended onto the end of the existing data in the table. If the specified table name doesn’t exist, then the table will be created, and your data will be inserted into the table.
​
Primary Keys
dual listbox
Choose one or more columns to be designated as the table’s primary key.When using Incremental Load, if you select a primary key, the loaded data will be merged with your existing data. If you don’t select a primary key, the loaded data will be appended to your existing data.
​
Clean Staged files
boolean
required
  • Yes: Staged files will be destroyed after data is loaded. This is the default setting.
  • No: Staged files are retained in the staging area after data is loaded.
​
Stage Access Strategy
drop-down
Select the stage access strategy. The strategies available depend on the cloud platform you select in Stage Platform.
  • Credentials: Connects to the external stage (AWS, Azure) using your configured cloud provider credentials. Not available for Google Cloud Storage.
  • Storage Integration: Use a Snowflake storage integration to grant access to Snowflake to read data from and write to a cloud storage location. This will reveal the Storage Integration property, through which you can select any of your existing Snowflake storage integrations.
​
Stage Platform
drop-down
required
Use the drop-down menu to choose where the data is staged before being loaded into your Snowflake table.
  • Amazon S3: Stage your data on an AWS S3 bucket.
  • Snowflake: Stage your data on a Snowflake internal stage.
  • Azure Storage: Stage your data in an Azure Blob Storage container.
  • Google Cloud Storage: Stage your data in a Google Cloud Storage bucket.
​
Internal Stage Type
drop-down
required
Select the Snowflake internal stage type. Use the Snowflake links provided to learn more about each type of stage.
  • User: Each Snowflake user has a user stage allocated to them by default for file storage. You may find the user stage convenient if your files will only be accessed by a single user, but need to be copied into multiple tables.
  • Named: A named stage provides high flexibility for data loading. Users with the appropriate privileges on the stage can load data into any table. Furthermore, because the stage is a database object, any security or access rules that apply to all objects will apply to the named stage.
Named stages can be altered and dropped. User stages cannot.
​
Named Stage
drop-down
required
Select your named stage. Read Creating a named stage to learn how to create a new named stage.
There is a known issue where named stages that include special characters or spaces are not supported.

​
Advanced Settings

​
Fetch Size
integer
The number of rows to fetch from the database in each batch. Increasing this value can improve performance when loading large datasets, but may increase memory usage. Leave blank to use the default setting.
​
Output Encoding
drop-down
Set the file format used to stage extracted records from this component before loading. The default is CSV. CSV is compact and widely compatible, while JSON Lines preserves the difference between NULL and empty-string values.
​
Key Sanitization Strategy
drop-down
Choose how to change the column names in your data so that they conform to the destination warehouse’s identifier rules:
  • None: (Default) Do not change the source column names.
  • Alphanumeric (lower case): Replace non-alphanumeric characters with underscores, change the name to lowercase, and add an underscore at the start of the name if it begins with a digit.
  • Alphanumeric (upper case): Replace non-alphanumeric characters with underscores, change the name to uppercase, and add an underscore at the start of the name if it begins with a digit.
If two source column names sanitize to the same result, the load fails with an error rather than dropping data.
​
Parse 'Null' & Empty Strings as NULL
boolean
required
Converts common strings that represent null into a null value. This is case-sensitive and works with the following strings: "", “NULL”, “NUL”, “Null”, “null”. The default is No.
Currently, this property is only applicable when using Snowflake as your destination.
​
Trim String Columns
boolean
required
When Yes, remove leading and trailing characters from a string column. The default is No.