Skip to main content
Connectors are a specific type of component that can be added to orchestration pipelines in and used as part of larger data workflows in , including complex transformations. A connector’s key function is to extract data from a specific data source and load that data into a table in a destination, such as a data warehouse or cloud storage location. When building pipelines, use one of ‘s built-in connectors to fetch data from your chosen data source, or create your own custom connector to connect to any REST API that returns data in JSON format. Visit Connect to any source to review all the connectors offers.
A Premium category has been introduced for selected connectors. These connectors require an additional subscription regardless of your current plan. For more information about Premium connectors, contact your account representative.

​
Connector types

Several types of connector are available in . Each type is designed to help you collect data from your chosen source in a specific way:
  • Custom connectors let you configure endpoints to fetch data using a REST API.
  • Flex connectors are pre-built connectors that also use REST APIs to fetch data. You can add and edit the endpoints these connectors use.
  • Data source connectors are pre-built connectors that you configure in to extract data from your chosen source.
  • File connectors extract data from specific files and load it into your destination.
Custom connectors, Flex connectors, data source connectors, and file connectors do not automatically adapt to schema drift. If the schema of your source table changes, you must manually update the component configuration and the target table to reflect those changes.

​
Custom connectors

You can create your own connectors based on any REST API that returns JSON data by using the Custom Connector interface. These custom connectors can then be brought into or Data Loader to build pipelines. You will require access and knowledge of the target API to create these. For more information, read Custom Connector overview, or to get started, read Custom connector setup. Expand this box to watch our video about using custom connectors.

​
Flex connectors

Flex connectors are pre-configured connectors available directly in orchestration pipelines in . They fetch data using REST APIs and come with preconfigured endpoints, which you can configure and add to as required. Expand this box to watch our video about using Flex connectors.

​
Data source connectors

Data source connectors are used to extract data from a specific source and load it directly into a table in a cloud data warehouse or cloud storage destination of your choice. When configuring these connectors, you can choose between Basic mode and Advanced mode. Basic mode will build a query for you based on the options you choose in the component properties, while Advanced mode lets you write the query yourself. For Snowflake and Google BigQuery projects, some data source connectors also offer an Incremental load option, which only loads records that have been added or updated since the last load.
  • Stripe Load is an example of a data source connector that offers incremental load.
  • Google Analytics is an example of a data source connector without incremental load.
If a connector appears grayed out on the canvas, this indicates that it has been superseded by a newer connector. This behavior may depend on the cloud data warehouse your project uses. For example, in Snowflake projects the HubSpot Query connector appears grayed out because it has been superseded by the HubSpot Load connector.
Expand this box to watch our video about using data source connectors.

​
File connectors

File connectors are used to load data into a table from one or more files of a specific type. These components automatically infer the schema from the structure of the selected file(s), create the corresponding table, and then load the data into the table.
  • CSV Load is an example of a file connector.

​
Streaming connectors

Streaming connectors are built for use in streaming pipelines. They allow you to capture data from a source database so you can then write it to cloud storage or a data warehouse. Streaming connectors can adapt to schema drift, if the schema of your source data changes. For more information, read Streaming pipelines overview, which lists all available streaming connectors and provides more details about schema drift. Expand this box to watch our video about using the Microsoft SQL Server streaming connector.