Skip to main content
Connectors are a type of component and let you query data from a specific source and load that data into a table in a data warehouse. Use connectors in your orchestration pipelines in Designer to fetch data and make it available to other Maia processes and larger data workflows, such as complex transformations.
A Premium category has been introduced for selected connectors. These connectors require an additional subscription regardless of your current plan. For more information about Premium connectors, contact your Maia account representative.
You can use the following types of connector to fetch data in Maia:
  • Data source connectors let you fetch data using connector-specific properties or an SQL-like query.
  • Custom connectors let you fetch data from any external data source via a REST API that returns JSON data.
  • Flex connectors are pre-built, editable custom connectors that you can configure to meet your needs to fetch data from frequently used data sources.
  • Streaming connectors let you capture data in streaming pipelines.
Custom connectors, Flex connectors, data source connectors, and file connectors do not automatically adapt to schema drift. If the schema of your source table changes, you must manually update the component configuration and the target table to reflect those changes.Components whose name ends in “Load”, such as Salesforce Load or CSV Load, check whether the destination table exists, but don’t check whether it contains data. The Load Strategy parameter controls what happens next—Fail if Exists stops the pipeline rather than writing to the table, Replace or Truncate and Insert overwrite existing data, and Append adds to it without removing anything. The default option varies by warehouse—check the individual component page for details.
Visit the Maia Data Connectors overview to view all the data source connectors and Flex connectors that Maia offers.

​
Data source connectors

Data source connectors are used to retrieve data from your chosen source using connector-specific properties or an SQL-like query. For example, the Google Sheets connector requires you to provide the names of the spreadsheet and worksheet to fetch data from, while the HubSpot Load connector requires you to write an SQL-like query. Connectors that use an SQL-like query let you choose between Basic mode and Advanced mode. Basic mode will build a query for you based on the options you choose in the component properties, while Advanced mode lets you write the query yourself. Connectors with the Load type property allow you to choose between Incremental load and Full load. Incremental load allows you to only load new or updated records each time your pipeline runs.
If a connector appears grayed out on the canvas, this indicates that it has been superseded by a newer connector. This behavior may depend on the cloud data warehouse your project uses. For example, in Snowflake projects the HubSpot Query connector appears grayed out because it has been superseded by the HubSpot Load connector.

​
SQL-like query video example

Expand this box to watch our video about using an SQL-like query to configure a data source connector.

​
Custom connectors

Custom connectors are connectors that you create to fetch data from external data sources that are not available as pre-built connectors in Maia. Custom connectors can only fetch data from REST APIs that return data in JSON format. They cannot fetch data in XML format, and cannot connect to self-signed HTTPS API endpoints. After creating a custom connector, you can use it in orchestration pipelines in Designer like any other connector. For more information about custom connectors, read Custom connector overview. To get started, read Creating custom connectors.

​
Custom connector video example

Expand this box to watch our video about using custom connectors.

​
Flex connectors

Flex connectors are pre-built, editable custom connectors provided in Maia that let you fetch data from frequently used external data sources. They include a curated selection of endpoints, authentication options, and parameters for third-party REST API endpoints that return data in JSON format. If a Flex connector is available for the data source you want to connect to, we recommend using this Flex connector instead of a new custom connector, which requires you to add and configure all the endpoints you want to use. You can add, edit, and remove the endpoints included in a Flex connector in the custom connector editor. For more information about Flex connectors, read Flex connector overview.

​
Flex connector video example

Expand this box to watch our video about using Flex connectors.

​
File connectors

File connectors are used to load data into a table from one or more files of a specific type. These components automatically infer the schema from the structure of the selected file(s), create the corresponding table, and then load the data into the table. CSV Load is an example of a file connector.

​
Streaming connectors

Streaming connectors are built for use in streaming pipelines. They allow you to capture data from a source database so you can then write it to cloud storage or a data warehouse. For more information about streaming pipelines, read Streaming pipelines overview. Each streaming connector is listed in the Sources section.

​
Streaming connector video example

Expand this box to watch our video about using the Microsoft SQL Server streaming connector.