2024

​

Connectors - New features

• Added the Retrieve Run endpoint to the dbt Cloud Flex connector.

​

Agents - Improvements

• The agent template for Azure now deploys with an environment type of “Workload profile - Dedicated” (was previously “Consumption Only”). This allows the agent to better facilitate complex networking, while keeping the ability for the agent to auto-update.

​

Streaming - Improvements

• Updated the CDC agent version from 2.98.7 to 2.101.2.
• For customers who may have experienced an out-of-memory issue with the agent, we have added a new advanced property matillion.compact-history that can be set to true in any affected pipeline definition. This will reduce the memory footprint of the agent by compacting the schema history of the pipeline.
• For pipelines that stream data directly to Snowflake, the agent will create a new column MTLN_CDC_LOAD_TIMESTAMP in each target table to store the load timestamp of each row.
• Library updates and security fixes.

​

API - New endpoints

​

- New features

• Added the new Skip on run action to the component action bar. This allows you to temporarily disable a component in a pipeline, without removing it. For more information, read Skip components.

• Cortex Parse Document is a transformation component that uses Snowflake Cortex to extract content from a document on an internal or external stage in the form of an object that contains JSON-encoded objects as strings.

​

- New features

• Added the option to copy and paste pipelines and pipeline folders in . For more information, read Managing pipelines and Copying pipeline folders.

​

- New features

• Added the option to rename pipeline folders in . For more information, read Pipeline folders.
• Added a new Logging tab to the detailed view that appears when you click a task in the Task history tab. This shows a detailed log of each step in a task, as described in Logging.

​

Streaming - Improvements

• Added a graph of number of rows changed over time to the Manage streaming pipelines dashboard.

​

- New features

• Added a Kafka connector. The Kafka connector is an orchestration component that lets you load data from Kafka topics into either schemaless JSON or JSON schema objects. New to Kafka? Read Apache Kafka’s Introduction.
• Added the following Flex connectors for developing data pipelines:
  • Box
  • Fanatics
  • Snapchat

• Added a new Compare changes feature that allows you to compare your local changes to the previous commit.
• Added commit indicators that show how many commits your local branch is ahead of or behind the remote branch when committing, pushing, or pulling changes.

• Added a GET pipeline executions endpoint, which returns a list of summarized pipeline executions.

​

API - New endpoints

​

Streaming - Improvements

• Updated the CDC agent version from 2.98.4 to 2.98.7.
• Enhanced the resilience of connecting the agent to .
• Library updates and security fixes.

​

- New features and improvements

• Added the following Flex connectors for developing data pipelines:
  • Attio
  • Buildkite

• The Python Pushdown component now includes a Warehouse parameter, allowing you to specify which Snowflake warehouse to run your Python scripts on.
• The Detect Changes component is now available on Databricks projects.

​

- New features and improvements

• Maia (formerly Copilot) is now available for Databricks and Amazon Redshift projects (previously only available for Snowflake projects). In , Maia introduces a new era of AI-driven data productivity tools, letting you prompt generative AI to create data pipelines, improving your efficiency in managing and processing data.
• Updated the S3 Unload component for Amazon Redshift projects. Previously, only CSV file types were supported. Now, the component also supports delimited, fixed with, and Parquet file types.

​

Streaming - Improvements

• Updated the CDC agent version from 2.98.2 to 2.98.4.
• Library updates and security fixes.

​

- New features

• is now available in Snowflake Partner Connect, allowing a quick and simple way to create an automatically configured Matillion account directly from a Snowflake instance.

​

- New features

• You can now connect your own Bitbucket repository to . This feature lets you connect your Bitbucket repository when you create a project in . Follow the documentation to get started:
  • Prerequisites for connecting a Bitbucket repository to
  • Troubleshooting guide for connecting a Bitbucket repository to

• You can now refresh an agent’s credentials, which generates a new client_secret to authenticate with the agent application in your infrastructure (Hybrid SaaS deployments only).

​

- New features

• JDBC Table Metadata to Grid is now available for Amazon Redshift projects.
• Truncate Table is now available for Databricks projects.

​

Streaming - Improvements

• Updated the CDC agent version from 2.97.14 to 2.98.2.
• Change behavior of the Snowflake destination - if the Replication Type is Copy Table or Copy Table with Soft Deletes and the source table does not have a primary key then the Replication Type will be set to Change Log for that table.
• Library updates and security fixes.

​

- Improvements

• Updated Python Pushdown. Added a Snowflake Secret parameter, which is activated when an external access integration is selected. You can select Snowflake secrets and assign them to Snowflake variables, which can then be used in the Python script.

​

Streaming - Improvements

• Updated the CDC agent version from 2.97.12 to 2.97.14.
• Fixed Azure Blob Storage authentication flow.

​

- New features

• Azure Document Intelligence. This component uses the Azure AI Document Intelligence API to automate the extraction of text, handwriting, layout elements, and other key data from forms and documents. The output format can be either Markdown or text.
• Azure Speech Transcribe. This component uses the Azure Speech service to transcribe speech to text from an audio file in your specified Azure Blob location and Speech resource endpoint. Speaker diarization clarifies which speaker spoke and when, increasing the accuracy of your transcriptions.

• Databricks Vector Search. This component lets you perform a search on an input table to find content that best answers specific questions, using vector embeddings to identify suitable answers within data located in your Databricks account. Only available for Databricks projects.

​

- New features

• Add Partition. This orchestration component lets users define the S3 directory structure for partitioned external table data. Amazon Redshift projects only.
• Create External Table. This orchestration component creates a table to reference data stored in an Amazon S3 bucket. The table references external data, meaning the table itself does not hold the data. Amazon Redshift projects only.

​

Streaming - Improvements

• Updated the CDC agent version from 2.95.33 to 2.97.12.
• Improved the process of agent shutdowns so that on clean shutdown the agent status shows as Stopped rather than Unknown.
• Library updates and security fixes.

​

- Improvements

• Snowflake projects now allow the choice of key-pair authentication when configuring environments, so a Snowflake private key can be used.
• Databricks projects now require a Databricks personal access token for authentication when configuring environments. Username/password has been deprecated by Databricks and can no longer be used.

• Additional options are available for Git reset, allowing a reset to your selected working state.

• Bash Pushdown. This component enables users running Full SaaS configurations (Matillion-hosted agents) to use SSH connections to run their Bash scripts in their own instances.

​

- New features

• Snowflake Vector Upsert. This component converts data into vector embeddings in a Snowflake table.

• You can now cross-query two databases with the Table Input and Multi Table Input transformation components for Amazon Redshift projects.

• You can now reorder your pipeline tabs in with click-and-drag.

​

- New features

• Vector Search. This component uses Snowflake vectors to perform a retrieval augmented generation (RAG) search on an input table. This component is available in public preview for Snowflake projects.

• Added SFTP as a data input type in the File Iterator component.
• Table Iterator is now available on Amazon Redshift projects.

• The Git Commit dialog now lists all uncommitted changes.
• Added a command palette to . In its first phase, the following command is supported:
  • Use cmd + k or ctrl + k to activate a file explorer dialog. Alternatively, click the search icon in the upper-right corner of the UI.

​

Streaming - Improvements

• Updated the CDC agent version from 2.95.27 to 2.95.33.
• Library updates and security fixes.

​

- New features

• Amazon Textract. This component uses the Textract API to extract text, handwriting, layout elements, and data from scanned documents. You can optionally choose to include footers and page numbers in the extraction process.
• Amazon Transcribe. This component uses the Transcribe API to extract data from media and audio files and then converts that data into transcriptions.

• Chunk Text is now out of public preview and generally available for all Snowflake projects.

​

- New features

• Cortex Embed. This transformation component lets you convert English-language text into a Cortex vector embedding.
• Remove From Grid. This orchestration component takes in grid variable data and remove data based on the user’s definition.

• NetSuite SuiteAnalytics. This new connector is an orchestration component that uses NetSuite SuiteAnalytics Connect to retrieve data and load it into a table.

​

- New features

• You can now connect your own GitLab repository to . This feature lets you connect your GitLab repository when you create a project in . Follow the documentation to get started:
  • Prerequisites for connecting a GitLab repository to
  • Troubleshooting guide for connecting a GitLab repository to
• Updated Schedules so that you can now specify a time and date to enable pipeline schedules from.
• Updated the “Connect your own GitHub” workflow so that you can now view the list of authorized accounts or organizations after a successful authorization with GitHub.

​

Streaming - New features

• Added the following destinations to streaming pipelines:
  • Amazon S3
  • Azure Blob Storage

​

- New features

• Shared pipelines are now available in . Any pipeline can be saved as a shared pipeline, helping you to maintain consistency of core functionality over multiple different projects across the organization.
• Added the Run Shared Pipeline component, which allows you to call a shared pipeline from within another pipeline.
• Added improvements to environments. When you create an environment, you can now set a default access level for that environment. This access level can also be set on a per-user basis.
  • As part of this change, all users in existing environments have been set to have Admin access by default, to avoid any disruption. Project admins can change a user’s environment access.

​

- New features

• Added a Document AI Predict component, which invokes the Snowflake PREDICT function and allows users to call their Document AI models from the context of a pipeline. This component is available in public preview.

​

Streaming - Improvements

• Updated the CDC agent version from 2.95.22 to 2.95.27.
• CDC will now retain snapshot metrics once a pipeline snapshot completes and streaming begins. Note: these will be lost after an agent restart.
• Improved resilience of connection between s and the platform.
• Library updates and security fixes.

​

Streaming - New features

• Streaming pipelines are now available in . Streaming pipelines are a complete end-to-end solution for near-real-time data ingestion, allowing you to capture and synchronize data from source databases and write to a target data warehouse.
  • Supported data sources are:
    • Db2 for IBM i
    • Microsoft SQL Server
    • MySQL
    • Oracle
    • PostgreSQL
  • Supported data targets are:
    • Snowflake
  • Streaming pipelines are available on our Hybrid SaaS offering, meaning you’ll need to create a streaming agent in your AWS or Azure infrastructure. Read Create an agent in your infrastructure to set up your agent.

​

- New features

• Options to run a single component or run a pipeline starting from a specific component have been added to the pipeline canvas.
• Options to control the behavior of variables in a branching pipeline have been added.
• The Push local changes dialog now shows a list of commits to be pushed.

​

- New features

• Users can now terminate running pipelines from the Pipeline Observability Dashboard.

​

API - New features

• The Pipeline execution API now supports overriding scalar or grid variables when making a call. This enhancement allows for greater flexibility and customization during pipeline execution.

​

Streaming - Improvements

• Updated CDC agent version from 2.95.21 to 2.95.22.

​

- New features

• Added the Cloud Pub/Sub component, which allows messages to be published to a topic on Google Cloud Platform (GCP). Other applications can subscribe to these topics to receive such messages.

​

- New features

• Google Cloud Storage is now available for Snowflake projects in the following areas:
  • Added the Google Cloud Storage Load component, which lets you load data stored on the Google Cloud Storage service into an existing Snowflake table.
  • Added the Google Cloud Storage Unload component, which writes data from a Snowflake table or view into a specified Google Cloud Storage (GCS) bucket in your specified file format (CSV, JSON, or Parquet).
  • Added Google Cloud Storage as a staging platform to all “Query” components (Facebook Query, Salesforce Query, Shopify Query, and so on).
• Added Google Cloud Storage as a source and destination to the Data Transfer component.
• Added Google Cloud Storage as a source and destination to the Excel Query component.
• Added Google Cloud Storage as a source and destination to the File Iterator component.

​

Streaming - Improvements

• Updated CDC agent version from 2.95.1 to 2.95.21.
• Fixed an issue where the thread cap for table writers could be reached when capturing changes on a large number of tables at once and cause the agent to stop streaming.
• The Oracle connector will now gracefully handle skipping transactions when the retention.ms property is configured.
• Library updates and security fixes.

​

- New features and improvements

• Updated the font across to a more modern, readable font.
• Added HTML as a supported data format to the Chunk Text component.
• Added the ability to use multiple Azure Key Vault vaults to store secrets. When you add a new secret definition, you can choose which of your key vaults the secret is stored in.

​

- New features

• Added a JDBC orchestration component for the Hybrid SaaS offering. The JDBC component allows you to run queries on a JDBC-compatible data source that you have a driver for. You can upload your drivers to , and so connect to any sources that Matillion doesn’t provide drivers for as standard.
• Added a Chunk Text orchestration component, which performs pushdown text chunking using a Python user-defined function (UDF) in Snowflake via the computational power of your Snowflake warehouse. You can choose text or Markdown as your data format.
• Added the following Flex connectors for developing data pipelines:
  • Matillion
  • SeatGeek
• Added a Refresh Table orchestration component, which invalidates cached entries of a specified table or view, including both data and metadata. Databricks projects only.

​

- New features

• Added a new Postgres Vector Upsert component, which lets you convert text data stored in your cloud data warehouse into embeddings and then store these embeddings as vectors in your Postgres vector database.
• Added Postgres as a vector database option when retrieval augmented generation (RAG) is enabled on the following components:
  • OpenAI Prompt
  • Azure OpenAI Prompt
  • Amazon Bedrock Prompt
  • Snowpark Container prompt
• You can now use Databricks volumes as your staging platform when using our Query components (Facebook Query, Salesforce Query, and so on).

​

- New features and improvements

• Added support for AWS Guardrails to the Amazon Bedrock Prompt component.
• Removed Pinecone Environment and Pinecone Project ID parameters from AI components where users can interact with Pinecone as these parameters are no longer required.
• The following new components that support Databricks LLM operations are available for Databricks projects in public preview:
  • Analyze sentiment
  • Classify
  • Extract
  • Fix grammar
  • Mask
  • Query
  • Similarity
  • Summarize
  • Translate

​

- New features

• Connecting your own Azure DevOps repo to projects is now generally available. This feature lets you connect your Azure DevOps repository when you create a project in . Follow the documentation to get started:
  • Overview of the Matillion Azure app
  • Prerequisites for connecting an Azure DevOps repository to
  • Installing the Matillion Azure app
  • Troubleshooting an Azure DevOps repository connection
• Added the following Flex connectors for developing data pipelines:
  • ClickUp
  • Rootly
• Added an SAP ODP orchestration component, for connecting directly through SAP ODP to access available data sources in SAPI and ABAP CDS views.
• Added the ability to upload external drivers to allow you to enable third-party connectors that don’t have a driver supplied by default in a Hybrid SaaS configuration, such as SAP ODP.
• Added support for the OAuth 2.0 client credentials authentication type in when using custom connectors.
• Updated data sampling for orchestration components to include a drop-down menu where you can set the number of rows sampled, for example 1, 10, 250, 1000. The default is 25 rows.

​

Streaming - Improvements

• Updated the CDC agent version from 2.93.8 to 2.95.1.
• Added an option to map date and time source columns to the corresponding Snowflake data types when using Snowflake as a target.
• Changed the MTLN_CDC_LAST_COMMIT_TIMESTAMP column when using Snowflake as a target with the Change Log transformation type.
• Library updates and security fixes.

​

- New features

• Added a component action bar that will be displayed when you click on a component in the canvas. The action bar currently supports copying the component and deleting the component.
• Updated data sampling for transformation components to include a drop-down menu where you can set the number of rows sampled, for example 1, 10, 250, 1000. The default is 25 rows.
• Added an Append To Grid component, which lets you append or prepend data to a pre-existing grid variable in . Running this component can add rows to existing columns, but doesn’t add columns to the grid.
• Added the following Flex connectors for developing data pipelines:
  • Float
  • Ortto

​

- New features

• Project administrators can now delete projects.

​

- New features

• Added support for GPT-4o to the OpenAI Prompt component. When this model is selected, the Image input type is available. Images should be in either Base64 format or a direct, public URL of the image. If you’re feeding images in Base64 format, you will need to add data:image/{format_of_your_image};base64, before the encoded image.
• Added a Volume to Delta Table orchestration component for Databricks, which lets users transfer data from a pre-existing volume in Databricks in to a Delta Lake table without replacing or deleting any existing data.
• Added a Convert String To Struct transformation component, which takes a string input and converts it to a struct data type.
• Renamed the Extract Nested Data component to Extract Structured Data for Databricks. This transformation component unpacks arrays of structured data into columns and rows of data in a table. Any existing pipelines will still display an Extract Nested Data component.
• The Query Result To Grid component is now available for Databricks projects.
• Added the following Flex connectors for developing data pipelines:
  • dbt Cloud
  • Pingdom
• A user’s account ID is now visible in the user menu (top right of UI), to help users easily identify their account.

​

Custom Connector - New features

• Added a Script paging option for connector pagination. Paging scripts give you complete control over response pagination in Custom Connector.

​

- New features

• Maia (formerly Copilot) for is now available in public preview for transformation pipelines in Snowflake projects. Maia lets you prompt generative AI to create data pipelines, improving your efficiency in managing and processing data. Read our docs to get started:
  • Maia overview
  • Start using Maia
• The Query Result to Scalar component is now available for Databricks projects. This orchestration component allows you to query a table and return a scalar value. This value can then be mapped to a project or pipeline variable for use in other pipeline components.

​

- New features

• Added Amazon Bedrock as an embedding provider and added Titan Embeddings G1 - Text as an embedding model for the following AI components:
  • OpenAI Prompt
  • AzureOpenAI Prompt
  • Amazon Bedrock Prompt
  • Snowpark Container Prompt (public preview)
  • Pinecone Vector Query
  • Pinecone Vector Upsert
• Added the ability to configure agents to work with a proxy server in AWS and Azure environments.

​

- New features

• The Snowpark Container Prompt is now available in public preview. This orchestration component lets you host your large language models within your Snowpark Container Services.
• The Pinecone Vector Upsert component is now available for Databricks and Amazon Redshift projects.
• Added the following Flex connectors for developing data pipelines:
  • Orbit
  • Persona
  • PivotalTracker
• The agent for Azure is now available in public preview. This lets you run a Hybrid SaaS configuration with the Matillion agent hosted in your own Azure infrastructure.

​

- New features

• The AI prompt components are now available for Databricks projects:
  • OpenAI Prompt
  • AzureOpenAI Prompt
  • Amazon Bedrock Prompt
• The following new components that support Snowflake Cortex LLM operations are available for Snowflake projects:
  • Cortex Completions
  • Cortex Extract Answer
  • Cortex Sentiment
  • Cortex Summarize
  • Cortex Translate

​

- New features

• The AI prompt components are now available for Amazon Redshift projects:
  • OpenAI Prompt
  • AzureOpenAI Prompt
  • Amazon Bedrock Prompt
• Added two new connectors:
  • Gmail, which lets you query the Gmail API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
  • SurveyMonkey, which lets you query the SurveyMonkey API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
• Added the following components for Snowflake projects:
  • Azure Queue Storage Message lets you post a message to Azure Queue Storage. Other applications can read those messages and perform further processing.
  • JDBC Table Metadata to Grid takes the metadata from a JDBC table and uses this data to populate a grid variable.
  • Table Metadata to Grid takes the metadata from a table and uses this data to populate a grid variable with data.
• The Git menu can now be accessed by clicking the active branch.
• The Data Transfer component now supports SFTP as a target.

​

Streaming - Improvements

• Updated CDC agent version from 2.93.1 to 2.93.8.
• Improved messaging and logging for some scenarios.
• Library updates and security fixes.

​

- New features

• The Python Pushdown component is now generally available for Snowflake users. This component lets you execute a Python script using the Snowpark service in your Snowflake account.

​

- New features

• Amazon Redshift is now generally available as a cloud data warehouse in .

​

- New features and improvements

• Added a Microsoft Exchange component, which lets you query the Microsoft Exchange API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
• Added a ServiceNow Query component, which lets you query the ServiceNow API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
• Added logging to Custom Connector and to Flex and custom connectors used in . You can choose one of five logging options: Error, Warn, Info, Trace, or Debug.
• Improved component sampling to enable column resizing.

​

Streaming - Improvements

• Updated version from 2.92.3 to 2.93.1.
• Removed a number of unnecessary dependency overrides and updated some minor dependencies.

​

- New features

• Added the following Flex connectors for developing data pipelines:
  • Firehydrant
  • Infobip
  • Jotform
  • Lob
  • Mailgun
  • Notion

​

- New features

• Maia (formerly Copilot) is now available as an individual assignable service in the Edit User menu.
  • Super Admins can enable and disable Maia access for all users in an account.
  • User Admins can enable and disable Maia access for all User Admins and Users in an account (not the Super Admin).

​

- Improvements

• Added an improvement to connecting a GitHub repository to a project. Now, when you connect an existing repository, any active branches will be cloned into and selectable from the Branches tab. Previously only the main branch was cloned. If any of the active branches in your repository only include files that are not Data Pipeline Language (DPL) files, those branches will appear as empty while in until you create a pipeline.

​

- New features

• Added the following Flex connector for developing data pipelines:
  • Azure Cosmos DB for NoSQL

​

Streaming - Improvements

• Updated CDC agent version from 2.92.0 to 2.92.3
• Library updates and security fixes
• Improved agent logging around data handling edge cases to help with debugging

​

- New features

• Added the ability to export pipelines and import them into a new project.
• Added the following connectors for developing data pipelines:
  • Databricks connector
  • Mandrill flex connector

​

Streaming - Improvements

• Updated CDC agent version from 2.91.5 to 2.92.0.
• Added optional STREAMING_AGENT_DEBUG_DISABLE_COMPRESSION Environment Variable to disable compression when uploading to Snowflake Staging area.
• Library updates and security fixes.

​

- New features

• The following AI components are now generally available to all users in :
  • Prompt components:
    • OpenAI Prompt
    • AzureOpenAI Prompt
    • Amazon Bedrock Prompt
  • Vector database operations:
    • Pinecone Vector Upsert
    • Pinecone Vector Query
• AI Note is also now generally available, which lets you invoke generative AI to annotate your data pipelines using notes.
• The project details page in now displays project metadata including the agent type, the cloud data platform, and the type of Git repository (Matillion hosted repository or customer’s GitHub repository).

​

- New features

• The ability to connect your own GitHub repo to is now generally available. This feature lets you connect your GitHub repository when you create a project in . Follow the documentation to get started:
  • Prerequisites for connecting a GitHub repository to
  • Installing the Matillion app in GitHub
  • Troubleshooting guide for connecting a GitHub repository to

​

API - New endpoints

• The API is now generally available and includes the following endpoints:

​

- New features

• Added a Text mode option to components and grid variables, to facilitate rapid input and editing of multiple properties.

​

Streaming - Improvements

• Updated CDC agent version from 2.90.16 to 2.91.5.
• Increased the JVM memory allocation to 60%.
• Fixed a bug with the Direct to Snowflake flow, in which a change to the scale of a numeric column of a table in the source database would not trigger a schema change in the target Snowflake table.
• Library updates and security fixes.

​

- New features

• Added support for lineage for transformation pipelines. With lineage, you get a visual representation of each transformation, which can make understanding the evolution of your data faster and easier. Lineage is currently in public preview and includes:
  • Transformation lineage at runtime.
  • Table-level lineage.
  • Column-level lineage.
  • Table lists.
  • Table metadata—column information and data types.
• Added the following Flex connectors for developing data pipelines:
  • Bugcrowd
  • Salesloft
  • Zendesk Ticketing

​

- Improvements

• Updated the AI Note feature with the ability to refine notes. When you click Refine when creating a note, you can elaborate on the note (adding detail), shorten the note’s length and detail, or regenerate the note.
• Added a Load Strategy parameter to connectors that support storage-only outputs. Choose between appending files in folder and overwriting files in folder, based on exact structure matching.

​

- New features

• Added the ability to access the Snowflake Session object from the Python Pushdown component, to allow reading from and writing to the Snowflake database from within a Python script.
• Added the following Flex connectors for developing data pipelines:
  • Asana
  • New Relic
  • SAP SuccessFactors
  • Ticketmaster
• Added Azure Blob Storage as a data input type for the Excel Query component.
• Added Azure Blob Storage as a source and target type for the Data Transfer component.

​

Streaming - Improvements

• Updated CDC agent version from 2.90.16 to 2.90.35.
• Logging improvements.
• Library updates and security fixes.

​

- New features

• Added the Query Result to Grid component, which queries a table and returns rows of data that are loaded into a predefined grid variable.

​

- New features

• Improved user interface for easier pipeline creation. Read UI basics for details.

​

Streaming - New features

• Added Snapshot Overview to display improved metrics when a pipeline is snapshotting.

​

- New features

• Added Azure Blob Storage as a data input type for the File Iterator component.

​

- New features

• Added support for grid variables in pipelines.
• Added a Grid Iterator component for working with grid variables.

​

- New features

• Added the Mailchimp component, which lets you query the Mailchimp API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
• Added the following Flex connector for developing data pipelines:
  • Mixpanel
• Added a toggle to the Schedules tab where users can easily enable or disable their schedules.

​

- New features

• Added the Azure SQL component, which lets you run an SQL query on an Azure SQL database and copy the results to a table. You can then use transformation components to enrich and manage the data in permanent tables.

​

- New features

• Databricks is now a generally available cloud data platform for .
• The following components have been added to for Databricks:
  • Except
  • Extract Nested Data
  • Map Values
  • Pivot
  • Unpivot

​

Streaming - Improvements

• Updated CDC agent version from 2.89.4 to 2.90.16.
• Library updates and security fixes.

​

- New features

• Added a Commands for dbt Core component. This component empowers you to run dbt commands as part of your orchestration pipelines.

​

- Improvements

• Improved the navigation menu, adding links to manage your custom connectors and agents.

​

- New features and improvements

• Added a Python Pushdown component to . This component lets you execute a Python script using the Snowpark service in your Snowflake account. This component is currently available as a public preview.
• Improved the schedule feature to include a Standard mode and an Advanced mode.
  • Standard mode lets you define intervals with the units Days (default), Weeks, Hours, and Minutes.
  • Advanced mode retains the Cron expression scheduling feature.

​

Streaming - New features

• Added option to configure or disable the table name prefix when using Snowflake as a target.

​

- New features

• Added the following Flex connector for developing data pipelines:
  • Intercom

​

- New features and improvements

• Added the following Flex connector for developing data pipelines:
  • Ongoing WMS
• Fixed an issue where using the Rename component would reorder columns.
• Fixed an issue where using the Convert Type component would reorder columns.

​

- New features

• Added the following Flex connector for developing data pipelines:
  • Dixa

​

- New features

• Added the following Flex connectors for developing data pipelines:
  • GitHub
  • Yelp