> ## Documentation Index
> Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# 2023

export const metl = "Matillion ETL";

export const cdc_runner = "CDC agent";

export const s_runner = "Streaming runner";

export const designer = "Designer";

export const maia = "Maia";

Here you'll find the 2023 changelog for {maia}. Just want to read about new features? Read our [New Features](https://roadmap.matillion.com/changelog) blog.

For an explanation of minimum agent version, read [Agent version tracks](/docs/guides/runner-overview#agent-version-tracks).

<Update label="December 12, 2023" description={designer}>
  ## Designer - New features

  * Added Google Cloud Storage as a "direct to storage" option when using Flex connectors.
  * Added REST API service support to [SharePoint Query](/docs/components/sharepoint-query). This service requires OAuth authentication.
  * Added the following Flex connector for developing data pipelines:
    * [SendGrid](/docs/components/sendgrid)
</Update>

<Update label="December 11, 2023" description="Streaming">
  ## Streaming - Improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.88.3` to `2.89.4`.
  * Added option to disable the table name prefix when using Snowflake as a target.
  * Fix support for Snowflake when using the `QUOTED_IDENTIFIERS_IGNORE_CASE` property.
  * Additional logging when unable to connect to the Matillion Platform.
</Update>

<Update label="December 8, 2023" description={designer}>
  ## Designer - New features

  * Added support for "connect your own GitHub repo" to {maia}. This feature lets you connect your GitHub repository when you create a [project](/docs/guides/projects) in {maia}. Follow the documentation to get started:
    * [Prerequisites for connecting a GitHub repository to {maia}](/docs/guides/connect-github-repository-prerequisites)
    * [Installing the {maia} Integration app in GitHub](/docs/guides/installing-matillion-app-github-marketplace)
    * [Troubleshooting guide for connecting a GitHub repository to {maia}](/docs/guides/troubleshooting-github-repository-connection).
</Update>

<Update label="December 7, 2023" description={designer}>
  ## Designer - New features and improvements

  * New [Update Scalar](/docs/components/update-scalar) component. This component allows you to change the value of any pipeline variable or project variable during pipeline execution, to a new value decided at the time the component is configured.
  * New [Print Variable](/docs/components/print-variables) component. This component prints the value that a pipeline or project variable has at that specific point in the pipeline execution.
  * Added the following Flex connector for developing data pipelines:
    * [Shopify](/docs/components/shopify)
  * Improved the [Git Merge](/docs/guides/git-merge) workflow to let users merge changes that have been pushed to a remote branch *into* your local branch. This improvement empowers users to validate and test changes before pushing merged changes to the remote repository.
</Update>

<Update label="November 28, 2023" description={designer + ", " + maia}>
  ## Designer - New features

  * Added Azure as a staging location to all connectors.
  * Added the following Flex connector for developing data pipelines:
    * [Zendesk Talk](/docs/components/zendesk-talk)

  ## Maia - New features

  * Added a **Recent pipelines** menu to {maia} UI. {maia} will dynamically display your most recent data pipelines, with identifying metadata such as the project platform, Git branch, and pipeline folder name.
</Update>

<Update label="November 21, 2023" description={designer}>
  ## Designer - New features

  * Added the following component for developing data pipelines:
    * [Query Result To Scalar](/docs/components/query-result-to-scalar)
</Update>

<Update label="November 14, 2023" description={designer}>
  ## Designer - New features

  * Added the following Flex connector for developing data pipelines:
    * [Square](/docs/components/square)
</Update>

<Update label="November 8, 2023" description={"Batch, Streaming"}>
  ## Batch - New features and improvements

  * Improved the **Set frequency** page of the batch pipeline wizard, including adding an advanced mode with support for Quartz cron expressions.

  ## Streaming - Improvements

  * Fixed issue where data would not load into tables for pipelines with Snowflake as a destination.
</Update>

<Update label="November 7, 2023" description={designer}>
  ## Designer - New features

  * Added the following connectors for developing data pipelines:

  * [Workday](/docs/components/workday)

  * [Pipedrive](/docs/components/pipedrive)
</Update>

<Update label="November 2, 2023" description={designer}>
  ## Designer - New features

  * Added the [Workday Custom Reports](/docs/components/workday-custom-reports) connector to {designer}. This component uses the Workday RaaS (Reporting as a Service) API to retrieve and store data—such as employee, financial, and business-related data—from Workday, to be loaded into a table.
</Update>

<Update label="November 1, 2023" description="Streaming">
  ## Streaming - New features and improvements

  * Added **Copy Table with Soft Deletes** as a transformation type for pipelines with Snowflake as a destination.
  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.87.8` to `2.88.2`.
  * Improved upload speed when writing to a Snowflake staging table.
</Update>

<Update label="October 31, 2023" description={designer}>
  ## Designer - New features

  * Added the following connectors for developing data pipelines:
    * [Braze](/docs/components/braze)
    * [Chargebee](/docs/components/chargebee)
    * [Eventbrite](/docs/components/eventbrite)
    * [Gong](/docs/components/gong)
    * [PayPal](/docs/components/paypal)
    * [Slack](/docs/components/slack)
  * Added Azure Storage as a storage destination for the following connectors:
    * ActiveCampaign
    * Amplitude
    * Anaplan
    * Braze
    * Brevo
    * Chargebee
    * CircleCI
    * Concord
    * Confluence
    * Datadog
    * Delighted
    * Eventbrite
    * Freshdesk
    * Gong
    * Klaviyo
    * LaunchDarkly
    * PagerDuty
    * PayPal
    * Pendo
    * Productboard
    * Recurly
    * Slack
    * Smartsheet
    * Snyk
    * TikTok
    * Toggl
    * Twilio
</Update>

<Update label="October 27, 2023" description={designer}>
  ## Designer - New features

  * Added the [Anaplan](/docs/components/anaplan) connector.
</Update>

<Update label="October 26, 2023" description={designer + ", Custom Connector"}>
  ## Designer - Improvements

  * Updated the **Add Components** panel with clickable filter buttons under the search bar. Click a filter button to only display components tagged for that filter. You can activate multiple filter buttons.
  * Updated the **Add component** call-to-action (CTA) dialog to include an output connection setting. When adding a component to the canvas, you can now confirm whether that component should be run:
    * On success (green connection line).
    * Unconditionally (gray connection line).
    * On failure (red connection line).
  * You can now sample Query components in orchestration pipelines. To do this:
    * Click on a *validated* Query component in your orchestration pipeline, e.g. Salesforce Query.
    * Click the **Sample** tab.
    * Click **Sample data**.
  * Added row count when sampling components in a transformation pipeline. To view the total rows:
    * Click on a validated transformation component, e.g, Table Input.
    * Click the **Sample** tab.
    * Click the refresh button.

  ## Custom Connector - New features

  * The ability to import pre-existing connectors using the Custom Connector import feature. This feature supports two types of connectors:
    * Custom connectors that have been previously exported.
    * {metl} connectors.
</Update>

<Update label="October 18, 2023" description={designer}>
  ## Designer - New features

  * Added support for [Flex connectors](/docs/components/connectors-overview#flex-connectors) to {designer}. In {maia}, Flex connectors let you connect to a curated set of endpoints to load data. 20 Flex connectors are initially available for {designer}:

  1. [ActiveCampaign](/docs/components/activecampaign)
  2. [Amplitude](/docs/components/amplitude)
  3. [Brevo](/docs/components/brevo)
  4. [CircleCI](/docs/components/circleci)
  5. [Concord](/docs/components/concord)
  6. [Confluence](/docs/components/confluence)
  7. [Datadog](/docs/components/datadog)
  8. [Delighted](/docs/components/delighted)
  9. [Freshdesk](/docs/components/freshdesk)
  10. [Klaviyo](/docs/components/klaviyo)
  11. [LaunchDarkly](/docs/components/launchdarkly)
  12. [PagerDuty](/docs/components/pagerduty)
  13. [Pendo](/docs/components/pendo)
  14. [Productboard](/docs/components/productboard)
  15. [Recurly](/docs/components/recurly)
  16. [Smartsheet](/docs/components/smartsheet)
  17. [Snyk](/docs/components/snyk)
  18. [TikTok](/docs/components/tiktok)
  19. [Toggl](/docs/components/toggl)
  20. [Twilio](/docs/components/twilio)
</Update>

<Update label="October 12, 2023" description={designer}>
  ## Designer - New features

  * Added the [Assert View](/docs/components/assert-view) component, which lets you verify that certain conditions are true of a view, or otherwise stop the query.
</Update>

<Update label="October 10, 2023" description={designer}>
  ## Designer - New features and improvements

  * Updated the **Add Components** panel to un-nest components. We believe this will make finding the component you're looking for much easier. Components are listed alphabetically and include a descriptor keyword, e.g. "Connectors" or "Flow". You can use these descriptors to filter out components that don't match that keyword. For example, typing "flow" in the search bar will return only the Flow components (And, End Failure, End Success, If, Or, Retry, Restart). The search also includes a number of synonym keywords to help filter components. For example, typing "logic" into the search bar will also return the Flow components.
  * Updated the pipeline canvas to include a **+** call-to-action button. When you select a component on the canvas, the **+** button will appear to the right. Clicking this button will open the **Add component** dialog. This dialog works similarly to the **Add Component** panel.
    * Components are listed alphabetically.
    * Descriptor and synonym keywords help you refine your search.
    * Clicking a component in this dialog will add that component to the canvas in place of the **+** button and close the dialog. The new component will be connected to the previously selected component.
    * This newly added component will become the selected component, meaning the **+** button will shift to the right.
  * Added the [Table Update](/docs/components/table-update) component, which lets you update a target table with a set of input rows. The rows to update are based on matching keys.
</Update>

<Update label="October 5, 2023" description={designer}>
  ## Designer - Improvements

  * The Git **Commit changes and push** option has been split into two separate operations: [Commit](/docs/guides/git-commit) and [Push local changes](/docs/guides/git-push).
</Update>

<Update label="September 20, 2023" description={designer}>
  ## Designer - Improvements

  * The [Data Transfer](/docs/components/data-transfer) component now supports SFTP as a source type.
</Update>

<Update label="September 19, 2023" description={designer}>
  ## Designer - New features and improvements

  * Oracle is now provided as a driver and a supported database type in the [Database Query](/docs/components/database-query) and [RDS Query](/docs/components/rds-query) components.
  * Added support for multi-line secret values for Full SaaS, Matillion hosted projects when creating [secrets and secret definitions](/docs/guides/secrets-and-secret-definitions).
  * Added an **Add schedule** call-to-action button to the {designer} UI to easily navigate from {designer} to the [Create a new schedule](/docs/guides/schedules) menu.
  * Added the ability to [associate newly created cloud credentials](/docs/guides/cloud-credentials#associate-cloud-provider-credentials-with-an-environment) to an environment in a single workflow. You can associate credentials from multiple providers, but only one set of credentials for each cloud provider. For example, you can associate both AWS and Azure credentials, but not two different AWS credentials.
  * Added a "hard reset" functionality to Git in {designer}. [Hard reset](/docs/guides/git-hard-reset) lets users reset their branch to the last local commit.
</Update>

<Update label="September 18, 2023" description={designer}>
  ## Designer - New features

  * Added the following component:
    * [Google Ads Query](/docs/components/google-ads-query)
</Update>

<Update label="September 14, 2023" description={maia}>
  ## Maia - New features and improvements

  * Registered {maia} customers with Account and User Administrator privileges can [edit the Account name and Subdomain name for their account](/docs/administration/manage-accounts#edit-account-details/).
  * Data Loader Batch pipeline runs are now observable from Pipeline run history.
  * Pipeline error messages are now displayed at the top of the **Pipeline run details** page.
</Update>

<Update label="September 12, 2023" description="Streaming">
  ## Streaming - New features and improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.83.5` to `2.87.8`.
  * Added "Copy Table With Soft Deletes" as a transformation type for pipelines with Snowflake as a destination.
  * Fixed an issue where Snowflake role names could not contain non-standard characters.
  * Improved connection reuse for Snowflake as a destination.
</Update>

<Update label="September 7, 2023" description={designer}>
  ## Designer - New features

  * Added the following components:
    * [Azure Blob Storage Load](/docs/components/azure-blob-storage-load)
    * [Azure Blob Storage Unload](/docs/components/azure-blob-storage-unload)
    * [Dynamics 365 Query](/docs/components/dynamics-365-query)
    * [Facebook Ads Query](/docs/components/facebook-ads-query)
    * [Facebook Query](/docs/components/facebook-query)
    * [Salesforce Marketing Cloud Query](/docs/components/salesforce-marketing-cloud-query)
</Update>

<Update label="August 31, 2023" description={designer + ", Streaming"}>
  ## Designer - New features

  * Added the ability to store [cloud provider credentials](/docs/guides/cloud-credentials) for authenticating with AWS and Azure.
  * The following components are now supported in a Matillion Full SaaS environment (previously were Hybrid SaaS only):
    * [Data Transfer](/docs/components/data-transfer)
    * [Excel Query](/docs/components/excel-query)
    * [File Iterator](/docs/components/file-iterator)
    * [RDS Bulk Output](/docs/components/rds-bulk-output)
    * [RDS Query](/docs/components/rds-query)
    * [S3 Load](/docs/components/s3-load)
    * [S3 Unload](/docs/components/s3-unload)
    * [SNS Message](/docs/components/sns-message)
    * [SQS Message](/docs/components/sqs-message)
  * Added the ability to use an S3 location as the stage platform on all query (load) components in a Matillion Full SaaS environment (previously was a Hybrid SaaS function only).

  ## Streaming - New features and improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.83.5` to `2.87.1`.
  * Added on-demand snapshots for pipelines with Db2 for IBM i as a source.
  * Added **Change Log** as a transformation type for pipelines with Snowflake as a destination.
</Update>

<Update label="August 30, 2023" description="Streaming">
  ## Streaming - Improvements

  * Added the in-client **Help** bar with contextual documentation to the "Create Pipeline" pages.
</Update>

<Update label="August 24, 2023" description={designer}>
  ## Designer - New features and improvements

  * Added the following components:
    * [Marketo Query](/docs/components/marketo-query)
  * Replaced the **+** button with an **Add** button that opens a context menu for creating pipelines and folders.
</Update>

<Update label="August 23, 2023" description="Streaming">
  ## Streaming - Improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.81.12` to `2.83.5`.
  * Fixed issue where on-demand snapshots could fail to trigger.
  * Additional validation of the stage format for the Snowflake destination.
  * Added `rs_id` and `ssn` metadata fields to the Oracle change records.
</Update>

<Update label="August 22, 2023" description={designer}>
  ## Designer - New features

  * Added the following components:
    * [Google BigQuery Query](/docs/components/google-bigquery)
    * [NetSuite Query](/docs/components/netsuite-query)
  * Added a **Getting started** wizard for users who load into an instance of {designer} with no existing pipelines.
</Update>

<Update label="August 16, 2023" description={designer}>
  ## Designer - New features

  * Added the following component:
    * [Bing Ads Query](/docs/components/bing-ads-query)
</Update>

<Update label="August 9, 2023" description={designer}>
  ## Designer - New features

  * Added new item **Invite your Teammates** to the help widget's Task checklist, to facilitate collaboration.
</Update>

<Update label="August 3, 2023" description={designer + ", Streaming"}>
  ## Designer - New features and improvements

  * Added the following components:
    * [File Iterator](/docs/components/file-iterator)
    * [Stream Input](/docs/components/stream-input)
  * Added the ability to organize pipelines into [folders](/docs/guides/pipelines#pipeline-folders).
  * Tooltips added to the {designer} UI for type of pipeline and type of component.
  * The {designer} product tour has been replaced with a checklist to help users with creating a pipeline, adding components, and running a pipeline.

  ## Streaming - New features and improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.81.6` to `2.81.12`.
  * Added Direct to Snowflake as a target. Read [Connect to Snowflake](https://docs.matillion.com/data-productivity-cloud/cdc/docs/snowflake-cdc-destination/) to learn more.
</Update>

<Update label="July 27, 2023" description={designer}>
  ## Designer - Improvements

  * Improved the code editor to provide features such as:
    * Syntax highlighting.
    * IntelliSense.
    * Validation support for SQL, Python, and Bash.
    * Command palette menu via `F1`.
    * Visual Studio Code shortcut keys, e.g. CMD/CTRL for multi-cursor editing.
</Update>

<Update label="July 25, 2023" description={designer + ", Streaming"}>
  ## Designer - New features

  * Added the following components:
    * [Email Query](/docs/components/email-query)
    * [HubSpot Query](/docs/components/hubspot-query)
    * [LDAP Query](/docs/components/ldap-query)
    * [Stripe Query](/docs/components/stripe-query)

  ## Streaming - Improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.77.1` to `2.81.6`.
  * Changed "Unavailable Value" placeholder—read [PostgreSQL connector](https://docs.matillion.com/data-productivity-cloud/cdc/docs/31624/#unavailable-field-values) to learn more.
</Update>

<Update label="July 19, 2023" description="Streaming">
  ## Streaming - New features

  * Enabled on-demand snapshotting functionality for Oracle, SQL Server, and MySQL databases.
</Update>

<Update label="July 18, 2023" description={designer + ", Streaming"}>
  ## Designer - New features

  * Added the [SharePoint Query](/docs/components/sharepoint-query) component. This component's driver currently supports:
    * Windows SharePoint Services 3.0
    * Microsoft Office SharePoint Server 2007
    * SharePoint Server 2010
    * SharePoint Server 2013
    * SharePoint Server 2016
    * SharePoint Online

  ## Streaming - New features and improvements

  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.68.1` to `2.76.1`.
  * Updated [{cdc_runner} version](/docs/streaming/agent-versions) from `2.76.1` to `2.77.1`.
  * Added on-demand snapshotting functionality for Oracle, SQL Server, and MySQL databases.
</Update>

<Update label="July 4, 2023" description={designer}>
  ## Designer - New features

  * Added the [Data Transfer](/docs/components/data-transfer) component. Users can transfer files from a chosen source to a chosen target using common network protocols.
    * Currently supported data sources include: FTP, HTTP, HTTPS, Amazon S3, and Windows Fileshare.
    * Currently supported targets: Amazon S3 and Windows Fileshare.
</Update>

<Update label="June 20, 2023" description="Streaming">
  ## Streaming - New features

  * Added on-demand [snapshot functionality for PostgreSQL](https://docs.matillion.com/data-productivity-cloud/cdc/docs/postgresql-snapshot). This feature requires your {cdc_runner} to be version 2.65.0 or higher.
</Update>

<Update label="June 14, 2023" description="Streaming">
  ## Streaming - Improvements

  * [{cdc_runner} version](/docs/streaming/agent-versions) updated to `2.67.4`.
  * Updates to the [Db2 for IBM i](https://docs.matillion.com/data-productivity-cloud/cdc/docs/db2-for-ibm-i-cdc-connector/) connector.
    1. Pipelines using this connector will now fail on startup if any of the journals for the tracked tables do not have IMAGES(\*BOTH) configured.
    2. Pipelines using this connector will now reconnect to the Db2 instance between snapshotting and streaming. This should rectify issues where the connection times out after a long-running snapshot.
</Update>

<Update label="April 12, 2023" description="Batch">
  ## Batch - New features

  * [Db2 for IBM i](https://docs.matillion.com/mdl/docs/db2for-ibmi-batch-pipeline) is now available as a Batch loading pipeline.
</Update>

<Update label="April 5, 2023" description={"Batch, Streaming"}>
  ## Batch - New features

  * Added support for POST API methods to Matillion Custom Connector.
  * Added BODY parameter support for Matillion Custom Connector.

  ## Streaming - Improvements

  * Improved performance related to journal lookup for the [Db2 for i](https://docs.matillion.com/mdl/docs/db2-for-ibm-i-cdc-connector) connector.
</Update>

<Update label="March 15, 2023" description="Streaming">
  ## Streaming - Improvements

  * Improved error handling during the pipeline startup process.
  * [Db2 for IBM i connector](https://docs.matillion.com/mdl/docs/db2-for-ibm-i-cdc-connector): Fix for the SQL catalog and system name differences.

  <Note>
    This connector only works for a single system. Remote system journaling is not supported.
  </Note>
</Update>

<Update label="March 1, 2023" description="Streaming">
  ## Streaming - Improvements

  * Users can now choose a schema before choosing a table when building a CDC pipeline. Read [Create a CDC Pipeline](https://docs.matillion.com/mdl/docs/create-and-manage-cdc-pipeline#choose-schema) to learn more
</Update>

<Update label="February 27, 2023" description="Streaming">
  ## Streaming - Improvements

  * Introduced a `starting` status to pipelines that are starting up.
  * Made the request to stop a pipeline non-blocking.
  * Db2 for i connector: Fix handling of tables where the table name and system name differ.
</Update>

<Update label="February 13, 2023" description="Streaming">
  ## Streaming - Improvements

  * Users can now edit pipelines in-place, without the need to delete and recreate. Read [Create and manage a CDC Pipeline](https://docs.matillion.com/mdl/docs/create-and-manage-cdc-pipeline) to learn more.
</Update>

<Update label="February 1, 2023" description="Batch">
  ## Batch - New features

  * Users can now **stop** a pipeline. Do this from the **Dashboard** page or the **Pipeline Details** page.
  * The **Stop pipeline** button will display the current status of a running pipeline.
</Update>

<Update label="January 25, 2023" description={"Batch, Streaming"}>
  ## Batch - New features

  * Users can now observe the URL being queried, so they know the endpoint they're working with. This feature is specific to the Custom Connector pipeline.

  ## Streaming - Improvements

  * Added an improvement so that users will be [alerted via email](https://docs.matillion.com/mdl/docs/cdc-pipeline-detailed-view#events) if their CDC pipeline fails.
  * {s_runner} deployment approach has improved. Users can now [quick-create](https://docs.matillion.com/mdl/docs/34004/#Using-the-Quick-create-Link) a {s_runner} within AWS, using the link from the Data Loader UI (it will pre-fill values into the template for you). Read [CloudFormation ECS Fargate Advanced Template](https://docs.matillion.com/mdl/docs/34004) for more information.
  * Fixed an issue that prevented data from being properly erased while creating destinations. Now, when editing one destination after another or adding new destinations, users should be able to establish pipelines with cleared data.
</Update>

<Update label="January 20, 2023" description="Streaming">
  ## Streaming - Improvements

  * Users can now manually reset their own key pair without needing to contact {maia} support. Read [Platform Keys](https://docs.matillion.com/mdl/docs/platform-keys#resetting-the-key-pair) for more information.
</Update>

<Update label="January 18, 2023" description={"Batch, Streaming"}>
  ## Batch - Improvements

  * Users can now download error logs for their pipelines as a `.txt` file.
  * Improved validation when creating and editing pipelines by adding per-field validation.
  * Users can now set table names when creating a pipeline (as opposed to the table name being assigned on the source name.)

  ## Streaming - Improvements

  * Column names are now sanitized to ensure conformity with the Avro file format rules.
  * Fixed an issue where special characters in table names would cause issues in the pipeline configuration.
  * Improvement added to now log an error if your Organization ID or Agent ID values are not in the correct format.
</Update>