Skip to main content
Production use of this feature is available for specific editions only. Contact our sales team for more information.
The AI Mask transformation component uses the Databricks ai_mask() function to invoke generative AI to identify and mask specified entities in unstructured text. This function uses a Databricks chat model serving endpoint made available by Databricks Foundation Model APIs. The input to this component is a column of data in string format. The component operates on a single column only, so if you have multiple text columns you want to mask in the input datastream, you will need to use multiple instances of the AI Mask component, and then combine the outputs downstream in your pipeline. You also need to specify the type of data you want to be masked (for example: name, email). To do this, use the Mask Labels property. The output is a column of string data with the labelled data masked.
Make sure you have read and understand the Requirements set out by Databricks before using this component.

​
Example

A simple example of how masking works is as follows. The input string is: “These comments were made by customer John Doe. For further clarification contact him at john.doe@company.com.” We want to redact the name and email before we share this data. To accomplish this, we run the AI Mask component with the labels person and email. The output string is: “These comments were made by customer [MASKED]. For further clarification contact him at [MASKED].”

​
Use case

The AI Mask component is used to mask sensitive information in any type of input text. Some typical uses of this include:
  • Automatically detect and redact Personally Identifiable Information (PII) or Protected Health Information (PHI) from text fields before storage, sharing, or analysis, to facilitate compliance with privacy standards such as GDPR and HIPAA.
  • Anonymize customer data (like account numbers, names, and emails) in support logs before sharing for training or analytics.
  • Mask sensitive data after loading from source connectors. Data loading components (such as Salesforce Load, HubSpot Load, etc.) don’t include built-in masking, so you can use Databricks AI Mask in a downstream transformation pipeline to redact sensitive information before it’s written to target tables.

​
Properties

​
Name
string
required
A human-readable name for the component.
​
Column
drop-down
required
Select the column that holds data you wish to mask.
​
Mask Labels
column editor
required
Add mask labels as text strings. Each label represents a type of information to be masked. For example, adding a mask label of name prompts the component to mask any names in a given row.
​
Include Input Columns
boolean
required
  • Yes: Includes both your input column and the newly created masked column. This will also include those input columns not selected in Column.
  • No: Only includes the new, masked column.