Skip to main content
Production use of this feature is only available on specific Maia plans. Contact our Sales team for more information.
The Embed transformation component uses the Google BigQuery AI.EMBED function to convert text into a vector embedding, using a Google embedding model. The component takes text from a single input column and creates a vector embedding for each row. The output of this component is a column containing the vector embedding as an ARRAY<FLOAT64> value. This column is named by Result column name, which is embedding by default.

​
Use case

This component lets you create embeddings directly in your transformation pipeline, which can then be used for semantic searching, clustering, and identifying similar content. For example, you can use this component to:
  • Generate embeddings for video transcripts, which you can then search to find relevant content.
  • Convert product descriptions into “Similar products” embeddings, which you can use to suggest products to your customers.
  • Categorize support tickets by matching them to the appropriate support team.
The component calls the embedding model once for each input row, so running it on large tables can incur significant inference costs. Usage is also subject to Google BigQuery’s generative AI quotas.

​
Prerequisites

The Google Cloud service account that calls the embedding model must have the Agent Platform User role (roles/aiplatform.user). Which service account this is depends on the Agent platform connection property:
  • Agent platform connection left blank: Google BigQuery calls the model as the Google Cloud service account that your Maia environment uses to access Google BigQuery. This is the service account in the cloud provider credentials associated with the environment or, for a Hybrid SaaS environment with no associated credentials, the Maia runner‘s own service account.
  • Agent platform connection set: Google BigQuery calls the model as the service account of the Google BigQuery connection. To create a connection and grant its service account the required role, read Run generative AI queries with a Google BigQuery Connection.

​
Properties

​
string
required
A human-readable name for the component.
​
drop-down
required
Select the input column to use to create embeddings. Only text (STRING) columns are listed.
​
drop-down
required
The embedding model to use to generate the embedding. The following options are currently available:
  • text-embedding-005: This is the default setting.
  • gemini-embedding-001
  • text-multilingual-embedding-002
  • [Custom]: Use an embedding model that isn’t listed here, which you name in Custom model.
text-embedding-005 supports English text only. For text in other languages, select gemini-embedding-001 or text-multilingual-embedding-002.To set the number of dimensions of the embedding, use Model parameters.
​
string
required
The name of the embedding model to use—for example, gemini-embedding-2. Read the AI.EMBED documentation for the embedding models that Google BigQuery supports.This property is only available when Model is set to [Custom].
​
drop-down
The intended use of the embedding, which helps the model produce better quality embeddings. If you leave this blank, the model uses RETRIEVAL_QUERY. The following options are available:
  • RETRIEVAL_QUERY
  • RETRIEVAL_DOCUMENT
  • SEMANTIC_SIMILARITY
  • CLASSIFICATION
  • CLUSTERING
  • QUESTION_ANSWERING
  • FACT_VERIFICATION
  • CODE_RETRIEVAL_QUERY
For semantic search, use RETRIEVAL_DOCUMENT to embed the texts that you search over, and RETRIEVAL_QUERY to embed the search queries. Read the AI.EMBED documentation to learn what each task type means.The gemini-embedding-2 model doesn’t accept a task type or a title.
​
string
A document title, which the model uses to improve the quality of the embedding.This property is only available, and only used, when Task type is set to RETRIEVAL_DOCUMENT.
​
string
The Google BigQuery connection to use to call the embedding model, in the format <project_id>.<location>.<connection_id>—for example, myproject.us.myconnection. Leave this blank to call the model as the Google Cloud service account that your environment uses to access Google BigQuery.
​
JSON
A JSON object of additional parameters to send to the model. You can set any field of the text embeddings parameters object. The JSON must be well-formed.For example, to return a 256-dimensional embedding for each row:
​
boolean
required
  • Yes: Outputs both your source input columns and the new embedding column. This also includes input columns not selected in Text column. This is the default setting.
  • No: Only outputs the new embedding column, and the status column if Include status is set to Yes.
​
string
required
The name of the column that holds the embedding. The default is embedding.When Include status is set to Yes, the name must be different from Status column name. When Include input columns is set to Yes, it must also be different from the names of the input columns.
​
boolean
required
  • Yes: Adds a column that reports the outcome of the embedding call for each row. Read Explanation of output for details.
  • No: Doesn’t add a status column. This is the default setting.
​
string
required
The name of the status column. The default is status.The name must be different from Result column name. When Include input columns is set to Yes, it must also be different from the names of the input columns.This property is only available when Include status is set to Yes.

​
Explanation of output

The output contains the input columns (if Include input columns is set to Yes), followed by the embedding column, named by Result column name, which holds each row’s embedding as an ARRAY<FLOAT64> value. If Include status is set to Yes, the status column comes last. The status is empty when the embedding call succeeds. Otherwise, it contains the API response status for that row. A row whose embedding call fails doesn’t stop the run. Its embedding is empty: NULL, which becomes an empty array when written to a table. To find which rows failed and why, set Include status to Yes.