Prerequisites
To create a knowledge graph, you need:- The Enable AI Agents permission enabled for your account. Without it, Context Engine isn’t visible in the left navigation.
- An account role with the Create knowledge graph permission.
- Warehouse data crawlers harvest your warehouse and structured sources supported via connectors to populate the graphs. This helps keep an accurate view of the data landscape. To run a Warehouse data crawler, your project must contain a schema that is populated with data.
- Pipeline execution crawlers harvest your pipeline executions for your chosen project and environment to build an operational understanding of how data flows through your organization and workflow. To run a Pipeline execution crawler, your project must contain at least one pipeline.
- External source crawlers harvest data from external applications to add business objects to your knowledge graph. To run an External source crawler, you need a connection to the source you want to crawl.
Decide whether to create a new graph
We recommend adding all your business information to one knowledge graph. This helps Maia Team understand your business logic and workflows in more detail, because it can see how new data connects to what it already knows. For example, add a new schema that holds sales data to an existing graph. Create a separate knowledge graph if the data:- Is subject to different data sovereignty requirements, for example if it needs to be processed in a specific region.
- Needs different access requirements.
Create a knowledge graph
- In the left navigation, click , then Context Engine.
- On the Context Engine dashboard, click Add knowledge graph.
-
Click Build a graph to create a new knowledge graph.
If the information relates to an existing knowledge graph, click Add to a graph instead. For more information, read Decide whether to create a new graph.
-
Select the type of crawl you want to perform for the new knowledge graph, and then follow the steps in the corresponding tab below:
- Data warehouse: Populate the knowledge graph with data from tables, columns, and schemas in a data warehouse.
- Pipeline executions: Populate the knowledge graph with data from the pipeline run history in a specific project and environment, including lineage data.
- External source: Populate the knowledge graph with data about business objects from an external application.
- Harvest documents: Populate the knowledge graph with unstructured content from documents, such as runbooks or support materials.
- Data warehouse
- Pipeline executions
- External source
- Harvest documents
- Select the type of cloud data warehouse you want to harvest data from.
- In Project, select a project that is connected to the warehouse you want to harvest data from.
- In Environment, select the environment you want to use to harvest data.
- (Optional) Click Manage next to the list of users in the selected project to view and edit the users who will be collaborators on this knowledge graph. To remove a collaborator, click the trash can icon next to their name.
- (Optional) In the Invite individual to collaborate drop-down, search for and click a Maia user to add them as a collaborator on this knowledge graph.
- Click Next.
- In the Select data step, select the data to crawl:
- If you’re crawling a Snowflake warehouse, select the database and schemas to crawl.
- If you’re crawling a Databricks warehouse, select the catalog and schemas to crawl.
- If you’re crawling an Amazon Redshift warehouse, select the schemas to crawl.
- If you’re crawling a Google BigQuery warehouse, select the datasets to crawl.
- (Optional) Select or deselect the Include data from pipeline executions checkbox to include or exclude pipeline executions in the selected project and environment in the crawl.
- Click Next.
- In the Schedule crawl step, choose how often this crawl should run. The crawl will run immediately after you create this knowledge graph, then follow this schedule.
- Select Standard or Advanced schedule settings.
- In Timezone, select the timezone for the crawl schedule.
- If you selected Standard settings, use the repeat drop-downs to set how often the crawler runs.
- If you selected Advanced settings, enter a cron expression to set how often the crawler runs.
- Click Run.
- Crawlers: Add and edit crawlers to ingest data, and monitor the status and run history of all crawlers in this knowledge graph.
- Documents: Upload documents to ingest unstructured data.
- Projects: Manage the projects that have access to this knowledge graph.
- Access: Manage the users who can contribute to this knowledge graph.
Use your knowledge graph with Maia Team
Your knowledge graph is ready to use. Maia Team automatically selects the most relevant knowledge graph for your prompt when you:- Create a task in Mission Control.
- Chat with Maia Team in Designer. For more information, read Knowledge graphs in chats.