Test pipelines

A test is a pipeline that runs another pipeline and checks its results. Use tests to verify that your pipelines behave correctly before you deploy them to production. Any pipeline can have multiple tests associated with it, allowing you to check different functions and scenarios for the pipeline. Tests are created on the canvas in the same way as an orchestration pipeline, and will appear as pipelines in the Files panel. Many components available to orchestration pipelines are also available in tests. In addition, a set of dedicated test components—such as assert and compare components—are available exclusively to tests. These components let you check the outputs of the pipeline to test. Available components are summarized below.

---

​

Video example

---

​

Test panel

The Test panel lists all tests associated with a pipeline and lets you manage and run the tests. To open the Test panel, click the Tests tab at the bottom of the canvas. Each test in the list has action icons that let you:

• Run the test.
• Edit the test (opens the test on the canvas).
• Delete the test.

To run all tests for the current pipeline at once, click Run all tests from the Test panel. Results are displayed in the panel once each test completes. Tests run sequentially. To create a new test from the Test panel, click Create test. This opens the Add test dialog.

---

​

Creating a test

You can create a test from the Test panel of the pipeline you want to test, or from the Files panel.

​

From the Test panel

1. Open the pipeline you want to test.
2. Click the Tests tab at the bottom of the canvas.
3. Click Add a test. The Add test dialog opens.
4. Complete the following properties in the dialog:
   • Test name: A name for the test.
   • Target pipeline: This is pre-filled with the current pipeline and can’t be edited.
   • Folder: The folder in which to save the test. The default is the folder containing the current pipeline, but you can select a different folder from the drop-down.
   • Description: An optional description of the test. Limited to 1000 characters.
5. Click Add.

​

From a file in the Files panel

1. In the Files panel, locate the pipeline you want to test.
2. Click the … (more options) menu next to the pipeline, then click Add test. The Add test dialog opens.
3. Complete the dialog properties as described above. The Target pipeline field is pre-filled with the pipeline you selected.
4. Click Add.

​

From a folder in the Files panel

1. In the Files panel, click the … (more options) menu next to any folder name, then click Add → Test. The Add test dialog opens.
2. Complete the dialog properties as described above. For Target pipeline, use the drop-down to select a pipeline. If you have a pipeline open on the canvas, that will be the default selection, but you can select any other pipeline in the folder.
3. Click Add.

In each case, opens a new test on the canvas, pre-populated with a test template.

---

​

Test structure

When you create a test manually (not through ), loads a template containing a set of pre-configured components and pipeline variables. The template isn’t a complete, functioning test—it’s intended as a starting point for you to add components to set up your test data and assert the expected results. The template establishes the following general structure:

1. Set up a test schema: Creates a temporary schema in your Cloud Data Warehouse to isolate the test from your production data.
2. Set up input data: Populates the test schema with the tables and data required by the pipeline to test.
3. Run the pipeline to test: Executes the pipeline you are testing.
4. Assert the results: Checks that the pipeline produced the expected output.
5. Tear down the test schema: Drops the test schema, removing all temporary data.

For example, if your pipeline simply creates a table, the test could create a test schema, run the pipeline, and then check that the table was created with the correct values. Once the test finishes, the test schema is dropped.

​

Test schema

The test schema is a temporary schema created in your Cloud Data Warehouse specifically for the test run. It allows you to run tests in isolation without affecting your production data. The template uses the SQL Script component (labeled as Create Test Schema on the canvas) to create the test schema. It also creates some pre-configured pipeline variables—for example, test_database and test_schema—that reference this schema. Use these variables throughout your test wherever you need to refer to the test schema. You can edit the default values of these variables if required.

​

Input data

After the schema is created, populate it with the tables and data needed by the pipeline to test. You can use components such as:

• Create Table From Values: Creates a table directly from values you enter in the component.
• SQL Script: Executes SQL statements to create and populate tables.
• Zero Copy Clone: (Snowflake only) Clones an existing schema, providing a fast way to create a copy of production data for testing.

​

Running the pipeline to test

Use the Run Pipeline To Test component to execute the pipeline you are testing. This component runs the target pipeline to completion before the test continues. The target pipeline runs within the context of the test schema, so it reads from and writes to the temporary tables you have set up.

​

Asserting results

After the pipeline to test has run, use assert or compare components to verify the results. The following test components are available for this purpose:

• Assert Table: Confirms that a specified table exists.
• Assert Table Values: Checks that a table contains the expected values.
• Assert External Table: Confirms that an external table has been correctly created.
• Assert Scalar Variables: Verifies that a variable holds the expected value.
• Compare Table Values: Checks that two tables contain identical values. This is useful if your pipeline creates a table and you want to verify it against a separately defined expected-results table.

---

​

Available components

Tests support a subset of orchestration pipeline components, as well as some components that are available exclusively to tests.

​

Dedicated test components

​

Available orchestration components