Reviewing a pipeline
Before using the Review feature for the first time, you need to add and configure a rules file, which contains the standards that will use to assess your pipeline quality. To do this:-
In the top right of the canvas, click the drop-down icon next to the Review button, then select Manage rules.
The Review button will be grayed out if you haven’t configured a rules file yet.
- In the rules file, edit the rules to suit your requirements. For more information, read Configuring pipeline quality rules below.
- Close the rules file and return to your pipeline.
- Click Review.
Reviewing all pipelines in a branch
You can review all orchestration and transformation pipelines within the current branch in a single action. To review all pipelines:- In the top right of the canvas, click the drop-down icon next to the Review button.
- Select Review all pipelines.
Fixing rule violations with Maia
From the Review results panel, you can use Maia to automatically fix selected rule violations. To fix rule violations using Maia:- In the Review results panel, review the list of rule violations identified across the pipelines in the branch.
- Use the checkboxes next to each rule violation to select the issues you want Maia to fix. The maximum selection is 50 rule violations.
- Optionally, use the checkbox in the table header to select or deselect all rule violations on the current page.
- Click Fix with Maia in the top-right corner of the panel. This will open the Maia chat with a pre-sent message asking Maia to fix the violations.
Reviewing pipelines before committing
You can choose to run a pipeline quality review as part of the commit process. When committing changes, the Commit changes dialog includes a checkbox to enable pipeline quality review. If the checkbox is selected, runs the pipeline quality review before completing the commit.- If the review identifies rule violations with an error enforcement level, the commit is blocked.
- If the review passes or only identifies rule violations with a warn enforcement level, the commit proceeds successfully.
Configuring pipeline quality rules
The rules that uses to review your pipeline quality are stored in a YAML file calledpipeline_quality_rules. This file is automatically added to the .matillion/config folder in your project when you click Manage rules for the first time. Do not move the pipeline_quality_rules file out of this folder.
Each rule contains one fixed parameter (id) and two or three editable parameters (enabled, enforcementLevel and config). The table below explains what each of these parameters represents.
You can download a copy of the default rules file here.
| Parameter | Value | Description | Editable |
|---|---|---|---|
id | Rule ID | A unique identifier for this rule. This is shown in the Rules results tab. | No |
enabled | true or false | Set to true to apply this rule to your pipeline, or false to skip this rule when reviewing your pipeline. | Yes |
enforcementLevel | error or warn | Choose which severity level to display in the Rules results tab if this rule is not followed. | Yes |
config | Varies depending on the rule | Configure a setting that is specific to this rule, as described below. | Yes |
Individual rules
The table below explains what each rule checks. If a rule has aconfig parameter, the Configuration details column explains how to define the value to check for.
| Rule ID | Applies to | Rule | Configuration details |
|---|---|---|---|
minimum-components | All pipelines | Reviewed pipelines must have at least the set number of components. | Set the minimum number of components, e.g. minComponents: 3. The default value is 1. |
maximum-components | All pipelines | Reviewed pipelines must have no more than the set number of components. | Set the maximum number of components, e.g. minComponents: 20. The default value is 10. |
component-naming-convention | All pipelines | The name of each component in the reviewed pipeline must adhere to a certain regular expression. | Enter a regular expression, e.g. regexMatchPattern: "^[a-zA-Z0-9]+$". |
unused-components | All pipelines | Reviewed pipelines must not contain any unused components. | – |
copied-variables | All pipelines | Copied variables must only be used in concurrent mode iterators. | – |
start-and-end-components | Orchestration pipelines | Reviewed orchestration pipelines must contain a Start component, and all end points of a pipeline must end in an End Success or End Failure component. | – |
maximum-outputs | Transformation pipelines | Reviewed transformation pipelines must have no more than the set number of output components, such as Table Output. | Set the maximum number of output components, e.g. maxOutputs: 2. The default value is 1. |
end-output-components | Transformation pipelines | Each path within a transformation pipeline must end with an output component, such as Table Output. | – |

