Skip to main content
This tutorial is designed for users who have signed up for a free trial of Maia or registered for a Maia account, and want help creating their first pipelines. In this tutorial, you’ll learn how to:
  • Create a new working branch
  • Build an orchestration pipeline
  • Commit, push, and publish your changes
  • Schedule your pipeline to run at regular intervals
This tutorial explains how to configure the Jira Load connector to fetch data from Jira and load data into a Snowflake warehouse, but these options are just examples. The configuration steps are similar for all connectors except custom connectors, so you can follow this tutorial using any pre-built connector of your choice. If you want to follow this tutorial using the Jira Load component, you’ll need valid login credentials for the Atlassian portal and Jira.
Throughout this tutorial, you’ll see information boxes like this. These contain additional information about steps in the tutorial, or links to related documentation that we think you might find interesting or useful, but this information isn’t necessary to complete the tutorial.

Example scenario

You’re a project manager who uses Jira to coordinate your team’s tasks, and you want better visibility of the work that your team hasn’t finished yet. You want to build a pipeline that fetches all outstanding Epics in your Jira project and loads this data into Snowflake so you can view and share it. Next, you want to share this pipeline with your colleagues by pushing it to the remote repository, so that they can view and edit it if necessary. Finally, you want to schedule it to run weekly, so you’re always up to date.

Prerequisites

Before you can follow this tutorial in Maia, you need a Maia account. To create an account, follow our guide to Sign up for a Maia account. This tutorial assumes you’re using the project that is created for you when you sign up for a Maia account, which is called My first project. If you want, you can add a new project and use that instead, but we recommend using My first project so you can get started as soon as your account is set up.

Create a new working branch

At the top of the Designer canvas, you’ll see your project name, the environment you’re working in, and the branch you’re working on—this is the main branch. Maia uses Git for version control, which helps you collaborate and manage data pipelines in your team. We recommend always working on pipelines on development branches, not on the main branch.
  • For more information about branches, read our Branches guide.
  • For more information about how Git works in Designer, read Git in Designer.
Let’s start by creating a new working branch, based on the main branch:
  1. At the top of the Designer canvas, click the name of your project.
  2. In the Branches tab, click Add branch.
  3. In Branch name, enter a unique name for your new branch. We recommend using a name that corresponds to the work you’re doing. For example, you could call this branch outstanding-jira-epics.
  4. In the Branch from drop-down menu, select main.
  5. In Location, select Remote.
Selecting main and remote in these steps means that your new branch will be based on the remote copy of the main branch, which contains all the work that your team has committed and pushed to the main branch. If you selected Local, your new branch would contain any changes that you’ve made to the main branch, so you should only select this option when you want your new branch to include changes that only exist on your local copy of main.
  1. In Default environment, select default.
Environments define the connection between a project and your chosen cloud data warehouse. The default environment was set up for you during registration, and we recommend using it for this tutorial. Later, you can create new environments with different default values, which saves you time configuring components. For more information, read Environments.
  1. Leave the Push new branch to remote checkbox clear.
This keeps your new branch as a local copy, visible only to you. Pushing a branch to the remote lets you collaborate with members of your team on a single branch, but for this tutorial, you don’t need to share the branch yet.
  1. Click Add branch.
Add branch dialog When your branch is ready, it will appear in the Branches list. Click the name of your new branch (for example, outstanding-jira-epics) to work on this branch in Designer.

Build an orchestration pipeline

Let’s build an orchestration pipeline that fetches data from Jira. To do this, you need to:
  • Create the pipeline
  • Add a Jira Load component

Create the pipeline

  1. Click Add orchestration pipeline on the Designer canvas.
  2. In Pipeline name, enter a unique name for your pipeline, for example fetch-jira-data.
  3. Optionally, in Description, enter a description for the pipeline. This is useful if other members of your team might need to work on or use this pipeline later.
  4. Click Add.
Add orchestration pipeline dialog Your new pipeline will open on the Designer canvas.
Whenever you open a pipeline in Designer, it opens in a tab that shows:
  • The type of pipeline (a blue “O” for orchestration pipelines, a green “T” for transformation pipelines, or a purple flask for test pipelines)
  • The name of the pipeline
  • The status of the file (“N” for a new file, “M” for a modified file, “R” for a renamed file)
For more information about the features and actions available in Designer, read Using Designer. For information about managing pipelines, read Pipelines.

Add a Jira Load component

All new orchestration pipelines contain a Start component. You don’t need to configure this component at all—it simply tells Maia where to start when running the pipeline. Now let’s add a Jira Load component to the pipeline. To do this, you add the component, authenticate a connection to your Jira account, configure the component properties to fetch the data you want, and then tell Maia where to load the extracted data.
  1. Click the + icon to the right of the Start component.
  2. Search for and click the Jira Load component.
Clicking the + icon to the right of an existing component automatically connects it to the next component that you select. This connection determines the order in which each component runs when you run the pipeline. For more information, read about connecting components.
  1. Click the Jira Load component to open the Properties panel on the right of the Designer canvas.
Each component’s properties are described in the Maia documentation. Click the Help icon in the top right of the Properties panel to go directly to that component’s documentation.
  1. In the Connect section of the Properties panel, in the Authentication type drop-down menu, select the authentication type you want to use: a Jira API token, or your Jira password.
  2. In Username, enter your Jira username.
  3. Depending on the authentication type you selected, you’ll need to provide either a Jira API token or your Jira password. In Maia, sensitive information like this is stored as a secret, which you can use to authenticate corresponding components. Let’s create a secret that represents your chosen authentication credential:
  4. Click the API token or Password property.
  5. Click the Secret drop-down menu, then click Add secret.
  6. In Name, give the secret a unique, identifiable name. For example, jake-jira-api-token or amy-jira-password.
  7. Optionally, in Description, enter a description. This is useful if you want other team members to use this secret as well, so they know what credential it represents.
  8. In the Values section, in the Default value field, enter the credential (in this case, your Jira API token or your Jira password).
  9. Click Add. Your new secret will automatically be selected in the Secret field.
Add secret definition dialog
  1. Enter your Atlassian URL. This is required for both authentication methods.
  • Almost all connectors in Maia require you to authenticate a connection to the data source. Different connectors support different authentication methods—for example, many connectors let you use an OAuth connection for authentication. Check the connector’s documentation for detailed authentication instructions.
  • Secrets and secret definitions let you use credentials in Designer without actually sharing passwords or API tokens. This allows everyone in your team to use a set of credentials securely.
  • You can use a secret or connection to authenticate multiple components, provided the credential works for that data source. For example, if you create an OAuth connection to your Google account, you could use that connection for any Google component.
  1. In the Configure section of the Properties panel, tell the Jira Load component what data to fetch. Configure the properties as shown in the table below—if a property isn’t listed here, leave it blank or set to its default value.
  • Load type: Selecting the Full load load type means that the component will fetch your entire dataset every time this component runs. The other option, Incremental load, lets you only load new and updated records from your dataset. For the purpose of this tutorial, use Full load, but Incremental load can be useful when you just want to extract new information from a large dataset.
  • Data Source Filter: These options let you narrow down the data that a connector fetches, without having to fetch the full dataset and then filter it in a transformation pipeline. In this case, you’re telling the connector only to keep rows where the Epic is not done. You can apply as many filters as you want in this property. If you apply more than one filter, you can use the Combine filters property to specify whether rows that meet all filter criteria (AND) or any filter criteria (OR) are fetched. Alternatively, you could create a transformation pipeline and use a Filter component to apply filters.
Configure section of the Properties panel
  1. In the Destination section of the Properties panel, you need to tell Maia where to load the data that you fetch using the Jira Load component. Configure the properties as shown in the table below—if a property isn’t listed here, leave it blank or set to its default value.
Destination section of the Properties panel That’s it—the Jira Load component is authenticated and you’ve configured it to fetch the data you need, then load that data into your chosen table in Snowflake. If the component is configured correctly, you’ll see a green check mark on the component icon on the canvas. This means that Maia has validated the component and confirmed that it will run successfully. It’s good practice to sample the data output by a component to check that it behaves the way you want it to. To do this:
  1. Click the component you want to sample.
  2. Click the Sample data tab at the bottom of the Designer canvas, then click Sample.
The table that appears shows all the data that will be extracted when this component runs. As you can see, your data source filter has been applied—the component only fetches data about Epics that aren’t done. For more information, read Sampling component output and metadata. Sample data tab showing Jira Load output
This tutorial explains how to build a pipeline and configure components yourself, but you can also ask Maia Team to help you build pipelines. Maia Team is a team of expert agents that enable you to work more efficiently in Maia.Why not give it a try now? Click the Maia icon on the Designer canvas to open a chat with Maia Team, and tell it what you want to do. For example, you could say Help me configure a Jira Load component to fetch data about outstanding epics and load it into Snowflake. Maia Team will process your instruction, ask for any information it needs (like the authentication method you want to use), and get started. For more information, read Building pipelines.

Commit and push your changes

Currently, your new pipeline only exists locally (that is, on your copy of the working branch), so no other members of your team can see your branch or the pipeline you’ve created. You now need to commit and push your changes to the remote repository. After you’ve done this, other users in your project will be able to view and access this branch when they pull remote changes. To commit your changes:
  1. Click the name of your branch at the top of the Designer canvas.
  2. In the drop-down menu, click Commit changes.
  3. In the Commit changes panel, enter a commit message that describes the changes you’re committing, such as Created an orchestration pipeline to fetch outstanding Epics from Jira.
Click Generate with Maia to ask Maia Team to write a commit message that describes all the work you’ve done on this branch. This can be useful if you’re committing complex changes in a number of files.
  1. Click Commit.
Commit changes panel Your changes have now been saved on your local branch. Now, let’s push this local commit to the remote branch. To do this:
  1. Click the name of your branch at the top of the Designer canvas.
  2. In the drop-down menu, click Push local changes.
  3. Select the Publish checkbox.
  4. In Environment, select the environment you’re working in—for the purpose of this tutorial, it’s your default environment.
  5. In Artifact version name, enter a name for this version of the pipeline, for example jira-outstanding-epics-v1.
  6. Click Push & Publish.
Push local changes panel
You only need to publish your changes if you want to include them in a schedule. In this case, you want to run this pipeline automatically on a weekly schedule, so it needs to be published. The artifact version is what you tell the schedule to run. For more information, read Schedules.

Schedule the pipeline

You now have a published pipeline to fetch the data you need from Jira and load it into Snowflake. You can run this pipeline manually by clicking Run on the Designer canvas any time you like. However, in this situation you want to fetch and load this data every week—the easiest way to do this is to create a schedule. For the purpose of this tutorial, let’s say your team has a weekly meeting from 9:00 to 10:00 AM on a Monday. In this meeting, everyone updates the status of their Jira Epics, so it makes sense to run the schedule after this meeting, so the pipeline fetches the latest updates from your team. You decide to run the schedule at 10:30 AM every Monday, to allow time for the meeting to run over and for your colleagues to update their Epics. To do this:
  1. At the top of the Designer canvas, click the name of your project.
  2. In the Schedules tab, click Add schedule.
  3. Configure the Add schedule fields as shown in the table below—if a field isn’t listed here, leave it blank or set to its default value.
  1. Click Add.
Schedule timing settings Your schedule will appear in the Schedules tab.

Next steps

Congratulations! You’ve built your first pipeline, shared it with your team by pushing your changes, and scheduled it to run once per week. Here are some suggestions for what to do next in Maia to build on your knowledge:
  • Keep building pipelines:
    • Add another connector to your orchestration pipeline to fetch additional data. Try another Jira Load connector to fetch more details about Jira issues, or a different connector to fetch data from a separate data source.
    • Build a transformation pipeline to dive deeper into your data. Transformation pipelines let you join datasets from multiple connectors, perform SQL operations to cleanse your data, or use AI components to understand your data.
  • Help Maia Team to help you work more efficiently:
    • Create a knowledge graph in Context Engine to give Maia Team more information about what your data means, how it’s structured, and how it connects, so Maia Team can work with your data the way your team does.
    • Create context files and skills that Maia Team can use when working on your prompts. These resources give Maia Team reusable knowledge and instructions, so you don’t have to repeat yourself when prompting.
  • Take a closer look at your data:
    • Ask Maia Team questions about your data. You could ask it which projects have the most outstanding Jira epics, or simply ask it what it thinks are the key takeaways for you to help your team progress.
    • Use our Lineage feature to see where your data originates, how it changes throughout your workflows, and where it goes. This is especially useful once you start building complex pipelines with multiple data sources and transformations.