Skip to main content
The Fixed Flow transformation component lets you generate a table of data for use downstream in the pipeline. This data can be hard-coded into the component, or it can be generated from variables at runtime. When you configure the Fixed Flow component, you first define the columns that the table will have, and then you create as many rows of data as you require in the table. The output of the Fixed Flow component is a table containing the specified data. The output can be joined to any number of downstream components that can then use that data.

​
Use case

The Fixed Flow component gives you a versatile tool for introducing data into your pipelines. Some common uses for this include:
  • Testing and debugging downstream components. During pipeline development, you may want to test downstream components (like filters, joins, and transformations) without using live data. The Fixed Flow will allow you to simulate a dataset with a few rows and columns to run your tests before connecting to actual data sources.
  • Creating static reference or lookup data. Your downstream components may need reference data that is static or rarely changes. For example, a lookup table of currency codes and country names. You could create this lookup table in a Fixed Flow component, and then it would be available to your pipeline without needing to connect to a separate external data source.

​
Properties

​
Name
string
required
A human-readable name for the component.
​
Columns
column editor
required
Name: The name of the input column.Type: Select from:
  • VARCHAR: This type is suitable for strings and text.
  • NUMBER: This type is suitable for whole-number types (no decimals).
  • FLOAT: This type is suitable for numeric types, with or without decimals.
  • BOOLEAN: This type is suitable for data that is either ‘true’ or ‘false’.
  • DATE: This type is suitable for dates without times.
  • TIMESTAMP: This type is a timestamp left unformatted (exists as Unix/Epoch Time).
  • TIME: This type is a formatted time object without date.
  • VARIANT: This is a flexible type that can be used for any purpose.
Size: For Text types, this is the maximum length. This is a limit on the number of bytes, not characters. For Numeric types, this is the total number of digits allowed, whether before or after the decimal point.Scale: Scale is the number of digits to the right of the decimal point in a Float. For example, the number 123.45 has a scale of 2.
​
Values
column editor
required
A dynamic table of values with one column for each column created above.

​
Date, DateTime, Time, and Timestamp validation

A variety of Date, Time, DateTime, and Timestamp values can be used. Some of the most common values accepted are listed below. The list of examples is not exhaustive, but covers many of the most commonly accepted formats. Date values:
  • 13/01/2021
  • 01/13/2021
  • 2021/1/13
  • 13/JAN/2021
  • 2021
  • JAN 13 2021
  • 13 January, 2021
  • January 13, 2021
Time values:
  • 23:12
  • 23:12:59
  • 23:12:59.123456 (milli/micro/nano are all supported)
Time values with offset:
  • 23:12 +01:00
  • 23:12:59 +01:00
  • 23:12:59.123456 +01:00 (milli/micro/nano are all supported)
DateTime and Timestamp values:
  • 13-01-2021 23:12:59.123456 +01:00
  • 2021-01-13 23:12:59.123456 +01:00
  • To avoid confusion between day and month values, the number 13 is used as the value for day in these examples.
  • The backslash character can be replaced by a dash, underscore, full stop, or colon.
  • Dates cannot contain time values.
  • Times cannot contain date values.
  • The DateTime/Timestamp values can be just the date, but not just the time.
  • The DateTime/Timestamp values accepted combine certain date patterns with all the time patterns, split by either a whitespace character or a T character.
  • There are a number of exceptions that are accepted, such as BC dates, negative dates, and also month values greater than 12. None of these are listed here, and we advise caution when using these values.