Skip to main content
Avro files written to this destination can store date, time, and timestamp values in one of two ways, depending on the setting you choose in the Dates and times strategy property when you create a Streaming pipeline:
  • Native Types: This adds Avro logical types to the schema, so that the pre-built pipelines can recognise, date, time, and timestamp columns and load them as the matching types. New pipelines use this by default.
  • Integers from Epoch: This adds no logical types, so date, time, and timestamp columns appear as plain integers in the destination after processing by the pre-built pipelines.
Both strategies write the same value column, but Native Types also adds the logical type to the schema.
If your streaming destination is Snowflake, read Streaming to a Snowflake destination for details.
Changing this property for an already existing pipeline may result in the schema changing and new columns being added to your destination tables. The following table shows how different source time types are written to the Avro file when using Integers from Epoch, and the column type that results when the data is loaded into each cloud data warehouse using the Matillion pre-built pipelines with Native Types.
The epoch integer value is the number of days since the Unix epoch for all data sources except Oracle, where it is the number of milliseconds since the Unix epoch.