> ## Documentation Index
> Fetch the complete documentation index at: https://docs.maia.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# S3 Load

export const m_runner = "Maia runner";

export const ComponentMetadata = ({warehouses, unsupportedWarehouses = [], componentType, connectionInputs, connectionOutputs}) => {
  const allWarehouses = [...warehouses.map(w => ({
    name: w,
    supported: true
  })), ...unsupportedWarehouses.map(w => ({
    name: w,
    supported: false
  }))];
  return <div style={{
    background: 'var(--colors-background-light, #f9fafb)',
    border: '1px solid var(--colors-border-default, #e5e7eb)',
    borderRadius: '12px',
    padding: '20px 28px',
    marginBottom: '28px',
    boxShadow: '0 1px 4px rgba(0,0,0,0.10)'
  }}>
      <table style={{
    width: '100%',
    borderCollapse: 'collapse'
  }}>
        <tbody>
          <tr>
            <td style={{
    fontWeight: '600',
    paddingRight: '32px',
    paddingBottom: '14px',
    whiteSpace: 'nowrap',
    verticalAlign: 'middle',
    width: '180px'
  }}>Project Availability</td>
            <td style={{
    paddingBottom: '14px',
    verticalAlign: 'middle'
  }}>
              <div style={{
    display: 'flex',
    flexWrap: 'wrap',
    gap: '8px'
  }}>
                {allWarehouses.map((w, i) => <span key={i} style={{
    background: w.supported ? '#dcfce7' : '#fee2e2',
    color: w.supported ? '#15803d' : '#b91c1c',
    border: `1px solid ${w.supported ? '#bbf7d0' : '#fca5a5'}`,
    borderRadius: '9999px',
    padding: '3px 12px',
    fontSize: '0.85rem',
    fontWeight: '500',
    whiteSpace: 'nowrap'
  }}>
                    {w.name} {w.supported ? '✅' : '❌'}
                  </span>)}
              </div>
            </td>
          </tr>
          <tr>
            <td style={{
    fontWeight: '600',
    paddingRight: '32px',
    paddingBottom: '14px',
    whiteSpace: 'nowrap',
    verticalAlign: 'middle'
  }}>Component Type</td>
            <td style={{
    paddingBottom: '14px',
    verticalAlign: 'middle'
  }}>{componentType}</td>
          </tr>
          <tr>
            <td style={{
    fontWeight: '600',
    paddingRight: '32px',
    paddingBottom: '14px',
    whiteSpace: 'nowrap',
    verticalAlign: 'middle'
  }}>Connection Inputs</td>
            <td style={{
    paddingBottom: '14px',
    verticalAlign: 'middle'
  }}>{connectionInputs}</td>
          </tr>
          <tr>
            <td style={{
    fontWeight: '600',
    paddingRight: '32px',
    whiteSpace: 'nowrap',
    verticalAlign: 'middle'
  }}>Connection Outputs</td>
            <td style={{
    verticalAlign: 'middle'
  }}>{connectionOutputs}</td>
          </tr>
        </tbody>
      </table>
    </div>;
};

<ComponentMetadata warehouses={["Snowflake", "Databricks", "Amazon Redshift"]} unsupportedWarehouses={["Google BigQuery"]} componentType="Connector, Orchestration" connectionInputs="One" connectionOutputs="Unlimited" />

The S3 Load component lets users load data into an existing table from objects stored in Amazon Simple Storage Service (Amazon S3).

The S3 Load component never truncates the target table. It has no built-in truncate table option and performs an append-only load. Properties like `Truncate Columns` and `Force Load` are unrelated to clearing the table: `Truncate Columns` governs string value length truncation, while `Force Load` controls whether already-loaded files are reloaded.

This component requires working AWS Credentials, with Read access to the bucket containing the source data files.

To access an S3 bucket from a different AWS account, read [Background: Cross-account permissions and using IAM roles](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-walkthroughs-managing-access-example4.html#access-policies-walkthrough-example4-overview).

<Note>
  If you're using a Matillion Full SaaS solution, you may need to [allow these IP address ranges](/docs/security/network-access-and-ip-allowlist-requirements#full-saas) from which Full SaaS {m_runner}s will call out to their source systems or to cloud data platforms.
</Note>

If the component requires access to a cloud provider (AWS, Azure, or Google Cloud), it will use credentials as follows:

* If using [Matillion Full SaaS](/docs/guides/runner-overview#matillion-full-saas): The component will use the [cloud credentials](/docs/guides/cloud-credentials) associated with your environment to access resources.
* If using [Hybrid SaaS](/docs/guides/runner-overview#hybrid-saas): By default the component will inherit the agent's execution role (service account role). However, if there are [cloud credentials](/docs/guides/cloud-credentials) associated to your environment, these will overwrite the role.

***

## Video example

<iframe width="560" height="315" src="https://www.youtube.com/embed/17U28vO_X9Q?si=krxyChcEKClqEuOQ&enablejsapi=1" title="YouTube video player" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share; fullscreen" referrerPolicy="strict-origin-when-cross-origin" allowFullScreen />

***

## Properties

<ResponseField name="Name" type="string" required>
  A human-readable name for the component.
</ResponseField>

<Tabs>
  <Tab title="Snowflake">
    {/* <!-- param-start:[stage] | warehouses: [snowflake] --> */}

    <ResponseField name="Stage" type="drop-down" required>
      Select a staging area for the data. Staging areas can be created through Snowflake using the [CREATE STAGE](https://docs.snowflake.com/en/sql-reference/sql/create-stage.html) command. Internal stages can be set up this way to store staged data within Snowflake. Selecting \[Custom] will avail the user of properties to specify a custom staging area on S3. Users can add a fully qualified stage by typing the stage name. This should follow the format databaseName.schemaName.stageName
    </ResponseField>

    {/* <!-- param-end:[stage] --> */}

    {/* <!-- param-start:[authentication] | warehouses: [snowflake] --> */}

    <ResponseField name="Authentication" type="drop-down" required>
      Select the authentication method. Users can choose either:

      * **Credentials:** Uses AWS security credentials.
      * **Storage Integration:** Use a Snowflake storage integration. A storage integration is a Snowflake object that stores a generated identity and access management (IAM) entity for your external cloud storage, along with an optional set of permitted or blocked storage locations (Amazon S3, Google Cloud Storage, or Microsoft Azure Blob Storage). More information can be found at [CREATE STORAGE INTEGRATION](https://docs.snowflake.com/en/sql-reference/sql/create-storage-integration.html).
    </ResponseField>

    {/* <!-- param-end:[authentication] --> */}

    {/* <!-- param-start:[storageIntegration] | warehouses: [snowflake] --> */}

    <ResponseField name="Storage Integration" type="drop-down" required>
      Select the storage integration. Storage integrations are required to permit Snowflake to read data from and write to a cloud storage location. Integrations must be set up in advance of selecting them. Storage integrations can be configured to support Amazon S3, Google Cloud Storage, or Microsoft Azure Blob Storage, regardless of the cloud provider that hosts your Snowflake account.
    </ResponseField>

    {/* <!-- param-end:[storageIntegration] --> */}

    {/* <!-- param-start:[s3ObjectPrefix] | warehouses: [snowflake] --> */}

    <ResponseField name="S3 Object Prefix" type="file explorer" required>
      To retrieve the intended files, use the file explorer to enter the container path where the S3 bucket is located, or select from the list of S3 buckets.

      This must have the format `S3://<bucket>/<path>`.
    </ResponseField>

    {/* <!-- param-end:[s3ObjectPrefix] --> */}

    {/* <!-- param-start:[pattern] | warehouses: [snowflake] --> */}

    <ResponseField name="Pattern" type="string" required>
      A string that will partially match all file paths and names that are to be included in the load. Defaults to `.*` indicating all files within the **S3 Object Prefix**. This property is a pattern on the complete path of the file, and is not just relative to the directory configured in the **S3 Object Prefix** property.

      The subfolder containing the object to load must be included here.
    </ResponseField>

    {/* <!-- param-end:[pattern] --> */}

    {/* <!-- param-start:[encryption] | warehouses: [snowflake] --> */}

    <ResponseField name="Encryption" type="drop-down" required>
      Decide how the files are encrypted inside the S3 bucket. This property is available when using an existing Amazon S3 location for staging.

      * **None:** No encryption.
      * **SSE KMS:** Encrypt the data according to a key stored on KMS. Read [AWS Key Management Service (AWS KMS)](https://aws.amazon.com/kms/) to learn more.
      * **SSE S3:** Encrypt the data according to a key stored on an S3 bucket. Read [Using server-side encryption with Amazon S3-managed encryption keys (SSE-S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html) to learn more.
    </ResponseField>

    {/* <!-- param-end:[encryption] --> */}

    {/* <!-- param-start:[kmsKeyId] | warehouses: [snowflake] --> */}

    <ResponseField name="KMS Key ID" type="drop-down">
      The ID of the KMS encryption key you have chosen to use in the **Encryption** property.
    </ResponseField>

    {/* <!-- param-end:[kmsKeyId] --> */}

    {/* <!-- param-start:[masterKey] | warehouses: [snowflake] --> */}

    <ResponseField name="Master Key" type="string" required>
      Your [client-side encryption master key](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/examples-crypto-masterkey.html). This property is only available when using client-side encryption.
    </ResponseField>

    {/* <!-- param-end:[masterKey] --> */}

    {/* <!-- param-start:[warehouse] | warehouses: [snowflake] --> */}

    <ResponseField name="Warehouse" type="drop-down" required>
      The Snowflake warehouse used to run the queries. The special value `[Environment Default]` uses the warehouse defined in the environment. Read [Overview of Warehouses](https://docs.snowflake.com/en/user-guide/warehouses-overview.html) to learn more.
    </ResponseField>

    {/* <!-- param-end:[warehouse] --> */}

    {/* <!-- param-start:[database] | warehouses: [snowflake] --> */}

    <ResponseField name="Database" type="drop-down" required>
      The Snowflake database. The special value `[Environment Default]` uses the database defined in the environment. Read [Databases, Tables and Views - Overview](https://docs.snowflake.com/en/guides-overview-db) to learn more.
    </ResponseField>

    {/* <!-- param-end:[database] --> */}

    {/* <!-- param-start:[schema] | warehouses: [snowflake] --> */}

    <ResponseField name="Schema" type="drop-down" required>
      The Snowflake schema. The special value `[Environment Default]` uses the schema defined in the environment. Read [Database, Schema, and Share DDL](https://docs.snowflake.com/en/sql-reference/ddl-database.html) to learn more.
    </ResponseField>

    {/* <!-- param-end:[schema] --> */}

    {/* <!-- param-start:[targetTable] | warehouses: [snowflake] --> */}

    <ResponseField name="Target Table" type="string" required>
      Select an existing table to load data into. The tables available for selection depend on the chosen schema.
    </ResponseField>

    {/* <!-- param-end:[targetTable] --> */}

    {/* <!-- param-start:[loadColumns] | warehouses: [snowflake] --> */}

    <ResponseField name="Load Columns" type="dual listbox">
      Choose the columns to load. If you leave this parameter empty, all columns will be loaded.
    </ResponseField>

    {/* <!-- param-end:[loadColumns] --> */}

    {/* <!-- param-start:[format] | warehouses: [snowflake] --> */}

    <ResponseField name="Format" type="drop-down" required>
      Select a pre-made file format that will automatically set many of the S3 Load component properties. These formats can be created through the [Create File Format](/docs/components/create-file-format) component.

      Users can add a fully qualified format by typing the format name. This should read as databaseName.schemaName.formatName
    </ResponseField>

    {/* <!-- param-end:[format] --> */}

    {/* <!-- param-start:[fileType] | warehouses: [snowflake] --> */}

    <ResponseField name="File Type" type="drop-down" required>
      Select the type of data to load. Available data types are: AVRO, CSV, JSON, ORC, PARQUET, and XML. Some file types may require additional formatting—this is explained in the Snowflake documentation.

      Component properties will change to reflect the selected file type. Click one of the tabs below for properties applicable to that file type.

      <Tabs>
        <Tab title="AVRO">
          {/* <!-- param-start:[compression] | warehouses: [snowflake] --> */}

          <ResponseField name="Compression" type="drop-down" required>
            Select the compression method if you wish to compress your data. If you do not wish to compress at all, select NONE. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[compression] --> */}

          {/* <!-- param-start:[nullIf] | warehouses: [snowflake] --> */}

          <ResponseField name="Null If" type="editor" required>
            Specify one or more strings (one string per row of the table) to convert to NULL values. When one of these strings is encountered in the file, it is replaced with an SQL NULL value for that field in the loaded table. Click + to add a string.
          </ResponseField>

          {/* <!-- param-end:[nullIf] --> */}

          {/* <!-- param-start:[trimSpace] | warehouses: [snowflake] --> */}

          <ResponseField name="Trim Space" type="boolean" required>
            When True, removes whitespace from fields. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[trimSpace] --> */}
        </Tab>

        <Tab title="CSV">
          {/* <!-- param-start:[compression] | warehouses: [snowflake] --> */}

          <ResponseField name="Compression" type="drop-down" required>
            Select the compression method if you wish to compress your data. If you do not wish to compress at all, select NONE. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[compression] --> */}

          {/* <!-- param-start:[recordDelimiter] | warehouses: [snowflake] --> */}

          <ResponseField name="Record Delimiter" type="string">
            Input a delimiter for records. This can be one or more single-byte or multibyte characters that separate records in an input file.

            Accepted values include: leaving the field empty; a newline character `\` or its hex equivalent `0x0a`; a carriage return `\\r` or its hex equivalent `0x0d`. Also accepts a value of NONE.

            If you set the **Skip Header** to a value such as 1, then you should use a record delimiter that includes a line feed or carriage return, such as `\` or `\\r`. Otherwise, your entire file will be interpreted as the header row, and no data will be loaded.

            The specified delimiter must be a valid UTF-8 character and not a random sequence of bytes.

            Do not specify characters used for other file type options such as Escape or Escape Unenclosed Field.

            The default (if the field is left blank) is a newline character.
          </ResponseField>

          {/* <!-- param-end:[recordDelimiter] --> */}

          {/* <!-- param-start:[fieldDelimiter] | warehouses: [snowflake] --> */}

          <ResponseField name="Field Delimiter" type="string">
            Input a delimiter for fields. This can be one or more single-byte or multibyte characters that separate fields in an input file.

            Accepted characters include common escape sequences, octal values (prefixed by \\), or hex values (prefixed by 0x). Also accepts a value of NONE.

            This delimiter is limited to a maximum of 20 characters.

            While multi-character delimiters are supported, the field delimiter cannot be a substring of the record delimiter, and vice versa. For example, if the field delimiter is "aa", the record delimiter cannot be "aabb".

            The specified delimiter must be a valid UTF-8 character and not a random sequence of bytes.

            Do not specify characters used for other file type options such as **Escape** or **Escape Unenclosed Field**.

            The Default setting is a comma: `,`.
          </ResponseField>

          {/* <!-- param-end:[fieldDelimiter] --> */}

          {/* <!-- param-start:[skipHeader] | warehouses: [snowflake] --> */}

          <ResponseField name="Skip Header" type="integer">
            Specify the number of rows to skip. The default is 0.

            If **Skip Header** is used, the value of the record delimiter will not be used to determine where the header line is. Instead, the specified number of CRLF will be skipped. For example, if the value of **Skip Header** = 1, skips to the first CRLF that it finds. If you have set the **Field Delimiter** property to be a single character without a CRLF, then skips to the end of the file (treating the entire file as a header).
          </ResponseField>

          {/* <!-- param-end:[skipHeader] --> */}

          {/* <!-- param-start:[skipBlankLines] | warehouses: [snowflake] --> */}

          <ResponseField name="Skip Blank Lines" type="boolean" required>
            When **True**, ignores blank lines that only contain a line feed in a data file and does not try to load them. Default setting is **False**.
          </ResponseField>

          {/* <!-- param-end:[skipBlankLines] --> */}

          {/* <!-- param-start:[dateFormat] | warehouses: [snowflake] --> */}

          <ResponseField name="Date Format" type="string">
            Define the format of date values in the data files to be loaded. If a value is not specified or is AUTO, the value for the DATE\_INPUT\_FORMAT session parameter is used. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[dateFormat] --> */}

          {/* <!-- param-start:[timeFormat] | warehouses: [snowflake] --> */}

          <ResponseField name="Time Format" type="string">
            Define the format of time values in the data files to be loaded. If a value is not specified or is AUTO, the value for the TIME\_INPUT\_FORMAT session parameter is used. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[timeFormat] --> */}

          {/* <!-- param-start:[timestampFormat] | warehouses: [snowflake] --> */}

          <ResponseField name="Timestamp Format" type="string">
            Define the format of timestamp values in the data files to be loaded. If a value is not specified or is AUTO, the value for the TIMESTAMP\_INPUT\_FORMAT session parameter is used.
          </ResponseField>

          {/* <!-- param-end:[timestampFormat] --> */}

          {/* <!-- param-start:[escape] | warehouses: [snowflake] --> */}

          <ResponseField name="Escape" type="string">
            Specify a single character to be used as the escape character for field values that are enclosed. Default is NONE.
          </ResponseField>

          {/* <!-- param-end:[escape] --> */}

          {/* <!-- param-start:[escapeUnenclosedField] | warehouses: [snowflake] --> */}

          <ResponseField name="Escape Unenclosed Field" type="string">
            Specify a single character to be used as the escape character for unenclosed field values only. Default is `\\`. If you have set a value in the property **Field Optionally Enclosed**, all fields will become enclosed, rendering the **Escape Unenclosed Field** property redundant, in which case, it will be ignored.
          </ResponseField>

          {/* <!-- param-end:[escapeUnenclosedField] --> */}

          {/* <!-- param-start:[trimSpace1] | warehouses: [snowflake] --> */}

          <ResponseField name="Trim Space" type="boolean" required>
            When True, removes whitespace from fields. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[trimSpace1] --> */}

          {/* <!-- param-start:[fieldOptionallyEnclosed] | warehouses: [snowflake] --> */}

          <ResponseField name="Field Optionally Enclosed" type="string">
            Specify a character used to enclose strings. The value can be NONE, single quote character `'`, or double quote character `"`. To use the single quote character, use the octal or hex representation `0x27` or the double single-quoted escape `''`. Default is NONE.

            When a field contains one of these characters, escape the field using the same character. For example, to escape a string like this: 1 "2" 3, use double quotation to escape, like this: 1 ""2"" 3.
          </ResponseField>

          {/* <!-- param-end:[fieldOptionallyEnclosed] --> */}

          {/* <!-- param-start:[nullIf1] | warehouses: [snowflake] --> */}

          <ResponseField name="Null If" type="editor" required>
            Specify one or more strings (one string per row of the table) to convert to NULL values. When one of these strings is encountered in the file, it is replaced with an SQL NULL value for that field in the loaded table. Click + to add a string.
          </ResponseField>

          {/* <!-- param-end:[nullIf1] --> */}

          {/* <!-- param-start:[errorOnColumnCountMismatch] | warehouses: [snowflake] --> */}

          <ResponseField name="Error On Column Count Mismatch" type="boolean" required>
            When True, generates an error if the number of delimited columns in an input file does not match the number of columns in the corresponding table. When False (default), an error is not generated and the load continues. If the file is successfully loaded in this case:

            * Where the input file contains records with more fields than columns in the table, the matching fields are loaded in order of occurrence in the file, and the remaining fields are not loaded.
            * Where the input file contains records with fewer fields than columns in the table, the non-matching columns in the table are loaded with NULL values.
          </ResponseField>

          {/* <!-- param-end:[errorOnColumnCountMismatch] --> */}

          {/* <!-- param-start:[emptyFieldAsNull] | warehouses: [snowflake] --> */}

          <ResponseField name="Empty Field As Null" type="boolean" required>
            When True, inserts NULL values for empty fields in an input file. This is the default setting.
          </ResponseField>

          {/* <!-- param-end:[emptyFieldAsNull] --> */}

          {/* <!-- param-start:[replaceInvalidCharacters] | warehouses: [snowflake] --> */}

          <ResponseField name="Replace Invalid Characters" type="boolean" required>
            Snowflake replaces invalid UTF-8 characters with the Unicode replacement character. When False (default), the load operation produces an error when invalid UTF-8 character encoding is detected.
          </ResponseField>

          {/* <!-- param-end:[replaceInvalidCharacters] --> */}

          {/* <!-- param-start:[encodingType] | warehouses: [snowflake] --> */}

          <ResponseField name="Encoding Type" type="drop-down" required>
            Select the string that specifies the character set of the source data when loading data into a table. Refer to the Snowflake documentation for more information.
          </ResponseField>

          {/* <!-- param-end:[encodingType] --> */}
        </Tab>

        <Tab title="JSON">
          {/* <!-- param-start:[compression] | warehouses: [snowflake] --> */}

          <ResponseField name="Compression" type="drop-down" required>
            Select the compression method if you wish to compress your data. If you do not wish to compress at all, select NONE. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[compression] --> */}

          {/* <!-- param-start:[enableOctal] | warehouses: [snowflake] --> */}

          <ResponseField name="Enable Octal" type="boolean" required>
            When True, enables the parsing of octal values. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[enableOctal] --> */}

          {/* <!-- param-start:[allowDuplicates] | warehouses: [snowflake] --> */}

          <ResponseField name="Allow Duplicates" type="boolean" required>
            When True, allows duplicate object field names. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[allowDuplicates] --> */}

          {/* <!-- param-start:[stripOuterArray] | warehouses: [snowflake] --> */}

          <ResponseField name="Strip Outer Array" type="boolean" required>
            When True, instructs the JSON parser to remove outer brackets. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[stripOuterArray] --> */}

          {/* <!-- param-start:[stripNullValues] | warehouses: [snowflake] --> */}

          <ResponseField name="Strip Null Values" type="boolean" required>
            When True, instructs the JSON parser to remove any object fields or array elements containing NULL values. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[stripNullValues] --> */}

          {/* <!-- param-start:[ignoreUtf8Errors] | warehouses: [snowflake] --> */}

          <ResponseField name="Ignore UTF8 Errors" type="boolean" required>
            When True, replaces any invalid UTF-8 sequences with Unicode characters. When False (default), UTF-8 errors will not produce an error in the pipeline run.
          </ResponseField>

          {/* <!-- param-end:[ignoreUtf8Errors] --> */}

          {/* <!-- param-start:[nullIf2] | warehouses: [snowflake] --> */}

          <ResponseField name="Null If" type="editor" required>
            Specify one or more strings (one string per row of the table) to convert to NULL values. When one of these strings is encountered in the file, it is replaced with an SQL NULL value for that field in the loaded table. Click + to add a string.
          </ResponseField>

          {/* <!-- param-end:[nullIf2] --> */}

          {/* <!-- param-start:[trimSpace2] | warehouses: [snowflake] --> */}

          <ResponseField name="Trim Space" type="boolean" required>
            When True, removes whitespace from fields. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[trimSpace2] --> */}
        </Tab>

        <Tab title="ORC">
          {/* <!-- param-start:[nullIf3] | warehouses: [snowflake] --> */}

          <ResponseField name="Null If" type="editor" required>
            Specify one or more strings (one string per row of the table) to convert to NULL values. When one of these strings is encountered in the file, it is replaced with an SQL NULL value for that field in the loaded table. Click + to add a string.
          </ResponseField>

          {/* <!-- param-end:[nullIf3] --> */}

          {/* <!-- param-start:[trimSpace3] | warehouses: [snowflake] --> */}

          <ResponseField name="Trim Space" type="boolean" required>
            When True, removes whitespace from fields. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[trimSpace3] --> */}
        </Tab>

        <Tab title="PARQUET">
          {/* <!-- param-start:[compression] | warehouses: [snowflake] --> */}

          <ResponseField name="Compression" type="drop-down" required>
            Select the compression method if you wish to compress your data. If you do not wish to compress at all, select NONE. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[compression] --> */}

          {/* <!-- param-start:[nullIf4] | warehouses: [snowflake] --> */}

          <ResponseField name="Null If" type="editor" required>
            Specify one or more strings (one string per row of the table) to convert to NULL values. When one of these strings is encountered in the file, it is replaced with an SQL NULL value for that field in the loaded table. Click + to add a string.
          </ResponseField>

          {/* <!-- param-end:[nullIf4] --> */}

          {/* <!-- param-start:[trimSpace4] | warehouses: [snowflake] --> */}

          <ResponseField name="Trim Space" type="boolean" required>
            When True, removes whitespace from fields. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[trimSpace4] --> */}
        </Tab>

        <Tab title="XML">
          {/* <!-- param-start:[compression] | warehouses: [snowflake] --> */}

          <ResponseField name="Compression" type="drop-down" required>
            Select the compression method if you wish to compress your data. If you do not wish to compress at all, select NONE. The default setting is AUTO.
          </ResponseField>

          {/* <!-- param-end:[compression] --> */}

          {/* <!-- param-start:[ignoreUtf8Errors1] | warehouses: [snowflake] --> */}

          <ResponseField name="Ignore UTF8 Errors" type="boolean" required>
            When True, replaces any invalid UTF-8 sequences with Unicode characters. When False (default), UTF-8 errors will not produce an error in the pipeline run.
          </ResponseField>

          {/* <!-- param-end:[ignoreUtf8Errors1] --> */}

          {/* <!-- param-start:[preserveSpace] | warehouses: [snowflake] --> */}

          <ResponseField name="Preserve Space" type="boolean" required>
            When True, the XML parser preserves leading and trailing spaces in element content. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[preserveSpace] --> */}

          {/* <!-- param-start:[stripOuterElement] | warehouses: [snowflake] --> */}

          <ResponseField name="Strip Outer Element" type="boolean" required>
            When True, the XML parser strips out any outer XML elements, exposing second-level elements as separate documents. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[stripOuterElement] --> */}

          {/* <!-- param-start:[disableSnowflakeData] | warehouses: [snowflake] --> */}

          <ResponseField name="Disable Snowflake Data" type="boolean" required>
            When True, the XML parser will not recognize Snowflake semi-structured data tags. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[disableSnowflakeData] --> */}

          {/* <!-- param-start:[disableAutoConvert] | warehouses: [snowflake] --> */}

          <ResponseField name="Disable Auto Convert" type="boolean" required>
            When True, the XML parser will disable automatic conversion of numeric and Boolean values from text to native representations. Default setting is False.
          </ResponseField>

          {/* <!-- param-end:[disableAutoConvert] --> */}
        </Tab>
      </Tabs>
    </ResponseField>

    {/* <!-- param-end:[fileType] --> */}

    {/* <!-- param-start:[onError] | warehouses: [snowflake] --> */}

    <ResponseField name="On Error" type="drop-down" required>
      Decide how to proceed upon an error.

      * **Abort Statement:** Aborts the load if any error is encountered. This is the default setting.
      * **Continue:** Continue loading the file.
      * **Skip File:** Skip file if any errors are encountered in the file.
      * **Skip File When n Errors:** Skip file when the number of errors in the file is equal to or greater than the specified number in the next property, n.
      * **Skip File When n% Errors:** Skip file when the percentage of errors in the file exceeds the specified percentage of n.
    </ResponseField>

    {/* <!-- param-end:[onError] --> */}

    {/* <!-- param-start:[n] | warehouses: [snowflake] --> */}

    <ResponseField name="n" type="integer" required>
      Specify the number of errors or the percentage of errors required to skip the file. This parameter only accepts integer characters. `%` is not accepted. Specify percentages as a number only.
    </ResponseField>

    {/* <!-- param-end:[n] --> */}

    {/* <!-- param-start:[sizeLimitB] | warehouses: [snowflake] --> */}

    <ResponseField name="Size Limit (B)" type="integer" required>
      Specify the maximum size, in bytes, of data to be loaded for a given COPY statement. If the maximum is exceeded, the COPY operation discontinues loading files. For more information, refer to the Snowflake documentation.
    </ResponseField>

    {/* <!-- param-end:[sizeLimitB] --> */}

    {/* <!-- param-start:[purgeFiles] | warehouses: [snowflake] --> */}

    <ResponseField name="Purge Files" type="boolean" required>
      When True, purges data files after the data is successfully loaded. Default setting is False.
    </ResponseField>

    {/* <!-- param-end:[purgeFiles] --> */}

    {/* <!-- param-start:[matchByColumnName] | warehouses: [snowflake] --> */}

    <ResponseField name="Match By Column Name" type="drop-down" required>
      Specify whether to load semi-structured data into columns in the target table that match corresponding columns represented in the data.

      * **Case Insensitive:** Load semi-structured data into columns in the target table that match corresponding columns represented in the data. Column names should be case-insensitive.
      * **Case Sensitive:** Load semi-structured data into columns in the target table that match corresponding columns represented in the data. Column names should be case-sensitive.
      * **None:** The COPY operation loads the semi-structured data into a variant column or, if a query is included in the COPY statement, transforms the data.
    </ResponseField>

    {/* <!-- param-end:[matchByColumnName] --> */}

    {/* <!-- param-start:[truncateColumns] | warehouses: [snowflake] --> */}

    <ResponseField name="Truncate Columns" type="boolean" required>
      When True, strings are automatically truncated to the target column length. When False (default), the COPY statement produces an error if a loaded string exceeds the target column length.
    </ResponseField>

    {/* <!-- param-end:[truncateColumns] --> */}

    {/* <!-- param-start:[forceLoad] | warehouses: [snowflake] --> */}

    <ResponseField name="Force Load" type="boolean" required>
      When True, loads all files, regardless of whether they have been loaded previously and haven't changed since they were loaded. Default setting is False.

      When set to True, this option reloads files and can lead to duplicated data in a table.
    </ResponseField>

    {/* <!-- param-end:[forceLoad] --> */}

    {/* <!-- param-start:[metadataFields] | warehouses: [snowflake] --> */}

    <ResponseField name="Metadata Fields" type="dual listbox" required>
      Snowflake metadata columns available to include in the load. For more information, read [Querying Metadata for Staged Files](https://docs.snowflake.com/en/user-guide/querying-metadata.html). This property is only available when an external stage is selected.
    </ResponseField>

    {/* <!-- param-end:[metadataFields] --> */}
  </Tab>

  <Tab title="Databricks">
    {/* <!-- param-start:[location] | warehouses: [databricks] --> */}

    <ResponseField name="Location" type="file explorer" required>
      To retrieve the intended files, use the file explorer to enter the container path where the S3 bucket is located, or select from the list of S3 buckets.

      This must have the format `S3://<bucket>/<path>`.
    </ResponseField>

    {/* <!-- param-end:[location] --> */}

    {/* <!-- param-start:[pattern] | warehouses: [databricks] --> */}

    <ResponseField name="Pattern" type="string" required>
      Files in the specified **Location** will only be loaded if their names match the pattern you specify here. You can use wildcards in the pattern. Enter `.*` to match all files in the location. This property is mandatory.
    </ResponseField>

    {/* <!-- param-end:[pattern] --> */}

    {/* <!-- param-start:[catalog] | warehouses: [databricks] --> */}

    <ResponseField name="Catalog" type="drop-down" required>
      Select a [Databricks Unity Catalog](https://docs.databricks.com/en/data-governance/unity-catalog/index.html). The special value `[Environment Default]` uses the catalog defined in the environment. Selecting a catalog will determine which databases are available in the next parameter.
    </ResponseField>

    {/* <!-- param-end:[catalog] --> */}

    {/* <!-- param-start:[schema] | warehouses: [databricks] --> */}

    <ResponseField name="Schema (Database)" type="drop-down" required>
      The Databricks schema. The special value `[Environment Default]` uses the schema defined in the environment. Read [Create and manage schemas](https://docs.databricks.com/en/data-governance/unity-catalog/create-schemas.html) to learn more.
    </ResponseField>

    {/* <!-- param-end:[schema] --> */}

    {/* <!-- param-start:[targetTable,table] | warehouses: [databricks] --> */}

    <ResponseField name="Table" type="string" required>
      Select an existing table to load data into. The tables available for selection depend on the chosen schema (database).
    </ResponseField>

    {/* <!-- param-end:[targetTable,table] --> */}

    {/* <!-- param-start:[loadColumns] | warehouses: [databricks] --> */}

    <ResponseField name="Load Columns" type="dual listbox">
      Choose the columns to load. If you leave this parameter empty, all columns will be loaded.
    </ResponseField>

    {/* <!-- param-end:[loadColumns] --> */}

    {/* <!-- param-start:[fileType] | warehouses: [databricks] --> */}

    <ResponseField name="File Type" type="drop-down" required>
      Select the type of data to load. Available data types are: CSV, JSON, PARQUET, and AVRO.

      Component properties will change to reflect the selected file type. Click one of the tabs below for properties applicable to that file type.

      <Tabs>
        <Tab title="CSV">
          {/* <!-- param-start:[header] | warehouses: [databricks] --> */}

          <ResponseField name="Header" type="boolean" required>
            Select **Yes** to use the first line of the file as column names. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[header] --> */}

          {/* <!-- param-start:[fieldDelimiter] | warehouses: [databricks] --> */}

          <ResponseField name="Field Delimiter" type="string" required>
            Enter the delimiter character used to separate fields in the CSV file. This can be one or more single-byte or multibyte characters that separate fields in an input file. If none is specified, the default is a comma.

            Accepted characters include common escape sequences, octal values (prefixed by `\\`), or hex values (prefixed by `0x`). This delimiter is limited to a maximum of 20 characters. The specified delimiter must be a valid UTF-8 character and not a random sequence of bytes.
          </ResponseField>

          {/* <!-- param-end:[fieldDelimiter] --> */}

          {/* <!-- param-start:[dateFormat] | warehouses: [databricks] --> */}

          <ResponseField name="Date Format" type="string" required>
            Describe the format of date values in the data files to be loaded. If none is specified, the default is `yyyy-MM-dd`.
          </ResponseField>

          {/* <!-- param-end:[dateFormat] --> */}

          {/* <!-- param-start:[timestampFormat] | warehouses: [databricks] --> */}

          <ResponseField name="Timestamp Format" type="string" required>
            Describe the format of timestamp values in the data files to be loaded. If none is specified, the default is `yyyy-MM-dd'T'HH:mm:ss.[SSS][XXX]`.
          </ResponseField>

          {/* <!-- param-end:[timestampFormat] --> */}

          {/* <!-- param-start:[encodingType] | warehouses: [databricks] --> */}

          <ResponseField name="Encoding Type" type="string" required>
            The encoding type to use when decoding the input files. If none is specified, the default is UTF-8.
          </ResponseField>

          {/* <!-- param-end:[encodingType] --> */}

          {/* <!-- param-start:[ignoreLeadingWhitespace] | warehouses: [databricks] --> */}

          <ResponseField name="Ignore Leading Whitespace" type="boolean" required>
            When **Yes**, removes whitespace from the beginning of fields. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[ignoreLeadingWhitespace] --> */}

          {/* <!-- param-start:[ignoreTrailingWhitespace] | warehouses: [databricks] --> */}

          <ResponseField name="Ignore Trailing Whitespace" type="boolean" required>
            When **Yes**, removes whitespace from the end of fields. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[ignoreTrailingWhitespace] --> */}

          {/* <!-- param-start:[inferSchema] | warehouses: [databricks] --> */}

          <ResponseField name="Infer Schema" type="boolean" required>
            If **Yes**, will attempt to determine the input schema automatically from the data contained in the CSV file. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[inferSchema] --> */}

          {/* <!-- param-start:[multiLine] | warehouses: [databricks] --> */}

          <ResponseField name="Multi Line" type="boolean" required>
            If **Yes**, will parse records which may span multiple lines. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[multiLine] --> */}

          {/* <!-- param-start:[nullValue] | warehouses: [databricks] --> */}

          <ResponseField name="Null Value" type="string" required>
            Sets the string representation of a null value. If not specified, the default value is an empty string.
          </ResponseField>

          {/* <!-- param-end:[nullValue] --> */}

          {/* <!-- param-start:[emptyValue] | warehouses: [databricks] --> */}

          <ResponseField name="Empty Value" type="string" required>
            Sets the string representation of an empty value. If not specified, the default value is an empty string.
          </ResponseField>

          {/* <!-- param-end:[emptyValue] --> */}

          {/* <!-- param-start:[recursiveFileLookup] | warehouses: [databricks] --> */}

          <ResponseField name="Recursive File Lookup" type="boolean" required>
            If **Yes**, partition inference is disabled. If not specified, the default is **No**. To control which files are loaded, use the **Pattern** property instead.
          </ResponseField>

          {/* <!-- param-end:[recursiveFileLookup] --> */}

          {/* <!-- param-start:[forceLoad] | warehouses: [databricks] --> */}

          <ResponseField name="Force Load" type="boolean" required>
            If **Yes**, files are loaded regardless of whether they've been loaded before. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[forceLoad] --> */}
        </Tab>

        <Tab title="JSON">
          {/* <!-- param-start:[dateFormat] | warehouses: [databricks] --> */}

          <ResponseField name="Date Format" type="string" required>
            Describe the format of date values in the data files to be loaded. If none is specified, the default is `yyyy-MM-dd`.
          </ResponseField>

          {/* <!-- param-end:[dateFormat] --> */}

          {/* <!-- param-start:[timestampFormat] | warehouses: [databricks] --> */}

          <ResponseField name="Timestamp Format" type="string" required>
            Describe the format of timestamp values in the data files to be loaded. If none is specified, the default is `yyyy-MM-dd'T'HH:mm:ss.[SSS][XXX]`.
          </ResponseField>

          {/* <!-- param-end:[timestampFormat] --> */}

          {/* <!-- param-start:[encodingType] | warehouses: [databricks] --> */}

          <ResponseField name="Encoding Type" type="string" required>
            The encoding type to use when decoding the input files. If none is specified, the default is UTF-8.
          </ResponseField>

          {/* <!-- param-end:[encodingType] --> */}

          {/* <!-- param-start:[multiLine1] | warehouses: [databricks] --> */}

          <ResponseField name="Multi Line" type="boolean" required>
            If **Yes**, will parse records which may span multiple lines. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[multiLine1] --> */}

          {/* <!-- param-start:[primitivesAsString] | warehouses: [databricks] --> */}

          <ResponseField name="Primitives As String" type="boolean" required>
            If **Yes**, primitive data types are interpreted as strings in JSON files. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[primitivesAsString] --> */}

          {/* <!-- param-start:[prefersDecimals, prefersDecimal] | warehouses: [databricks] --> */}

          <ResponseField name="Prefers Decimals" type="boolean" required>
            If **Yes**, all floating-point values will be treated as a decimal type in JSON files. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[prefersDecimals, prefersDecimal] --> */}

          {/* <!-- param-start:[allowComments] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Comments" type="boolean" required>
            If **Yes**, will allow JAVA/C++ comments in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowComments] --> */}

          {/* <!-- param-start:[allowUnquotedFieldNames] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Unquoted Field Names" type="boolean" required>
            If **Yes**, will allow unquoted JSON field names. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowUnquotedFieldNames] --> */}

          {/* <!-- param-start:[allowSingleQuotes] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Single Quotes" type="boolean" required>
            If **Yes**, will allow single quotes in addition to double quotes in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowSingleQuotes] --> */}

          {/* <!-- param-start:[allowNumericLeadingZeros] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Numeric Leading Zeros" type="boolean" required>
            If **Yes**, will allow leading zeros in numbers in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowNumericLeadingZeros] --> */}

          {/* <!-- param-start:[allowBackslashEscapingAnyCharacter] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Backslash Escaping Any Character" type="boolean" required>
            If **Yes**, will allow the quoting of all characters using the backslash quoting mechanism in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowBackslashEscapingAnyCharacter] --> */}

          {/* <!-- param-start:[allowUnquotedControlChars] | warehouses: [databricks] --> */}

          <ResponseField name="Allow Unquoted Control Chars" type="boolean" required>
            If **Yes**, will allow JSON strings to include unquoted control characters in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[allowUnquotedControlChars] --> */}

          {/* <!-- param-start:[dropFieldIfAllNull] | warehouses: [databricks] --> */}

          <ResponseField name="Drop Field If All Null" type="boolean" required>
            If **Yes**, will ignore columns that contain only null values in JSON records. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[dropFieldIfAllNull] --> */}

          {/* <!-- param-start:[recursiveFileLookup] | warehouses: [databricks] --> */}

          <ResponseField name="Recursive File Lookup" type="boolean" required>
            If **Yes**, partition inference is disabled. If not specified, the default is **No**. To control which files are loaded, use the **Pattern** property instead.
          </ResponseField>

          {/* <!-- param-end:[recursiveFileLookup] --> */}

          {/* <!-- param-start:[forceLoad1] | warehouses: [databricks] --> */}

          <ResponseField name="Force Load" type="boolean" required>
            If **Yes**, files are loaded regardless of whether they've been loaded before. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[forceLoad1] --> */}
        </Tab>

        <Tab title="PARQUET">
          {/* <!-- param-start:[mergeSchema] | warehouses: [databricks] --> */}

          <ResponseField name="Merge Schema" type="boolean" required>
            If **Yes**, will merge schema from all PARQUET and AVRO part-files. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[mergeSchema] --> */}

          {/* <!-- param-start:[recursiveFileLookup] | warehouses: [databricks] --> */}

          <ResponseField name="Recursive File Lookup" type="boolean" required>
            If **Yes**, partition inference is disabled. If not specified, the default is **No**. To control which files are loaded, use the **Pattern** property instead.
          </ResponseField>

          {/* <!-- param-end:[recursiveFileLookup] --> */}

          {/* <!-- param-start:[forceLoad2] | warehouses: [databricks] --> */}

          <ResponseField name="Force Load" type="boolean" required>
            If **Yes**, files are loaded regardless of whether they've been loaded before. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[forceLoad2] --> */}
        </Tab>

        <Tab title="AVRO">
          {/* <!-- param-start:[mergeSchema1] | warehouses: [databricks] --> */}

          <ResponseField name="Merge Schema" type="boolean" required>
            If **Yes**, will merge schema from all PARQUET and AVRO part-files. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[mergeSchema1] --> */}

          {/* <!-- param-start:[recursiveFileLookup1] | warehouses: [databricks] --> */}

          <ResponseField name="Recursive File Lookup" type="boolean" required>
            If **Yes**, partition inference is disabled. If not specified, the default is **No**. To control which files are loaded, use the **Pattern** property instead.
          </ResponseField>

          {/* <!-- param-end:[recursiveFileLookup1] --> */}

          {/* <!-- param-start:[forceLoad3] | warehouses: [databricks] --> */}

          <ResponseField name="Force Load" type="boolean" required>
            If **Yes**, files are loaded regardless of whether they've been loaded before. If not specified, the default is **No**.
          </ResponseField>

          {/* <!-- param-end:[forceLoad3] --> */}
        </Tab>
      </Tabs>
    </ResponseField>

    {/* <!-- param-end:[fileType] --> */}
  </Tab>

  <Tab title="Amazon Redshift">
    {/* <!-- param-start:[schema] | warehouses: [redshift] --> */}

    <ResponseField name="Schema" type="drop-down" required>
      Select the table schema. The special value `[Environment Default]` uses the schema defined in the environment. For more information on using multiple schemas, read [Schemas](https://docs.aws.amazon.com/redshift/latest/dg/r_Schemas_and_tables.html).
    </ResponseField>

    {/* <!-- param-end:[schema] --> */}

    {/* <!-- param-start:[targetTableName] | warehouses: [redshift] --> */}

    <ResponseField name="Target Table Name" type="drop-down" required>
      Select an existing table to load data into. The selected target table is where data will be loaded into from the S3 object.

      When selecting a target table with the Redshift **SUPER** data type, the S3 Load component will only offer the following data file types: **JSON**, **ORC**, and **PARQUET**. However, if you select to include columns of data file types other than **SUPER** in the **Load Columns** property, all data file types will be available for selection.
    </ResponseField>

    {/* <!-- param-end:[targetTableName] --> */}

    {/* <!-- param-start:[loadColumns] | warehouses: [redshift] --> */}

    <ResponseField name="Load Columns" type="dual listbox" required>
      Select which of the target table's columns should be loaded.
    </ResponseField>

    {/* <!-- param-end:[loadColumns] --> */}

    {/* <!-- param-start:[s3UrlLocation] | warehouses: [redshift] --> */}

    <ResponseField name="S3 URL Location" type="file explorer" required>
      To retrieve the intended files, use the file explorer to enter the container path where the S3 bucket is located, or select from the list of S3 buckets.

      This must have the format `S3://<bucket>/<path>`.
    </ResponseField>

    {/* <!-- param-end:[s3UrlLocation] --> */}

    {/* <!-- param-start:[s3ObjectPrefix] | warehouses: [redshift] --> */}

    <ResponseField name="S3 Object Prefix" type="string" required>
      All files that begin with this prefix will be included in the load into the target table.
    </ResponseField>

    {/* <!-- param-end:[s3ObjectPrefix] --> */}

    {/* <!-- param-start:[iamRoleArn] | warehouses: [redshift] --> */}

    <ResponseField name="IAM Role ARN" type="drop-down" required>
      Select an IAM role [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference-arns.html) that is already attached to your Redshift cluster, and that has the necessary permissions to access S3.

      This setting is optional, since without this style of setup, the credentials of the environment will be used.

      Read the [Redshift documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/authorizing-redshift-service.html) for more information about using a role ARN with Redshift.
    </ResponseField>

    {/* <!-- param-end:[iamRoleArn] --> */}

    {/* <!-- param-start:[dataFileType, avroLayout, csvQuoter] | warehouses: [redshift] --> */}

    <ResponseField name="Data File Type" type="drop-down" required>
      The type of expected data to load. Some may require additional formatting, explained in the [Amazon documentation](https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html).

      Available options are: **Avro**, **CSV**, **Delimited**, **Fixed Width**, **JSON**, **ORC**, **PARQUET**.

      Component properties will change to reflect the choice made here and give options based on the specific data file type.
    </ResponseField>

    {/* <!-- param-end:[dataFileType, avroLayout, csvQuoter] --> */}

    {/* <!-- param-start:[fixedWidthSpec] | warehouses: [redshift] --> */}

    <ResponseField name="Fixed Width Spec" type="string" required>
      Fixed width only. Loads the data from a file where each column width is a fixed length, rather than columns being separated by a delimiter. For more information, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-fixedwidth).

      To use variables in this field, type the name of the variable prefixed by the dollar symbol and surrounded by \{ } brackets, as follows: `${variable}`. Once you type `${`, a drop-down list of autocompleted suggested variables will appear. This list updates as you type; for example, if you type `${date`, functions and variables containing `date` will be listed.
    </ResponseField>

    {/* <!-- param-end:[fixedWidthSpec] --> */}

    {/* <!-- param-start:[jsonLayout] | warehouses: [redshift] --> */}

    <ResponseField name="JSON Layout" type="string" required>
      (JSON only) Defaults to "auto", which will work for the majority of JSON files if the fields match the table field names. More information about JSON file types can be found in the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-json).
    </ResponseField>

    {/* <!-- param-end:[jsonLayout] --> */}

    {/* <!-- param-start:[delimiter,fieldDelimiter] | warehouses: [redshift] --> */}

    <ResponseField name="Delimiter" type="string" required>
      (CSV only) Specify a delimiting character to separate columns. The default character is a comma: ,

      A \[TAB] character can be specified using "\ ".
    </ResponseField>

    {/* <!-- param-end:[delimiter,fieldDelimiter] --> */}

    {/* <!-- param-start:[fieldOptionallyEnclosed] | warehouses: [redshift] --> */}

    <ResponseField name="CSV Quoter" type="string" required>
      (CSV only) Specify the character to be used as the quote character when using the CSV file type.
    </ResponseField>

    {/* <!-- param-end:[fieldOptionallyEnclosed] --> */}

    {/* <!-- param-start:[explicitIds] | warehouses: [redshift] --> */}

    <ResponseField name="Explicit IDs" type="drop-down" required>
      Select whether to load data from the S3 objects into an **IDENTITY** column. Read the [Redshift documentation](https://docs.aws.amazon.com/redshift/latest/dg/r_COPY_command_examples.html#r_COPY_command_examples-load-venue-with-explicit-values-for-an-identity-column) for more information.
    </ResponseField>

    {/* <!-- param-end:[explicitIds] --> */}

    {/* <!-- param-start:[s3BucketRegion] | warehouses: [redshift] --> */}

    <ResponseField name="S3 Bucket Region" type="drop-down" required>
      Select the Amazon S3 region that hosts the selected S3 bucket. This setting is optional and can be left as "None" if the bucket is in the same region as your Redshift cluster.
    </ResponseField>

    {/* <!-- param-end:[s3BucketRegion] --> */}

    {/* <!-- param-start:[compression, compressionMethod] | warehouses: [redshift] --> */}

    <ResponseField name="Compression Method" type="drop-down" required>
      Specify whether the input is compressed in any of the following formats: **BZip2**, **gzip**, **LZop**, **Zstd**, or not at all (None).
    </ResponseField>

    {/* <!-- param-end:[compression, compressionMethod] --> */}

    {/* <!-- param-start:[encoding,encodingType] | warehouses: [redshift] --> */}

    <ResponseField name="Encoding" type="drop-down" required>
      Select the encoding format the data is in. The default for this setting is **UTF-8**.
    </ResponseField>

    {/* <!-- param-end:[encoding,encodingType] --> */}

    {/* <!-- param-start:[replaceInvalidCharacters] | warehouses: [redshift] --> */}

    <ResponseField name="Replace Invalid Characters" type="string" required>
      Specify a single character that will replace any invalid unicode characters in the data. The default is: **?**.
    </ResponseField>

    {/* <!-- param-end:[replaceInvalidCharacters] --> */}

    {/* <!-- param-start:[removeQuotes] | warehouses: [redshift] --> */}

    <ResponseField name="Remove Quotes" type="drop-down" required>
      (Delimited, Fixed Width only) When "Yes", removes surrounding quotation marks from strings in the incoming data. All characters within the quotation marks, including delimiters, are retained.

      If a string has a beginning single or double quotation mark but no corresponding ending mark, the `COPY` command fails to load that row and returns an error.

      The default setting is "No".

      For more information, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-conversion.html#copy-removequotes).
    </ResponseField>

    {/* <!-- param-end:[removeQuotes] --> */}

    {/* <!-- param-start:[maximumErrors] | warehouses: [redshift] --> */}

    <ResponseField name="Maximum Errors" type="integer" required>
      The maximum number of individual parsing errors permitted before the load will fail. If the number of errors reaches this limit, the load fails; errors below the limit are skipped and logged. The Amazon default is 1000.
    </ResponseField>

    {/* <!-- param-end:[maximumErrors] --> */}

    {/* <!-- param-start:[dateFormat] | warehouses: [redshift] --> */}

    <ResponseField name="Date Format" type="string" required>
      Defaults to 'auto'. Users can specify their preferred date format manually if they wish. For more information on date formats, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/r_DATEFORMAT_and_TIMEFORMAT_strings.html).
    </ResponseField>

    {/* <!-- param-end:[dateFormat] --> */}

    {/* <!-- param-start:[timeFormat] | warehouses: [redshift] --> */}

    <ResponseField name="Time Format" type="string" required>
      Defaults to 'auto'. Users can specify their preferred time format manually if they wish. For more information on time formats, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/r_DATEFORMAT_and_TIMEFORMAT_strings.html).
    </ResponseField>

    {/* <!-- param-end:[timeFormat] --> */}

    {/* <!-- param-start:[ignoreHeaderRows] | warehouses: [redshift] --> */}

    <ResponseField name="Ignore Header Rows" type="integer" required>
      Specify the number of rows at the top of the file to ignore. The default is `0` (no rows ignored).
    </ResponseField>

    {/* <!-- param-end:[ignoreHeaderRows] --> */}

    {/* <!-- param-start:[acceptAnyDate] | warehouses: [redshift] --> */}

    <ResponseField name="Accept Any Date" type="drop-down" required>
      * **Yes:** Invalid dates such as '45-65-2020' are not considered an error, but will be loaded as NULL values. This is the default setting.
      * **No:** Invalid dates will return an error.
    </ResponseField>

    {/* <!-- param-end:[acceptAnyDate] --> */}

    {/* <!-- param-start:[ignoreBlankLines] | warehouses: [redshift] --> */}

    <ResponseField name="Ignore Blank Lines" type="drop-down" required>
      (CSV only) When "Yes", ignores blank lines that only contain a line feed in a data file and does not try to load them. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[ignoreBlankLines] --> */}

    {/* <!-- param-start:[truncateColumns] | warehouses: [redshift] --> */}

    <ResponseField name="Truncate Columns" type="drop-down" required>
      * **Yes:** Any instance of data in the input file that is too long to fit into the specified target column width will be truncated to fit, instead of causing an error.
      * **No:** Any data in the input file that is too long to fit into the specified target column width will cause an error. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[truncateColumns] --> */}

    {/* <!-- param-start:[fillRecord] | warehouses: [redshift] --> */}

    <ResponseField name="Fill Record" type="drop-down" required>
      When "Yes", allows data files to be loaded when contiguous columns are missing at the end of some records. The missing columns are filled with either zero-length strings or NULLs, as appropriate for the data types of the columns in question. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[fillRecord] --> */}

    {/* <!-- param-start:[trimBlanks] | warehouses: [redshift] --> */}

    <ResponseField name="Trim Blanks" type="drop-down" required>
      * **Yes:** Removes the trailing white space characters from a VARCHAR string. This property only applies to columns with a VARCHAR data type.
      * **No:** Does not remove trailing white space characters from the input data.
    </ResponseField>

    {/* <!-- param-end:[trimBlanks] --> */}

    {/* <!-- param-start:[nullAs] | warehouses: [redshift] --> */}

    <ResponseField name="NULL As" type="string" required>
      Loads fields that match the specified NULL string. The default is `\\N` with an additional `\` at the start to escape. Case-sensitive.

      For more information, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-conversion.html#copy-null-as).
    </ResponseField>

    {/* <!-- param-end:[nullAs] --> */}

    {/* <!-- param-start:[emptyFieldAsNull, emptyAsNull] | warehouses: [redshift] --> */}

    <ResponseField name="Empty As Null" type="drop-down" required>
      When "Yes", empty columns in the input file will become NULL values. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[emptyFieldAsNull, emptyAsNull] --> */}

    {/* <!-- param-start:[blanksAsNull] | warehouses: [redshift] --> */}

    <ResponseField name="Blanks As Null" type="drop-down" required>
      When "Yes", loads blank columns, which consist of only whitespace characters, as NULL. This is the default setting.

      This option applies only to CHAR and VARCHAR columns. Blank fields for other data types, such as INT, are always loaded with NULL. For example, a string that contains three space characters in succession (and no other characters) is loaded as a NULL. The default behavior, without this option, is to load the space characters as is.
    </ResponseField>

    {/* <!-- param-end:[blanksAsNull] --> */}

    {/* <!-- param-start:[compUpdate] | warehouses: [redshift] --> */}

    <ResponseField name="Comp Update" type="drop-down" required>
      When "On", compression encodings are automatically applied during a COPY command. This is the default setting.

      This is usually a good idea to optimize the compression used when storing the data.
    </ResponseField>

    {/* <!-- param-end:[compUpdate] --> */}

    {/* <!-- param-start:[statUpdate] | warehouses: [redshift] --> */}

    <ResponseField name="Stat Update" type="drop-down" required>
      When "On", governs automatic computation and refreshing of optimizer statistics at the end of a successful COPY command. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[statUpdate] --> */}

    {/* <!-- param-start:[escape, escaped] | warehouses: [redshift] --> */}

    <ResponseField name="Escape" type="drop-down" required>
      (Delimited only) When "Yes", the backslash character `\` in input data is treated as an escape character. The character that immediately follows the backslash character is loaded into the table as part of the current column value, even if it is a character that normally serves a special purpose. For example, you can use this parameter to escape the delimiter character, a quotation mark, an embedded newline character, or the escape character itself when any of these characters is a legitimate part of a column value.

      The default setting is "No".
    </ResponseField>

    {/* <!-- param-end:[escape, escaped] --> */}

    {/* <!-- param-start:[roundDecimals] | warehouses: [redshift] --> */}

    <ResponseField name="Round Decimals" type="drop-down" required>
      When "Yes", any decimals are rounded to fit into the column in any instance where the number of decimal places in the input is larger than defined for the target column. This is the default setting.
    </ResponseField>

    {/* <!-- param-end:[roundDecimals] --> */}

    {/* <!-- param-start:[manifest] | warehouses: [redshift] --> */}

    <ResponseField name="Manifest" type="drop-down" required>
      When "Yes", the given object prefix is that of a manifest file. The default setting is "No".

      For more information, read the [AWS documentation](https://docs.aws.amazon.com/redshift/latest/dg/loading-data-files-using-manifest.html).
    </ResponseField>

    {/* <!-- param-end:[manifest] --> */}
  </Tab>
</Tabs>

## S3 access

To access an S3 bucket from a different AWS account, the following is required:

* Set up cross-account access via AWS roles.
* The user must type in the bucket they want to access or use a variable to load/unload to those structures.

***

## File patterns with Snowflake

In Snowflake, the **Pattern** parameter in the COPY INTO syntax is a pattern on the complete path of the file and is not just relative to the directory configured in the **S3 Object Prefix** parameter.

The table below provides an example of S3 Object Prefix and Pattern behaviors, including success and failure states.

| S3Object Prefix                | Pattern                                | Outcome | Comments                                                                   |
| ------------------------------ | -------------------------------------- | ------- | -------------------------------------------------------------------------- |
| s3://testbucket/               | testDirectory/alphabet\_0\_0\_0.csv.gz | Success | This is the format that the S3 Load Generator will generate.               |
| s3://testbucket/testDirectory/ | testDirectory/alphabet\_0\_0\_0.csv.gz | Success | Loads the file successfully because the pattern is matching the full path. |
| s3://testbucket/testDirectory/ | .\*.csv.gz                             | Success | Would load all files ending in .csv.gz in the testDirectory directory.     |
| s3://testbucket/testDirectory/ | alphabet\_0\_0\_0.csv.gz               | Failure | Does not load the file because the pattern does not match.                 |