Amazon Component Release Notes

This page lists the main features added to the Amazon component.

Feature Highlights

Version 2.1.0

Folder creation with the tool AWS S3

The existing S3 TOOL allows to retrieve a remote file which is on S3 and store it locally, through the operation "getfile" which will download it locally in the local path defined.

When the defined path on the parameter "Local File Path" doesn’t exist then the tool AWS S3 will now automatically create the corresponding folder.

Before this change, an error was thrown if the folder did not exist.

Addition of a sample project

A sample project has been added to help for Amazon Component usage.

The new sample project is shipped with the Component, so you simply have to install the new version and you’ll have at your disposal the sample project.

Check this article to learn how to use the sample projects shipped inside a Component.

Snowflake External Storage Attribute

Snowflake Component allows to use Amazon S3 as storage location.

Amazon Component has been updated to automatically provide the availables S3 nodes of the workspace in the Snowflake Metadata.

Below is an example of the Snowflake Metadata with the list of available nodes:

release notes amazon s3 in snowflake metadata

This requires to use also the corresponding Snowflake Component which has been updated according for this list.

Version 2.0.3

Minor improvements and fixed issues

This version contains some minor improvements and fixed issues, which can be found in the complete changelog.

Version 2.0.2

This version contains some issues which can be found on the full changelog.

Version 2.0.1

LOAD Rdbms to Redshift (with copy)

Allow using stringDelimiter and fieldDelimiter at the same time.

Amazon S3 with Metadata Configuration

To load data from your local databases, the data need to be exported in a file and this file is copied in a Amazon S3 bucket.

Metadata Configurations on the Amazon S3 bucket name were ignored.

Change Data Capture (CDC)

Multiple improvements have been performed to homogenize the usage of Change Data Capture (CDC) in the various Components.

Parameters have been homogenized, so that all Templates should now have the same CDC Parameters, with the same support of features.

Multiple fixes have also been performed to correct CDC issues. Refer to the changelog for the exact list of changes.

Change Log

Version 2023.1.16

Bug fixes

  • DI-9788: Updated Joda Time third-party libraries.

Version 2023.1.10

Bug fixes

  • DI-9473: Updated Amazon Ion third-party libraries.

Version 2023.1.9

Bug fixes

  • DI-8957: TOOL AWS - S3: Fix missing element and xPath expression for assume role authentication.

Version 2023.1.8

Bug fixes

  • DI-8961: TOOL AWS - S3: Processes stop on the AssumeRole step when the the Large Files API is used.

Version 2023.1.6

Bug fixes

  • DI-8690: AWS connections close before file transfers are really complete.

Version 2023.1.5

Feature improvements

  • DI-8181: Third-Party library upgrade.

Version 2023.1.2

Feature improvements

  • DI-6824: In Amazon S3 metadata, the Secret Key field is now encrypted and hidden like a password.

Bug fixes

  • DI-7296: The AWS - S3 tool fails to run with an Attempt to resolve method: listObjectsV2() on undefined variable or class name: s3 error.

Version 2023.1.0

Feature improvements

  • DI-1212: The Integration Redshift with Slowly Changing Dimension template is now available. It supports SCD with Redshift.

  • DI-5862: The LOAD Xml to Redshift template is automatically suggested when loading WSDL or HTTP REST data into Redshift.

Bug fixes

  • DI-3292: In the TOOL AWS tool, the backslash (\) character can now be specified in the local file path.

  • DI-6124: Multiple third-party libraries upgrade.

  • DI-6516: Jackson - Third-party library upgrade.

Version 5.3.12 (Component Pack)

Bug fixes

  • DI-8942: ClassNotFound exception in Designer when loading some components.

Version 5.3.8 (Component Pack)

Feature improvements

  • DI-6251: The Path-Style Access option has been added to Amazon s3 metadata and the Action Process Ref TOOL AWS - S3. The option allows users to enable path-style URL access to AWS in all regions.

  • DI-6239: The Disable Certificate Checking option has been added to Amazon s3 metadata and the Action Process Ref TOOL AWS - S3. The option allows users to disable certificate vertification for development and testing purposes.

Version 2.0.1 (Amazon Component)

Feature improvements

  • DI-1910: Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet

  • DI-1909: Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking

Bug fixes

  • DI-986: Tool Amazon S3 - when performing a "getfile" operation, the file copied locally (on the runtime server) remains locked after the end of the session

  • DI-1774: Amazon S3 - Retrieving a configured bucket name from a folder did not take the configuration into account

  • DI-1779: Load Rdbms to Redshift (with COPY) - Allow using stringDelimiter and fieldDelimiter at the same time

  • DI-1887: Redshift Metadata - unsupported datatype "Bytea" is removed from available datatypes and converted to Varchar datatype automatically

  • DI-1908: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates on Lock / Unlock CDC steps

Version 2.0.2 (Amazon Component)

Bug fixes

  • DI-3349: Removing unnecessary libraries

Version 2.0.3 (Amazon Component)

Feature improvements

  • DI-2554: TOOL Amazon S3 - add possibility to get multiple files

  • DI-2723: Template - Load File to Redshift with Copy - add support for positional file

  • DI-2725: TOOL Amazon S3 - new parameter "S3 Base Url"

  • DI-2759: Amazon Redshift technology - add delimiters around Reserved words

  • DI-2776: Template - Load Xml to Redshift (with copy) - new dedicated optimized Template to load data from XML to Redshift

  • DI-2780: Template - Integration Redshift - new parameter "Useless Update Detection Method"

Bug fixes

  • DI-1955: TOOL Amazon S3 - some regions such as EU-NORTH-1 are not working

  • DI-2727: Template - Integration Redshift - Fix "Test If table exists" step to work with variables in temporary object masks

  • DI-2733: Template - LOAD Rdbms to Redshift (with COPY) - generated temporary file names may unexpectedly contain object delimiters

Version 2.1.0 (Amazon Component)

Feature improvements

  • DI-2532: Addition of demonstration project for Amazon Component usage

  • DI-2872: When performing a getFile operation with S3 TOOL, if the local folder defined in the “Local File Path” does not exists then the tool will now automatically create it

  • DI-2854: Microsoft Azure Storage and Amazon S3 Bucket nodes available in the workspace are now proposed automatically in Snowflake Metadata in External Storage attribute

Version 3.0.0 (Component Pack)

Feature improvements

  • DI-4508: Update Components and Designer to take into account dedicated license permissions

  • DI-4727: Rebranding: Templates and sample projects

  • DI-4962: Improved component dependencies and requirements management

Version 3.0.1 (Component Pack)

Bug fixes

  • DI-5212: Redshift Templates - the "Check if Table Exists" action does not work when there are variables in the schema load mask

  • DI-5201: Redshift Templates - parallelism issues when the templates are used mutliple time in one process