Amazon Component Release Notes
This page lists the main features added to the Amazon component.
Feature Highlights
Version 2.1.0
Folder creation with the tool AWS S3
The existing S3 TOOL allows to retrieve a remote file which is on S3 and store it locally, through the operation "getfile" which will download it locally in the local path defined.
When the defined path on the parameter "Local File Path" doesn’t exist then the tool AWS S3 will now automatically create the corresponding folder.
Before this change, an error was thrown if the folder did not exist.
Addition of a sample project
A sample project has been added to help for Amazon Component usage.
The new sample project is shipped with the Component, so you simply have to install the new version and you’ll have at your disposal the sample project.
Check this article to learn how to use the sample projects shipped inside a Component.
Snowflake External Storage Attribute
Snowflake Component allows to use Amazon S3 as storage location.
Amazon Component has been updated to automatically provide the availables S3 nodes of the workspace in the Snowflake Metadata.
Below is an example of the Snowflake Metadata with the list of available nodes:
This requires to use also the corresponding Snowflake Component which has been updated according for this list. |
Version 2.0.1
Amazon S3 with Metadata Configuration
To load data from your local databases, the data need to be exported in a file and this file is copied in a Amazon S3 bucket.
Metadata Configurations on the Amazon S3 bucket name were ignored.
Change Data Capture (CDC)
Multiple improvements have been performed to homogenize the usage of Change Data Capture (CDC) in the various Components.
Parameters have been homogenized, so that all Templates should now have the same CDC Parameters, with the same support of features.
Multiple fixes have also been performed to correct CDC issues. Refer to the changelog for the exact list of changes.
Change Log
Version 5.3.8 (Component Pack)
New Features
-
DI-6251: The Path-Style Access option has been added to Amazon s3 metadata and the Action Process Ref TOOL AWS - S3. The option allows users to enable path-style URL access to AWS in all regions.
-
DI-6239: The Disable Certificate Checking option has been added to Amazon s3 metadata and the Action Process Ref TOOL AWS - S3. The option allows users to disable certificate vertification for development and testing purposes.
Version 2.0.1 (Amazon Component)
New Features
-
DI-1910: Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet
-
DI-1909: Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking
Bug Fixes
-
DI-986: Tool Amazon S3 - when performing a "getfile" operation, the file copied locally (on the runtime server) remains locked after the end of the session
-
DI-1774: Amazon S3 - Retrieving a configured bucket name from a folder did not take the configuration into account
-
DI-1779: Load Rdbms to Redshift (with COPY) - Allow using stringDelimiter and fieldDelimiter at the same time
-
DI-1887: Redshift Metadata - unsupported datatype "Bytea" is removed from available datatypes and converted to Varchar datatype automatically
-
DI-1908: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates on Lock / Unlock CDC steps
Version 2.0.3 (Amazon Component)
New Features
-
DI-2554: TOOL Amazon S3 - add possibility to get multiple files
-
DI-2723: Template - Load File to Redshift with Copy - add support for positional file
-
DI-2725: TOOL Amazon S3 - new parameter "S3 Base Url"
-
DI-2759: Amazon Redshift technology - add delimiters around Reserved words
-
DI-2776: Template - Load Xml to Redshift (with copy) - new dedicated optimized Template to load data from XML to Redshift
-
DI-2780: Template - Integration Redshift - new parameter "Useless Update Detection Method"
Bug Fixes
-
DI-1955: TOOL Amazon S3 - some regions such as EU-NORTH-1 are not working
-
DI-2727: Template - Integration Redshift - Fix "Test If table exists" step to work with variables in temporary object masks
-
DI-2733: Template - LOAD Rdbms to Redshift (with COPY) - generated temporary file names may unexpectedly contain object delimiters
Version 2.1.0 (Amazon Component)
New Features
-
DI-2532: Addition of demonstration project for Amazon Component usage
-
DI-2872: When performing a getFile operation with S3 TOOL, if the local folder defined in the “Local File Path” does not exists then the tool will now automatically create it
-
DI-2854: Microsoft Azure Storage and Amazon S3 Bucket nodes available in the workspace are now proposed automatically in Snowflake Metadata in External Storage attribute