Snowflake component release notes

This page lists the main features added to the Snowflake Component.

Changelog

Version 2024.1.2

Features

  • DI-9413: Added the ability to change the report timeout of Snowpipe operations.

Bug fixes

  • DI-9275: Fixed an issue which stopped columns from being set to boolean type.

Version 2024.1.0

Features

  • DI-7621: Implement serializer functionality in Designer. Use the serializer in mappings to parse structured data from a source and flatten it to string data.

  • DI-7927: Add support for the Snowpipe Streaming API in processes.

  • DI-8596: Update Snowflake backend to the latest version.

  • DI-8655: Update Replicator Rdbms to Snowflake: when in synchronize mode, use a merge to integrate data into the target table.

Bug fixes

  • DI-7370: Remove AUTONOMOUS SQL transaction type, as it has no effect.

  • DI-8808: Fix component dependencies.

  • DI-9201: Fix duplicate query generation when loading a JSON to multiple Snowflake tables.

Version 2023.4.0

Features

  • DI-8183: Upgrade third-party library.

Version 2023.3.0

Features

  • DI-6430: Replicator Snowflake: add the splitFileSize parameter.

  • DI-6604: Add BINARY, OBJECT, ARRAY, GEOGRAPHY, GEOMETRY, VARBINARY data types in the Snowflake technology.

  • DI-6628: Snowflake: implement TargetFilter.

  • DI-6640: Add DESERIALIZER Snowflake String as JSON Template.

  • DI-6641: Add DESERIALIZER Snowflake VARIANT as XML Template.

  • DI-7182: Replicator Rdbms to Snowflake: Add timestampFormat parameter to choose the format and precision of the timestamp columns.

  • DI-7183: Replicator Rdbms to Snowflake: Add emptyFieldAsNull and nullValueReplacement parameters.

  • DI-7623: Update templates using S3 features to use the existing AWS S3 TOOL instead of dedicated code.

  • DI-7767: Add MERGE incremental method to Snowflake Integration template.

  • DI-7983: Add delimiters around objects name in the generated queries when they contain reserved characters.

  • DI-7992: New template Load JSON to Snowflake.

Bug fixes

  • DI-7180: Replicator Rdbms to Snowflake: incorrect execution condition generated when using Snowpipe with some technologies.

  • DI-7305: SQL Server datatypes with maximum size such as varchar(max) should be reverse engineered as returned by the JDBC Driver, and the max keyword specified only at datatype creation.

  • DI-8040: In the reverse wizard, the list of schemas proposed is unexpectedly not filtered based on the selected catalog.

Version 2023.2.0

Features

  • DI-6683: The Positional File Processing property has been added to the Load file to Snowflake template to support positional files.

Version 2023.1.13

New Features

  • DI-9620: Added Semarchy Data Intelligence harvesting features.

Version 2023.1.10

New Features

  • DI-9413: Added the ability to change the timeout of the Snowpipe getStatusReport process.

Bug Fixes

  • DI-9275: Fixed an issue which stopped columns from being set to boolean type.

Version 2023.1.9

Bug Fixes

  • DI-9060: Add support for binary data types.

Version 2023.1.8

Features

  • DI-7983: Add delimiters around object names in generated queries when they contain reserved characters.

Version 2023.1.7

Bug fixes

  • DI-8808: Fix component dependencies.

Version 2023.1.6

Features

  • DI-8183: Upgrade third-party library.

Version 2023.1.5

Bug fixes

  • DI-8113: Load Rdbms to Snowflake: incorrect file path generated when using s3 as external storage

  • DI-8286: Third-Party library upgrade.

Version 2023.1.3

Features

  • DI-7766: Snowflake JDBC driver Third-Party library upgrade.

Version 2023.1.2

Features

  • DI-6683: The Positional File Processing property has been added to the Load file to Snowflake template to support positional files.

Version 2023.1.1

Bug fixes

  • DI-6578: In the Load Rdbms to Snowflake template, when the String Delimiter option is enabled, the mapping fails to compile.

Version 2023.1.0

Features

  • DI-1093: The Load External Storage File to Snowflake template is now available for loading files already present in an external storage into a target Snowflake table.

  • DI-5015: Designing and reading JSON data contained in a VARIANT column is now supported.

  • DI-6197: Snowflake JDBC driver upgraded to version 3.13.22.

  • DI-6224: The Replicator Rdbms to Snowflake tool is now available. This tool can be used to replicate data to Snwoflake using Snowpipe.

  • DI-6235: Google Cloud is now supported as an external storage location.

  • DI-6301: The Storage Access Method parameter has been added to the metadata. It allows defining the method to use to access the external storage: using credentials or using storage integration defined in Snowflake.

  • DI-6341: The Snowpipe finger tab has been added to the metadata. Snowpipe allows ingesting data files from Snowflake stages directly into Snowflake tables without having to launch the warehouse.

Bug fixes

  • DI-5029: Some error messages returned by the Load Wsdl to Rdbms template may not be relevant in finding the cause of the error.

Version 5.3.12 (component pack)

Features

  • DI-6683: Template Load file to Snowflake: add the Positional File Processing property to support positional files.

Bug fixes

  • DI-6578: Template Load RDBMS to Snowflake: when the String Delimiter option is enabled, the mapping does not compile.

Version 5.3.8 (component pack)

Features

  • DI-6262: The technology definition has been updated to mention that indexes are not supported. Thus, the generic templates do not try to create indexes on this database.

Version 5.3.7 (component pack)

Features

  • DI-5015: Support designing and reading JSON data contained in a VARIANT column.

Bug fixes

  • DI-5029: When being used with Snowflake as a target, error messages returned by "Load Wsdl to Rdbms Template" might not be relevant to find the real cause of the error.

  • DI-6197: Update the Snowflake JDBC driver third-party library to version 3.13.22.

Version 3.0.0 (component pack)

Features

  • DI-4508: Update Components and Designer to take into account dedicated license permissions

  • DI-4731: Rebranding: Template messages

  • DI-4962: Improved component dependencies and requirements management

  • DI-5014: Support QUALIFY clause in Snowflake

Version 2.1.3 (Snowflake component)

Features

  • DI-4855: Support "Technical Columns Prefix" on Reject and Integration Templates

Version 2.1.2 (Snowflake component)

Features

  • DI-4849: Snowflake - add support for additional timestamp types

Version 2.1.1 (Snowflake component)

Features

  • DI-4404: Snowflake - all work tables are now created as transient

Version 2.1.0 (Snowflake component)

Features

  • DI-2854: Microsoft Azure Storage and Amazon S3 Bucket nodes available in the workspace are now proposed automatically in Snowflake Metadata in External Storage attribute

Version 2.0.3 (Snowflake component)

Bug fixes

  • DI-2696: Snowflake - full outer joins were not working

  • DI-2741: Template - LOAD Rdbms to Snowflake - generated temporary file names may unexpectedly contain object delimiters

Version 2.0.2 (Snowflake component)

Features

  • DI-1056: Addition of a new Process Tool which allows to perform operations on a Snowflake Warehouse, such as starting or suspending a warehouse

  • DI-1910: Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet

  • DI-1909: Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking

Bug fixes

  • DI-1554: Load Rdbms to Snowflake Template - an exception was displayed when loading data through S3 storage, because the Module specification on S3 step was missing, avoiding the step to properly find the required dependencies

  • DI-1908: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates on Lock / Unlock CDC steps

  • DI-1907: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates when querying the source data