Semarchy xDI 2023.4 release notes

This document provides the latest information about Semarchy xDI version 2023.4, including new features and Bug Fixes.

Support information

Semarchy xDI version 2023.4 is a mainstream support (MS) release.

MS product versions are released throughout the year. They showcase the latest features and enhancements, and will receive active support for a limited period.

Alternatively, long-term support (LTS) versions offer a longer maintenance period with patches and fixes, and benefit from extended support.

For more details about our releases and support policies, see our Support and maintenance terms.

New Features

Databricks connector

Connectors allow xDI users to fully exploit the technologies they have chosen to collect, store, and analyze their data. With the introduction of the Databricks connector, xDI can now natively use Databricks’s powerful SQL capabilities for inbound and outbound data integration with this database.

See Databricks database.

Deserializers

Continuing on the major mappings design experience improvements introduced in Semarchy xDI 2023.1 through Deserializers, Semarchy xDI 2023.4 expands this capability with dedicated templates to de-serialize JSON data contained in MySQL and Redshift database columns.

See Deserializer.

Connectors updates

In addition to introducing Connectors for new technologies, existing connectors are also regularly updated. These updates may be required by changes to the technologies themselves, or to include additional functionality.

MongoDB

Added support for MongoDB version 6.x.

Elasticsearch

A number of improvements have been made to the Elasticsearch connector, specifically:

  • API Key authentication.

  • TLS/SSL certificate customization.

See Getting started with Elasticsearch.

Vertica

Support for writing hierarchical data types (ARRAY and ROW types).

See Vertica Array and Row types.

Connector site per version

The installation of connectors has been improved with the introduction of dedicated URLs for each quarterly release in the components update site. Using these URLs, designers can install connectors from specific release versions.

See Install components in Semarchy xDI Designer.

Other changes

Improvements to other technologies such as Greenplum, Salesforce, Couchbase and Kafka have been introduced. For example:

  • Support for external value resolvers in Salesforce and Kafka connection properties.

  • Improved organization: support sub-metadata with Couchbase and MongoDB metadata.

  • Target Filter support in Amazon Redshift.

  • Many improvements in Greenplum and Amazon templates.
    New parameters, fixes…

Default runtime database moved to HSQL

The built-in database in the Semarchy xDI Runtime has been changed from H2 to HSQL. This built-in database is intended for quick-start testing and demonstration purposes. Migrating to HSQL remediates security and performance issues caused by H2.

See Built-in HSQL database and Migrate the built-in database from H2 to HSQL.

Miscellaneous

Notice about Snowflake GCP usage

Snowflake JDBC driver versions prior to 3.13.25 will stop working with Google Cloud Platform on November 1, 2023. See the Snowflake FAQ.

If you are using Snowflake with Google Cloud Platform as the provider, ensure that you are using a Snowflake JDBC Driver at version 3.13.25 or above in Semarchy xDI.

The Snowflake JDBC driver optionally proposed in the Semarchy xDI Snowflake connector version 2023.1.5 and above is already up to date. If you are using a prior version of the connector, consider upgrading the connector version, or edit your module to replace the JDBC Driver version with the appropriate one.

Changelog

2023.4.0

Semarchy xDI Designer

Breaking Changes

  • DI-8378: Move H2 to a dedicated component, and the JDBC library as an optional third-party pack.

    If you were using H2 and are using a local license server, you need to ask for an updated license file for Semarchy xDI Designer to use this connector. If you are using the Semarchy central license server, the permissions are already up to date and you will automatically be able to use the connector in Semarchy xDI Designer.
New Feature

  • DI-5841: Order the input parameters alphabetically in the delivery editor and scheduler editor.

  • DI-6325: File metadata wizard: provide additional fields inside the reverse helper.

  • DI-8177: Upgrade eclipse target platform.

  • DI-8443: The RDBMS reverse engineering does not update the position of columns already existing in the metadata.

  • DI-8450: Add Databricks SQL component.

  • DI-8609: The default component installation URL is updated to use a version-specific URL.

Bug Fixes

  • DI-7653: When Google BigQuery is the target, some source datatypes are not converted to the correct target datatypes.

  • DI-7747: Using the external value syntax (%ext{}ext%) in a driver class parameter does not work.

  • DI-8246: Mapping OUTER JOIN"* expressions may be incorrectly generated in some situations.

  • DI-8270: When copying or pasting items from a mapping to another, the generated copies are incorrect and broke the second mappings, which become invalid.

  • DI-8295: JSON: the error java.lang.Exception: close element: root doesn’t match to current is displayed when reverse engineering arrays of string values.

  • DI-8413: File Metadata: unable to perform a transformation script on a file.

  • DI-8526: Welcome Page links are redirecting to incorrect URLs.

  • DI-8538: When working with projects located outside of the workspace, the designer may be unable to build mappings because the project build file location may be incorrectly defined in some situations.

Semarchy xDI Runtime

Breaking Changes
New Feature

  • DI-8022: Spring Boot third-party library upgrade.

  • DI-8114: Add scripts to shut down the built-in HSQL database.

  • DI-8198: HSQL third-party library upgrade.

Bug Fixes

  • DI-699: Delimited/Positional files: when an empty line is encountered, the file driver unexpectedly stops proceeding the rows and ignore the next ones.

  • DI-5930: Null values are unexpectedly loaded as "<|null|>" inside the target table when reading Salesforce data.

  • DI-7747: Using the external value syntax (%ext{}ext%) in a driver class parameter does not work.

  • DI-7974: When a delivery is run as API/Web Services and the response code is mapped to a parameter, stack trace messages are unexpectedly added to the response body.

  • DI-8125: Loss of precision when loading a REAL into a DOUBLE.

  • DI-8232: Jackson third-party library upgrade.

  • DI-8288: When binding a null value in an integer column, the value "0" is written instead of "NULL".

  • DI-8413: File Metadata: unable to perform a transformation script on a file.

  • DI-8445 / DI-8448: Spring third-party library upgrade.

  • DI-8477: The error An invalid XML character (Unicode: 0x90) might be displayed with Kafka technology using the Avro schema Registry.

  • DI-8505: An HTTP 500 error with the message "Internal Server Error" is unexpectedly returned when an error is caught in a delivery run as a web service.

  • DI-8523: When scheduling a delivery with command line execution mode, the execution fails if the delivery contains a “start delivery” process action configured with “non-memory” mode.

Semarchy xDI Production Analytics

New Feature

  • DI-5841: Order the input parameters alphabetically in the delivery editor and scheduler editor.

  • DI-8179: Upgrade eclipse target platform.

Bug Fixes

  • DI-2589: The error Failed to load the sqljdbc_auth.dll is displayed when connecting through integrated security to a Microsoft SQL Server session log database.

  • DI-7651: In the administration, the runtime test button may fail with errors such as password authentication failed for the user.

  • DI-7668: Guava third-party library upgrade.

  • DI-8228: Jackson third-party library upgrade.

  • DI-8386: D3JS third-party library upgrade.

  • DI-8502: The Package Manager is unstable when loading multiple packages with drag and drop.

License Service

No changes in this version.