Pre-Upgrade Actions

Depending on the version of Semarchy xDM from which you upgrade, you need to perform actions prior to the upgrade procedure.

The changes and actions listed may impact your configuration, please review them carefully.
The actions are listed incrementally, please read all the sub-sections in this document until you reach your current Semarchy xDM product version.

For all versions

Browser Support

Semarchy xDM discontinues the support for old web browsers. The updated list of supported web browser is available in the system requirements.

Recommended Action
Administrators should inform all Semarchy xDM users of the new client requirements.

Upgrade from Versions Before 5.3.4

Javax.json library for Tomcat

This release no longer includes the javax.json library to avoid possible conflicts with other libraries (JMS drivers) or application server-provided implementations. Javax.json is part of JEE 7, and applications servers like JBoss, Glassfish, or Oracle Weblogic provide a built-in implementation of this library.

However, Tomcat does not provide a built-in implementation of javax.json. When deploying Semarchy xDM in an existing Tomcat server, make sure to copy the org.glassfish.jakarta.json_*.jar file from the mdm-server/additional-libraries/ folder to the Tomcat lib/ folder.

Upgrade from Versions Before 5.3.0

Single Deployment per Application Server

This release introduces a new mechanism to bootstrap a startup configuration for Semarchy xDM, using environment properties and variables. Due to this startup configuration mechanism, it is no longer possible, starting with this version, to perform multiple deployments of Semarchy xDM in the same application server.

Required Action
Before starting the upgrade, administrators should review their infrastructure for possible Semarchy xDM deployments in the same application server instance, in different deployment contexts.

For instances having multiple deployments in the same application server, it is recommended to perform an out-of-place upgrade. The replicated environment should be configured to take into account this limitation.

Integrated Datasources

This release introduces new built-in datasources to connect the backend databases, such as those hosting the repository and the data locations.

With this new mechanism, datasources are no longer configured at the application server level (in the semarchy.xml configuration file). They are configured in Semarchy xDM:

  • The repository connection (formerly SEMARCHY_REPOSITORY datasource) is now configured as two datasources, set in the startup configuration, via a configuration file or environment variables:

    • The Repository Datasource, which reproduces the SEMARCHY_REPOSITORY, with credentials allowing to read from and write into the repository schema.

    • A new Repository Read-Only Datasource, with credentials allowing to read a subset of the tables stored in the repository schema. This datasource uses a new dedicated user with read-only user privileges to the repository.

  • Other datasources are now configured as platform datasource in the Semarchy xDM Configuration.

    • Administrators must re-configure these datasources after the repository upgrade based on their current application server configuration.

    • Components using datasources, including Data locations, xDM Dashboard, xDM Discovery, the Variable Value Providers, and the enrichment/validation Plugins will refer to these datasources.

Required Action
Before starting the upgrade, administrators must:

The startup configuration is required to start the Semarchy xDM platform during the upgrade process.

Administrators should also gather the connection information for all datasources used by Semarchy xDM to re-configure these datasources during the post-upgrade actions.

For a Tomcat installation, the original datasources are defined in the semarchy.xml file.

Integrated Authentication

Semarchy xDM introduces in this release a new system to manage user authentication. With this new system:

  • Locally defined users authenticate with a built-in Internal Identity Provider, which stores users, passwords, and their assigned roles in the repository.

    This identity provider replaces users stored at the application server level. For example, the UserDatabaseRealm (tomcat-users.xml file) and DataSourceRealm (users stored in a database) for Tomcat.
  • Third-party Identity Providers (SSO such as Google, Active Directory, Azure AD, etc.) are configured in a new Identity Management built-in feature of the Semarchy xDM platform, which supports user authentication, authorizations, and profile synchronization.

    This feature replaces the identity provider configuration performed at the application server level. For example, the Realms and Valves configured in semarchy.xml for Tomcat.
  • A new built-in Role Lookup feature replaces Tomcat’s JDBC Database Realm to assign roles to users based on the content of a database table.

With these features, administrators no longer need to edit files on the server to configure the authentication and authorization for the Semarchy xDM platform.

When upgrading the repository:

  • You will log in using a Setup Token. The setup token is string of your choice that you set in a SEMARCHY_SETUP_TOKEN environment variable. See Startup Configuration for more information.

  • After connecting with the setup token, you will provide an administrator username and password to seed a first administrator (semarchyAdmin) user.

After the upgrade:

  • The Semarchy xDM platform will be configured with the Internal Identity Provider only and this administrator user.

  • Existing user profiles will be converted to users, with no password, with internal authentication disabled. These users may still be able to connect from a third-party identity provider. Administrators will need to enable the internal authentication, reset the passwords, and grant roles for the users to log in with the Internal Identity Provider.

Using first administrator credentials, administrators will be able to:

  • Reconfigure the users previously stored at the application server level.

  • Configure Identity Management, including:

    • The third-party authentication (single sign-on) systems as Identity Providers with optionally their Role Mapping.

    • If applicable, the Role Lookup feature, to assign roles to users based on database table contents.

Required Action
Before starting the upgrade, administrators should plan the post-upgrade actions to restore the authentication configuration. This includes the configuration of the third-party identity providers as well as the migration of their users stored in databases/files to the Internal Identity Provider.

For more information, review the following documentation sections:

For a Tomcat installation, the identity providers (SSO) configuration is contained in the semarchy.xml configuration file, and the local users used for form authentication are in the tomcat-users.xml file.

Platform Privileges

This release introduces changes to the platform privileges that you can assign to roles:

  • Two new privileges named User Management and Role Management are introduced to manage the users and roles. The semarchyAdmin role is required to configure Identity Management.

  • A new privilege named Dashboard Design grants limited access to the Dashboard Builder to create charts and dashboards only. This role complements the existing Dashboard Management role.

Recommended Action
Administrators should review the new platform privileges and plan to apply relevant changes to the existing roles.

Secrets Management

Starting with this version, you can configure how secrets - that is, sensitive data that need to be encrypted and optionally decrypted, such as passwords, tokens, or keys - are managed in Semarchy xDM. By default, these secrets are stored with default encryption.

The storage of the secrets can be configured to use more secure mechanisms:

  • A Key Management Service (KMS) defines how secrets stored in the repository are encrypted using, for example, a key local to the Semarchy server or a third-party system such as AWS Key Manager Service or Azure Key Vault.

  • Secrets Manager configuration allows storing secrets in an external system (for example AWS Secrets Manager or Azure Key Vault).

After the upgrade, Semarchy xDM is configured to use a default KMS for encrypting secrets, labelled Insecure. This KMS provides encryption capabilities at installation time using a default built-in key.

Recommended Action
Administrators should review the new secret management mechanisms and consider configuring, after the upgrade, stronger encryption or an external secrets manager.

Stored procedures for PostgreSQL

This release supports stored procedures for PostgreSQL version 11+. Stepper triggers now use procedures by default instead of functions.

Recommended Action
When upgrading from PostgreSQL 10- to 11+, administrators should review the stepper triggers definition and update legacy functions to procedures. Alternately, they can set the usePostgreSQLFunctionsAsProceduresInStepperTriggers=true system property to continue using functions as procedures.

Upgrade from Versions Before 5.2.0

Infrastructure & System Requirements

The following changes in the system requirements may impact your setup:

  • Oracle Database versions 11gR1, 11gR2, and 12gR1 are no longer supported.

  • PostgreSQL version 9.5.x is no longer supported.

  • Eclipse Jetty versions before 9.4 are no longer supported.

  • Wildfly versions before 17.x are no longer supported.

Recommended Action

  • Review the system requirements.

  • If your infrastructure uses a java machine, database, or application server version that is no longer supported, make sure to plan for an upgrade of your infrastructure before upgrading to this release.

  • Make sure to upgrade older driver versions to the newer ones.

Upgrade from Versions Before 5.1.0

New License Management

Semarchy xDM introduces in this release a new system to manage licenses. This new license system simplifies the management of licenses and xDM instances at an enterprise level. With the license system, new or upgraded xDM instances run during a grace period, during which they must be declared and activated.

The new license management applies to newly created xDM 5.1 instances, as well as instances upgraded to version 5.1.

Recommended Action
Before starting the upgrade, administrators should plan the upgrade of their various environments with the following facts in mind:

  • The repository upgrade process erases the existing license key in xDM instances when upgrading from a version before version 5.1.0.

  • Such instances have a 7 days grace period during which they work normally. After that period, they will stop working.

  • Before the grace period is over, administrators should activate the instance, using the instructions described in Manage the License.

Our global Technical Support Center (support@semarchy.com) team is available for assistance during the upgrade process and to help planning the license change.

Oracle JDBC Datasource Configuration

Starting with this version, the JDBC connections to an Oracle Database hosting a repository or data location must be made with the oracle.jdbc.J2EE13Compliant property set to true.

When this option is not set, errors such as the following one will be raised in the application log.

com.semarchy.mdm.runtime.data.InvalidDataAccessResourceUsageException: java.lang.RuntimeException: Unexpected DB value .... (Class oracle.sql.TIMESTAMP for logicalType TIMESTAMP)

Required Actions

Administrators should review their Oracle datasources configuration and make sure to have them configured with the oracle.jdbc.J2EE13Compliant property set to true.

New Indexes on MD Tables

The data location upgrade in version 5.1.0 creates new performance indexes for all the MD_xxx tables, using the B_SUGGMERGEID column.

For performance optimization, database administrators may have already created manually such indexes, which may enter in conflict with the new ones, created during the data location upgrade.

Required Actions

Administrators should check all MD_xxx tables in data locations for customized indexes manually created on the B_SUGGMERGEID column. If such indexes exist, they should be dropped before the data location upgrade.

Upgrade from Versions Before 5.0.0

Infrastructure & System Requirements

The following changes in the system requirements may impact your setup:

  • Apache Tomcat 7.x-8.0.x and Oracle WebLogic 12c Release 1 (12.1.x) application servers are no longer supported

  • Oracle Database version 10g is no longer supported for the repository and data locations.

  • Oracle JDBC driver version is updated to version 12.2.x (ojdbc8.jar).

  • PostgreSQL JDBC driver version is updated to version 42.2.5.

Other changes to the Semarchy xDM web application:

  • The web application uses JEE 7

  • Support for Java 11.

Recommended Action

  • Review the system requirements.

  • If your infrastructure uses a java machine, database, or application server version that is no longer supported, make sure to plan for an upgrade of your infrastructure before upgrading to this release.

  • Make sure to upgrade older driver versions to the newer ones.

  • Review the installation instructions to review possible changes required to switch to Java 11.

Upgrade from Versions Before 4.0.0

This section applies to all installations upgrading from a version before version 4.0.0

Infrastructure & System Requirements

The following changes in the system requirements may impact your setup:

  • Semarchy xDM now requires Java 8. A Java Runtime Environment (JRE) or Development Kit (JDK) version 1.8.0 or above is now required.

  • The following application servers are no longer supported as they do not support Java 8.

    • Glassfish 3.x (4.x is the minimum required version)

    • JBoss 7.x (Wildfly 8 is the minimum required version)

Required Action
If your current infrastructure does not meet the new requirements, upgrade your infrastructure before upgrading Semarchy xDM.

Browser Support

Semarchy xDM discontinues the support for old web browsers. The updated list of supported web browsers is available in the system requirements.

Recommended Action
Administrators should inform all Semarchy xDM users of the new client requirements.

SOAP Web Services Decommission

SOAP web services available in previous versions of Semarchy xDM are decommissioned in this release. A new REST API is made available for new users, providing similar integration capabilities for querying data in the hub and publishing data into the MDM hub. Note that certain features available in the SOAP Web Services are not available in the REST API, such as workflow management and administrative services.

Recommended Action
Existing users relying on the SOAP Web Services, and willing to continue using them temporarily before moving to the new REST API should contact our global Technical Support Center (support@semarchy.com).

Pulse Dashboard Not Unavailable

This version does not include the Pulse dashboards in the generated application. Pulse is replaced by Semarchy xDM Discovery for profiling and Semarchy xDM Dashboard for metrics.

Recommended Action
Existing users currently using Pulse should contact our global Technical Support Center (support@semarchy.com) for more information about these new components.

Data Edition Decommission

The data edition feature is removed in this release. A data location now hosts one deployed model edition at a time and provides access to the current state of the data in the hub, based on the deployed model edition.

The upgrade process prunes the data in closed data editions from the data locations. Only data from the latest data edition is preserved in each data location.

Recommended Action
Existing users willing to archive the contents from previous data editions should export this information from the data location tables using scripting or data integration flows, or keep a backup of the data location contents performed before the upgrade.

Error Storage

Error storage has been changed in this release. The golden errors and source error tables no longer store the entire history of errors, but only the latest error instances.

The data location upgrade process prunes the history of errors on golden and source records, preserving only the latest error instances.

Recommended Action
Existing users willing to preserve the history of errors should export this information from the data location tables using scripting or data integration flows, or keep a backup of the data location contents performed before the upgrade.