Upgrade Procedure Steps

This document provides instructions for the steps of the Semarchy xDM upgrade process.

Make sure to select follow the steps corresponding to the upgrade path that you have selected.

Stop All Server Instances and Connections

Before upgrading, you must stop all applications and user connections to Semarchy xDM. During the upgrade:

No application should access the Semarchy xDM services and APIs.
No user should access the Semarchy user interfaces for design-time or data management operations.
No user or application should access the repository or data locations' database schemas.

You should contact your database administrator and middleware administrator to make sure that the unavailability of Semarchy xDM during the upgrade process is correctly managed.

Stop all Semarchy xDM Servers. This is done either by:

Stopping Semarchy xDM from your application server administration console.
Stopping the application server if it is dedicated to Semarchy xDM only.

Backup Your Installation

Before upgrading, you should make sure that:

The schema(s) containing the Semarchy xDM repositories are backed up; Use the database utilities for performing such backup operations.
The schema(s) containing the Data Locations are backed up; Use the database utilities for performing such backup operations.
The Semarchy xDM installation folders in the application servers are backed up.
Any other relevant content (plug-in Jars, etc.) is also backed up.

Before going further, make sure that all the required backups are done, and that the entire Semarchy xDM environment is stopped and not accessed by any user or application.

Duplicate Database Schemas (Out of Place Upgrade)

If you have decided to perform an out-of-place upgrade, you must replicate (Copy) all the schemas (the repositories, data locations, etc.) using the database utilities.
The new product version, considered as a separate installation, will point to this copy of the schemas.

Use a clear naming convention for these new schemas to be able to identify them easily. For example, if the original schemas are called SEM_REPOSITORY, SEM_DLOC1, etc., the new one may be called SEM_5_3_REPOSITORY, SEM_5_3_DLOC1, etc.

With PostgreSQL, when replicating the databases, make sure that the search_path parameter is correctly set to $user, public, extensions in the database copies.

Install the New Server Version

Perform the following operation for every application server into which Semarchy xDM was deployed.

When installing a new server version for an upgrade, pay attention to the versions of the application server and additional libraries.

Some libraries are compatible with a certain version of Java or the application server and will work with your current configuration. Do not systematically upgrade the application server and the libraries when upgrading Semarchy to the latest version. If you decide to upgrade these components to newer versions, make sure to review and test the compatibility and configuration.

Semarchy xDM introduced in version 5.3 built-in support for third-party identity providers (SSO with LDAP, Google, Active Directory, Azure AD, etc.), which replaces the configuration performed at the application server level. The libraries used for authentication, such as the Tomcat tools for authentication and role mapping (the com.semarchy.tool.jee.tomcat-<tomcat_version>.jar file) are no longer required:

For an in-place upgrade, these libraries must be removed to avoid possible conflicts in SSO authentication.
For an out-of-place upgrade, these libraries should not be copied in the new server installation.

In-Place Upgrade

To install the new version for an in-place upgrade, deploy the new Semarchy xDM Web Archive file on top of the previous version.

Refer to your application server documentation for more information about re-deploying applications on top of previous installations. The simple instructions for Apache Tomcat are provided below.

To perform an in-place upgrade for Apache Tomcat.

Upload the new semarchy.war file to a temporary directory in the Tomcat server, for example /tmp (Unix/Linux) or c:\temp (windows).
Make a copy of the existing configuration file (<tomcat>/conf/Catalina/localhost/semarchy.xml) to the same directory on the server.
Connect to the Apache Tomcat Manager (http://<tomcat_host>:<tomcat_port>/manager/).
In the applications list, note the Context Path of the existing Semarchy xDM application, and then click the Undeploy button to un-deploy it.
In the Deploy directory of WAR file located on the server section, type in the context path and enter the file paths to the semarchy.xml and semarchy.war files.
For example:
- Context Path (required): /semarchy.
- XML Configuration file URL: /tmp/semarchy.xml (Unix/Linux) or c:\temp\ semarchy.xml (windows)
- WAR or Directory URL: /tmp/semarchy.war (Unix/Linux) or c:\temp\ semarchy.war (windows)
Click the Deploy button.

Out-of-Place Upgrade

To install the new version for out-of-place upgrade:

Deploy the new version Semarchy xDM Web Archive file as a new deployment, in a new application server instance, separated from the previous version.
Refer to your application server documentation for more information about the deployment procedure.

When performing an out-of-place update, make sure to deploy the new version as a new deployment in a different application server instance. It is not possible to deploy multiple Semarchy xDM applications in the same application server instance.

Configure the New Server

Perform the following operation for each deployment of the new Semarchy xDM version.

In-Place Upgrade

For in-place upgrades, the updated deployment reuses the configuration already in place. Check this configuration in the application server console or configuration files. The following points should be checked:

The startup configuration is set up for Semarchy xDM to connect the repository schema.

If upgrading from a version before 5.3, create the startup configuration and apply it to each of the servers where Semarchy xDM is deployed.
If upgrading from a version before 5.3, the SEMARCHY_SETUP_TOKEN environment variable is set with the value that will be requested to authenticate and perform the repository upgrade.
If upgrading from a version before 5.3, make sure that extraneous libraries used for SSO authentication (for example, the com.semarchy.tool.jee.tomcat-<tomcat_version>.jar file) are removed.
The JavaMail Session servers are configured and working.

Out-of-Place Upgrade

To configure the new version in an out-of-place upgrade:

Configure the startup configuration in the new deployment to connect to the copy of the original repository schemas (e.g.: SEM_5_3_REPOSITORY).

If upgrading from a version before 5.3, make sure to create the startup configuration and apply it to each of the new servers where Semarchy xDM is deployed.
If upgrading from a version before 5.3, set the SEMARCHY_SETUP_TOKEN environment variable to a value that will be requested to authenticate and perform the repository upgrade.
If upgrading from a version before 5.3, make sure that extraneous libraries used for SSO authentication (for example, the com.semarchy.tool.jee.tomcat-<tomcat_version>.jar file) are not installed.
Make sure that the existing JavaMail Session resource is configured or available for the new deployment.

At that stage, the copy of the original environment should be fully functional.

Start One New Server Instance

If upgrading from a version before 5.3.0, make sure that:

you have configured startup configuration required by Semarchy xDM to connect the repository.
you have set the SEMARCHY_SETUP_TOKEN environment variable to a value that will be requested to authenticate and perform the repository upgrade.

Perform this operation for a single new deployed version instance. If you have multiple repositories, you should repeat this operation for each repository.
Start your application server or connect to your application server and start the deployed version.

During the first startup, review the application logs for possible issues with the startup configuration. An error in this configuration will prevent Semarchy xDM to start as it cannot connect the repository schema.

Upgrade the Repository

Repository upgrade is required only for major and minor versions. For patches, repository upgrade is not needed and the Repository Upgrade dialog does not appear when you connect.

The repository upgrade is an operation that cannot be undone. Make sure you have made a backup of the repository schema before proceeding.

Before the Repository Upgrade

Versions Before 5.3.0

If upgrading from a version before 5.3.0, or if upgrading to a release that includes changes to the identity management framework built-in Semarchy xDM, you will be prompted for a setup token to log in. The setup token is an arbitrary string that must be provided in a SEMARCHY_SETUP_TOKEN environment variable as part of the startup configuration.

Repository stored in Oracle

The Oracle database has introduced in version 11g and 12c a new feature called Automatic Column Group Detection that automatically creates extended statistics for column groups. These extended statistics prevent the upgrade process to rename certain columns in the repository, causing an ORA-54032: column to be renamed is used in a virtual column expression error to appear during the repository upgrade process.

Before upgrading a repository hosted in an Oracle 11g or 12c instance, it is required to drop these statistics. They will be automatically re-created by the database as needed.

To drop unwanted extended statistics for repositories hosted in Oracle 11g and 12c:

Connect to the database hosting the repository schema with a query tool (SQL Developer, SQL*Plus), using the repository schema user.
Run the script provided below:

begin
  for xrec in (
    select
      TABLE_NAME, EXTENSION
    from
      USER_STAT_EXTENSIONS
    where
      TABLE_NAME like 'MTA%'
      and (
        EXTENSION like '%FROMEDITION%'
        or EXTENSION like '%TOEDITION%'
      )
  )
  loop

    DBMS_STATS.DROP_EXTENDED_STATS (
      ownname=>null,
      tabname=>xrec.table_name,
      extension=>TO_CHAR(xrec.extension)
    );

  end loop;
end;

Once the unwanted extended statistics are dropped, proceed with the repository upgrade.

Repository Upgrade

To upgrade the repository:

Connect to the new instance you have started, using a user account with the semarchyAdmin role or using the Setup Token configured in the startup configuration.
At the first connection, the Semarchy xDM Upgrade dialog appears.
Review the dialog.
If upgrading from a version before 5.3, or if upgrading to a release that includes changes to the identity management framework built-in Semarchy xDM, need to provide an administrator username and password. This user is created or updated by the upgrade process with this password and granted the semarchyAdmin role.
Select I have made a backup of the repository and data location schemas and then click Upgrade Instance.

The repository upgrade starts.

The repository upgrade process can only be triggered by a user with the semarchyAdmin role, or when logged in with a setup token.

Upgrade the Data Locations

Data locations upgrade is required only for major and minor versions. For patches, a data location upgrade is not needed.

When a data location needs or is upgrading:

Users cannot access applications in the data location. Applications using the data location will appear disabled on the Semarchy xDM welcome page.
Model designer cannot deploy model editions in the data location.

To reduce application downtime, it is recommended to plan the data location upgrade immediately after the repository upgrade.

The data location upgrade is an operation that cannot be undone. Make sure you have made a backup of the data location schemas before proceeding.

Development data locations upgrade is possible only if the current (latest) model state is deployed and if this model is valid. Before upgrading a development data location, make sure that the model does not raise any design-time validation errors.

Before the Data Locations Upgrade

Versions Before 5.3.0

Semarchy xDM introduces in this release a new mechanism to connect the backend databases, including those hosting data locations. Datasources are no longer configured at the application server level but in the repository.

The repository upgrade automatically creates empty datasources for the components that refer to application server datasources. However, it does not recover the datasource configurations from the application server.

Referenced JNDI datasources are upgraded as empty platform datasources with normalized names. The normalized names are derived from the JNDI names by performing the following operations:

The java:comp/env/jdbc/ prefix is removed.
The remaining part is normalized by converting the characters that are not alphanumeric or underscore to _.

For example, in a data location, a reference to a JNDI datasource named java:comp/env/jdbc/DATA_LOCATION_1 is upgraded to a platform datasource named DATA_LOCATION_1. That datasource is not configured.

Before upgrading the data locations, administrators must re-configure their datasources in Semarchy xDM .

For a Tomcat installation, the original datasources configuration should be retrieved from the semarchy.xml file.

When re-configuring data location datasources, make sure to set the following technology-specific driver properties:

For Oracle datasources, the oracle.jdbc.J2EE13Compliant property must be set to true.
For PostgreSQL datasources, the defaultRowFetchSize property should be set to 2000.

Data Locations stored in Oracle

Before upgrading a data location hosted in an Oracle 11g or 12c instance, it is required to drop the extended statistics that may prevent column renaming. For more information, refer to Before the Repository Upgrade.

To drop unwanted extended statistics:

For each data location:
1. Connect to the database hosting the data location schema with a query tool (SQL Developer, SQL*Plus), using the data location schema user.
2. Run the script provided below:

begin
  for xrec in (
    select
      TABLE_NAME, EXTENSION
    from
      USER_STAT_EXTENSIONS
    where (
        TABLE_NAME like 'GI_%'
        or TABLE_NAME like 'GD_%'
        or TABLE_NAME like 'SD_%'
  or TABLE_NAME like 'MI_%'
      )
      and (
        EXTENSION like '%B_IS_VALIDATED%'
        or EXTENSION like '%B_STATUS%'
  or EXTENSION like '%B_HASSUGGMERGE%'
      )
  )
  loop

    DBMS_STATS.DROP_EXTENDED_STATS (
      ownname=>null,
      tabname=>xrec.table_name,
      extension=>TO_CHAR(xrec.extension)
    );

  end loop;
end;

Once the unwanted extended statistics are dropped, proceed with the upgrade.

Data Locations Upgrade

To upgrade the data locations:

Connect to the new instance you have started, using a user account with the semarchyAdmin role or having a role with the Application Management privilege.
On the Welcome page, click Application Builder.
In the Management perspective, a warning message appears in the Management view to give the list of data locations that need an update.
Click the link under this list to perform the upgrade. The Data Location Upgrade dialog appears.
Review the dialog.
Click the checkbox and then click the Upgrade button.

The upgrade starts. When it completes, the window is refreshed, and the warning message disappears.

Restart All Server Instances

The upgrade is now finished.
You can restart all the Semarchy xDM instances in the new version, and open the access through the user interface, services, APIs, and database tables.