Upgrade the Semarchy xDI Runtime

This document provides instructions to upgrade the Semarchy xDI Runtime.

Pre-upgrade steps

This section describes the operations to perform before upgrading Semarchy xDI Runtime.

Review the release notes

The release notes contain the latest information about the Semarchy xDI release, such as new features, bug fixes, breaking changes, and removals.

System requirements

Review the system requirements for the new release. Take special note of supported Java versions, as you may need to install a newer one depending on which Runtime version you are upgrading from.

Stop the Runtime

If it is running, stop the existing Runtime.

Back up the Runtime

Make a backup of the existing Runtime installation directory.
Back up the sessions log database.
Back up the scheduler database.

Upgrade steps

Get and install the Runtime

Download the new version of the Semarchy xDI Runtime from the Semarchy website.
Decompress the downloaded file and move the content to a new installation directory.
Check and set the execute permission on all the .bat (Windows) and .sh (Linux or macOS) scripts located at the root of the new installation directory.
Replicate the read/write permissions for other files from the previous installation directory. In particular, check and set write permissions for the following subdirectories:
- ./temp
- ./build
- ./sessions
Copy the content of the following directories from the previous Runtime installation to the new installation:
- ./build/deliveries
- ./build/packages
- ./scheduler (if you are using the Runtime’s scheduler)
- ./sessions (if you use the Runtime’s internal log database)
- ./temp (if your Processes store specific files in this directory)

Copy modules

Copy the modules from the previous installation to the new installation directory.

By default, modules are in the ./modules subdirectory of the Runtime installation directory.

Configure the Runtime

Configure the new Runtime based on the previous Runtime’s configuration.

When upgrading the xDI Runtime to version 2024.x, you should pay attention to the following points:

The xDI Runtime built-in H2 database has been replaced by an HSQL database, only accessible from the runtime itself. See Built-in HSQL database and Migrate the built-in database from H2 to HSQL.
The xDI Runtime scheduler is now configured in the main runtime configuration file, engineParameters.xml. The legacy engineScheduler.properties file is still supported but is no longer available by default. See Runtime scheduler.

Migrate the built-in database from H2 to HSQL

The xDI Runtime built-in H2 database was replaced by an HSQL database in version 2023.4.0.

If you are upgrading from a version prior to 2023.4.0, and you were using the built-in H2 database, you must migrate your H2 data to HSQL by hand if you want to keep your data. See [_migrate_the_built_in_database_from_h2_to_hsql_2]

Update the scheduler database

If you upgrade from a version before 2023.1.0, and you are using the xDI Runtime Scheduler, you must manually update the Scheduler database before starting the upgraded xDI Runtime.

The database structure has changed following an upgrade of the third-party library used for the Scheduler, which requires an update of the database structure.

Start the new Runtime

Start the new Runtime.

Post-upgrade steps

Depending on the version of Semarchy xDI from which you upgrade, you need to perform actions after the upgrade procedure.

The actions are listed incrementally, please read all the sub-sections in this document until you reach your current Semarchy xDI product version. If no action is indicated in this document for your versions, then no specific action is required.

For example, when upgrading from S17 to 2023.1.x, install version 2023.1.x and follow the upgrade steps for S20, for 5.3.x, then for 2023.1.x.

Generic steps

Once you have validated the new Runtime is running properly, perform the following operations:

Remove the old Runtime folder.

Upgrading from versions before 2023.2.0

The thread count for the runtime scheduler thread pool was changed from 50 to 3. If you are copying over a configuration, make sure you are not using the old value.

In the new engineParameters.xml file, this number is found with the org.quartz.threadPool.threadCount parameter.

Upgrading from versions before 2023.1.0

If you are upgrading from a version before 2023.1.0:

Switch from RMI to HTTP:
- Replace all connections to the xDI Runtime from RMI to HTTP protocol.
  - Replace rmi(s)://hostname:port with http(s)://hostname:port in all applicable locations.
    If you were accessing the runtime without specifying the protocol, HTTP is now the default protocol used. In this situation, you only need to change the port.
  - Replace the RMI port with the HTTP port. The default port for HTTP is 42200.
- Fine below a list of the Semarchy xDI locations where this must be changed:
  - xDI Analytics Runtime Configuration.
  - Runtime Start Delivery process action.
  - Runtime connect and schedule delivery command-lines.
  - Runtime executeRemoteCommand(s) scripting functions.
Update Jython scripts:
The default Jython third-party library shipped with xDI Runtime has been upgraded.
- The related jythonVersion Runtime parameter has been commented by default for the Runtime to use the shipped version by default.
- The new Jython version shipped replaces the javos with the os module. Scripts using that module must be updated accordingly.

Upgrading from version S17

If you are upgrading from version S17, follow the upgrade steps of all the major intermediate versions between your version and 2023.1.x version.

Appendix

Update the scheduler database structure

Summary

If you upgrade from a version before 2023.1.0, and you are using the xDI Runtime Scheduler, you must manually update the scheduler database before starting the upgraded xDI Runtime.

The database structure has changed following an upgrade of the third-party library used for the Scheduler, which mandates a manual update of the database structure.

The procedure differs if you are using the built-in database or an external database.

Update the built-in scheduler database

If you were using the default, built-in H2 scheduler database, database structure is upgraded when you migrate from H2 to HSQL. See Migrate the built-in database from H2 to HSQL.

Update an external scheduler database

If you are using an external Scheduler database:

Make sure the xDI Runtime is not running.
Back up the database.
Update the database by running the SQL script corresponding to your provider. The scripts are located in runtime/scripts/scheduler/upgrade/2023.1.0/upgrade_tables_<database-provider>.sql.

Run this script on the external database using Semarchy xDI Designer or an external query tool.

Migrate the built-in database from H2 to HSQL

Summary

The Semarchy xDI Runtime built-in H2 database has been replaced by an HSQL database, only accessible from the runtime itself.

If you were using the built-in H2 database for quickstart or temporary purposes and don’t mind about losing the quickstart data, we recommend using the new built-in HSQL database directly without migrating the old data.
If you were using the built-in H2 database as a production database for the session logs and scheduler:
- As a reminder, the built-in database is provided for quickstart purposes as a convenience and is NOT recommended for production use. We highly recommend using one of the certified database servers instead. See Built-in HSQL database.
- If you want to continue using the built-in database and keep the previous data, you must manually migrate the H2 built-in database data to the new HSQL built-in database, as explained in the guide below.

The built-in HSQL database is, by default, only accessible from the runtime itself, and is NOT recommended for production use.

For production use, we recommend using the certified database servers.

In the migration guide, source runtime refers to the previous runtime version that contains the built-in H2 database, and target runtime refers to the new runtime version with HSQL as the built-in database.

Pre-migration steps

Backup the scheduler and sessions folders from the source runtime directory.
With Semarchy xDI Designer:
1. Create an H2 module named h2_legacy_driver.
2. Add version 1.2.140 of the H2 JDBC driver to the module.
Prepare the migration folder:
1. Download the Migration Tool.
2. Unzip the archive file at the location of your choice in the file system of the machine hosting the target runtime. It will be used to migrate the database from H2 to HSQL.
3. Copy the previously created h2_legacy_driver module folder to the xdi-built-in-database-migration/modules folder.
4. Copy the scheduler and sessions folders from the source runtime to the xdi-built-in-database-migration/source_data folder.
Prepare the target runtime:
1. Copy the previously created h2_legacy_driver module folder to the modules folder of the target runtime.
2. Copy the sessionLogs_migration.deliv and sessionLogs_migration.deliv files from the xdi-built-in-database-migration/deliveries folder to the build\deliveries folder of the target runtime.
3. Ensure the target runtime configuration (engineParameters.xml file) matches the following prerequisites:
  1. The temporary folder must be configured with the default value temp.
  2. The admin user must exist with admin-password as the password. It is required by the migration script to stop the runtime.

Known Limitations:

Scheduler: The source and target schemas must be the default value SCHEDULER.
SesssionLogs: The source and target schemas must be the default value LOGS.

Migration steps

Open a prompt command and run the startMigration.bat (Windows) or startMigration.sh (Linux) script.

The following arguments are available:

Common arguments:

Arguments Description

runtimePath <path>

The target runtime path. This argument is mandatory.

The target runtime path must not contain any spaces or special characters.

deliveryLogLevel <logLevel>

Set the log level used by the migration deliveries. The default value is 0 if this parameter is not set.

Scheduler database arguments:

Arguments Description

schedulerMigration

When this argument is defined, the script migrates the scheduler database from H2 to HSQL.

Only one of the schedulerMigration or sessionLogsMigration argument must be used at the same time.

deleteHsqlScheduler

Deletes the target HSQL scheduler database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.

This argument can be used only when migrating the scheduler database with schedulerMigration.

h2SchedulerLogin <login>

The login of the source H2 scheduler database. To specify an empty value, omit this argument.

h2SchedulerPassword <password>

The password of the source H2 scheduler database. To specify an empty value, omit this argument.

hsqlSchedulerLogin <login>

The login of the target HSQL scheduler database. To specify an empty value, omit this argument.

hsqlSchedulerPassword <password>

The password of the target HSQL scheduler database. To specify an empty value, omit this argument.

schedulerInstanceId <instanceId>

The target scheduler instance identifier. It must be equal to the org.quartz.scheduler.instanceId parameter of the engineParameters.xml` file.

The default value is RUNTIME_HSQL_STD if this parameter is not set.

schedulerInstanceName <instanceName>

The target scheduler instance name. It must be equal to the org.quartz.scheduler.instanceName parameter of the engineParameters.xml`file.

The default value is RUNTIME_HSQL_STD if this parameter is not set.

Sessions logs database arguments:

Arguments Description

sessionLogsMigration

When this argument is defined, the script migrates the session log database from H2 to HSQL.

Only one of the schedulerMigration or sessionLogsMigration argument must be used at the same time.

deleteHsqlSessionLogs

Deletes the target HSQL session logs database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.

This argument can be used only when migrating sessions logs with sessionLogsMigration.

h2SessionLogsLogin <login>

The login of the source H2 session logs database. To specify an empty value, omit this argument.

h2SessionLogsPassword <password>

The password of the source H2 session logs database. To specify an empty value, omit this argument.

hsqlSessionLogsLogin <login>

The login of the target HSQL session logs database. To specify an empty value, omit this argument.

hsqlSessionLogsPassword <password>

The password of the target HSQL session logs database. To specify an empty value, omit this argument.

Session logs database migration example:

Windows:

startMigration.bat -runtimePath C:\Semarchy\runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password

Linux:

bash startMigration.sh -runtimePath /Semarchy/runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password

Scheduler database migration example:

Windows:

startMigration.bat -runtimePath C:\Semarchy\runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD

Linux:

bash startMigration.sh -runtimePath /Semarchy/runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD