Upgrade the Semarchy xDI Runtime
This document provides instructions to upgrade the Semarchy xDI Runtime.
Pre-upgrade steps
This section describes the operations to perform before upgrading Semarchy xDI Runtime.
Review the release notes
The release notes contain the latest information about the Semarchy xDI release, such as new features, bug fixes, breaking changes, and removals.
System requirements
Review the system requirements for the new release. Take special note of supported Java versions, as you may need to install a newer one depending on which Runtime version you are upgrading from.
Back up the Runtime
-
Make a backup of the existing Runtime installation directory.
-
Back up the sessions log database.
-
Back up the scheduler database.
Upgrade steps
Get and install the Runtime
-
Download the new version of the Semarchy xDI Runtime from the Semarchy website.
-
Decompress the downloaded file and move the content to a new installation directory.
-
Check and set the execute permission on all the
.bat
(Windows) and.sh
(Linux or macOS) scripts located at the root of the new installation directory. -
Replicate the read/write permissions for other files from the previous installation directory. In particular, check and set write permissions for the following subdirectories:
-
./temp
-
./build
-
./sessions
-
-
Copy the content of the following directories from the previous Runtime installation to the new installation:
-
./build/deliveries
-
./build/packages
-
./scheduler
(if you are using the Runtime’s scheduler) -
./sessions
(if you use the Runtime’s internal log database) -
./temp
(if your Processes store specific files in this directory)
-
Copy modules
Copy the modules from the previous installation to the new installation directory.
By default, modules are in the ./modules
subdirectory of the Runtime installation directory.
Configure the Runtime
Configure the new Runtime based on the previous Runtime’s configuration.
When upgrading the xDI Runtime to version 2024.x, you should pay attention to the following points:
-
The xDI Runtime built-in H2 database has been replaced by an HSQL database, only accessible from the runtime itself. See Built-in HSQL database and Migrate the built-in database from H2 to HSQL.
-
The xDI Runtime scheduler is now configured in the main runtime configuration file,
engineParameters.xml
. The legacyengineScheduler.properties
file is still supported but is no longer available by default. See Runtime scheduler.
Migrate the built-in database from H2 to HSQL
The xDI Runtime built-in H2 database was replaced by an HSQL database in version 2023.4.0.
If you are upgrading from a version prior to 2023.4.0, and you were using the built-in H2 database, you must migrate your H2 data to HSQL by hand if you want to keep your data. See [_migrate_the_built_in_database_from_h2_to_hsql_2]
Update the scheduler database
If you upgrade from a version before 2023.1.0, and you are using the xDI Runtime Scheduler, you must manually update the Scheduler database before starting the upgraded xDI Runtime.
The database structure has changed following an upgrade of the third-party library used for the Scheduler, which requires an update of the database structure.
Post-upgrade steps
Depending on the version of Semarchy xDI from which you upgrade, you need to perform actions after the upgrade procedure.
The actions are listed incrementally, please read all the sub-sections in this document until you reach your current Semarchy xDI product version. If no action is indicated in this document for your versions, then no specific action is required.
For example, when upgrading from S17 to 2023.1.x, install version 2023.1.x and follow the upgrade steps for S20, for 5.3.x, then for 2023.1.x. |
Generic steps
Once you have validated the new Runtime is running properly, perform the following operations:
-
Remove the old Runtime folder.
Upgrading from versions before 2023.2.0
The thread count for the runtime scheduler thread pool was changed from 50 to 3. If you are copying over a configuration, make sure you are not using the old value.
In the new engineParameters.xml
file, this number is found with the org.quartz.threadPool.threadCount
parameter.
Upgrading from versions before 2023.1.0
If you are upgrading from a version before 2023.1.0:
-
Switch from RMI to HTTP:
-
Replace all connections to the xDI Runtime from RMI to HTTP protocol.
-
Replace
rmi(s)://hostname:port
withhttp(s)://hostname:port
in all applicable locations.
If you were accessing the runtime without specifying the protocol, HTTP is now the default protocol used. In this situation, you only need to change the port. -
Replace the RMI port with the HTTP port. The default port for HTTP is
42200
.
-
-
Fine below a list of the Semarchy xDI locations where this must be changed:
-
xDI Analytics Runtime Configuration.
-
Runtime Start Delivery process action.
-
Runtime connect and schedule delivery command-lines.
-
Runtime executeRemoteCommand(s) scripting functions.
-
-
-
Update Jython scripts:
The default Jython third-party library shipped with xDI Runtime has been upgraded.-
The related
jythonVersion
Runtime parameter has been commented by default for the Runtime to use the shipped version by default. -
The new Jython version shipped replaces the
javos
with theos
module. Scripts using that module must be updated accordingly.
-
Upgrading from version S17
If you are upgrading from version S17, follow the upgrade steps of all the major intermediate versions between your version and 2023.1.x version.
Appendix
Update the scheduler database structure
Summary
If you upgrade from a version before 2023.1.0, and you are using the xDI Runtime Scheduler, you must manually update the scheduler database before starting the upgraded xDI Runtime.
The database structure has changed following an upgrade of the third-party library used for the Scheduler, which mandates a manual update of the database structure.
The procedure differs if you are using the built-in database or an external database.
Update the built-in scheduler database
If you were using the default, built-in H2 scheduler database, database structure is upgraded when you migrate from H2 to HSQL. See Migrate the built-in database from H2 to HSQL.
Update an external scheduler database
If you are using an external Scheduler database:
-
Make sure the xDI Runtime is not running.
-
Back up the database.
-
Update the database by running the SQL script corresponding to your provider. The scripts are located in
runtime/scripts/scheduler/upgrade/2023.1.0/upgrade_tables_<database-provider>.sql
.Run this script on the external database using Semarchy xDI Designer or an external query tool.
Migrate the built-in database from H2 to HSQL
Summary
The Semarchy xDI Runtime built-in H2 database has been replaced by an HSQL database, only accessible from the runtime itself.
-
If you were using the built-in H2 database for quickstart or temporary purposes and don’t mind about losing the quickstart data, we recommend using the new built-in HSQL database directly without migrating the old data.
-
If you were using the built-in H2 database as a production database for the session logs and scheduler:
-
As a reminder, the built-in database is provided for quickstart purposes as a convenience and is NOT recommended for production use. We highly recommend using one of the certified database servers instead. See Built-in HSQL database.
-
If you want to continue using the built-in database and keep the previous data, you must manually migrate the H2 built-in database data to the new HSQL built-in database, as explained in the guide below.
-
The built-in HSQL database is, by default, only accessible from the runtime itself, and is NOT recommended for production use. For production use, we recommend using the certified database servers. |
In the migration guide, source runtime refers to the previous runtime version that contains the built-in H2 database, and target runtime refers to the new runtime version with HSQL as the built-in database. |
Pre-migration steps
-
Backup the
scheduler
andsessions
folders from the source runtime directory. -
With Semarchy xDI Designer:
-
Create an H2 module named h2_legacy_driver.
-
Add version 1.2.140 of the H2 JDBC driver to the module.
-
-
Prepare the migration folder:
-
Download the Migration Tool.
-
Unzip the archive file at the location of your choice in the file system of the machine hosting the target runtime. It will be used to migrate the database from H2 to HSQL.
-
Copy the previously created h2_legacy_driver module folder to the
xdi-built-in-database-migration/modules
folder. -
Copy the
scheduler
andsessions
folders from the source runtime to thexdi-built-in-database-migration/source_data folder
.
-
-
Prepare the target runtime:
-
Copy the previously created h2_legacy_driver module folder to the
modules
folder of the target runtime. -
Copy the sessionLogs_migration.deliv and sessionLogs_migration.deliv files from the
xdi-built-in-database-migration/deliveries
folder to thebuild\deliveries
folder of the target runtime. -
Ensure the target runtime configuration (engineParameters.xml file) matches the following prerequisites:
-
The temporary folder must be configured with the default value
temp
. -
The
admin
user must exist withadmin-password
as the password. It is required by the migration script to stop the runtime.
-
-
Known Limitations:
|
Migration steps
Open a prompt command and run the startMigration.bat
(Windows) or startMigration.sh
(Linux) script.
The following arguments are available:
-
Common arguments:
Arguments Description runtimePath <path>
The target runtime path. This argument is mandatory.
The target runtime path must not contain any spaces or special characters.
deliveryLogLevel <logLevel>
Set the log level used by the migration deliveries. The default value is 0 if this parameter is not set.
-
Scheduler database arguments:
Arguments Description schedulerMigration
When this argument is defined, the script migrates the scheduler database from H2 to HSQL.
Only one of the
schedulerMigration
orsessionLogsMigration
argument must be used at the same time.deleteHsqlScheduler
Deletes the target HSQL scheduler database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.
This argument can be used only when migrating the scheduler database with
schedulerMigration
.h2SchedulerLogin <login>
The login of the source H2 scheduler database. To specify an empty value, omit this argument.
h2SchedulerPassword <password>
The password of the source H2 scheduler database. To specify an empty value, omit this argument.
hsqlSchedulerLogin <login>
The login of the target HSQL scheduler database. To specify an empty value, omit this argument.
hsqlSchedulerPassword <password>
The password of the target HSQL scheduler database. To specify an empty value, omit this argument.
schedulerInstanceId <instanceId>
The target scheduler instance identifier. It must be equal to the
org.quartz.scheduler.instanceId
parameter of the engineParameters.xml` file.The default value is
RUNTIME_HSQL_STD
if this parameter is not set.schedulerInstanceName <instanceName>
The target scheduler instance name. It must be equal to the
org.quartz.scheduler.instanceName
parameter of the engineParameters.xml`file.The default value is
RUNTIME_HSQL_STD
if this parameter is not set. -
Sessions logs database arguments:
Arguments Description sessionLogsMigration
When this argument is defined, the script migrates the session log database from H2 to HSQL.
Only one of the
schedulerMigration
orsessionLogsMigration
argument must be used at the same time.deleteHsqlSessionLogs
Deletes the target HSQL session logs database prior to the migration. Use this argument if you want to ensure that the target database is emptied before the migration.
This argument can be used only when migrating sessions logs with
sessionLogsMigration
.h2SessionLogsLogin <login>
The login of the source H2 session logs database. To specify an empty value, omit this argument.
h2SessionLogsPassword <password>
The password of the source H2 session logs database. To specify an empty value, omit this argument.
hsqlSessionLogsLogin <login>
The login of the target HSQL session logs database. To specify an empty value, omit this argument.
hsqlSessionLogsPassword <password>
The password of the target HSQL session logs database. To specify an empty value, omit this argument.
Session logs database migration example:
Windows:
startMigration.bat -runtimePath C:\Semarchy\runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password
Linux:
bash startMigration.sh -runtimePath /Semarchy/runtime -sessionLogsMigration -deleteHsqlSessionLogs -h2SessionLogsLogin sa -hsqlSessionLogsLogin backend-user -hsqlSessionLogsPassword backend-password
Scheduler database migration example:
Windows:
startMigration.bat -runtimePath C:\Semarchy\runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD
Linux:
bash startMigration.sh -runtimePath /Semarchy/runtime -schedulerMigration -deleteHsqlScheduler -h2SchedulerLogin sa -hsqlSchedulerLogin backend-user -hsqlSchedulerPassword backend-password -schedulerInstanceId RUNTIME_HSQL_STD -schedulerInstanceName RUNTIME_HSQL_STD