Configure Runtime session logging

The Runtime stores session execution information in a log database.

The default log database is an HSQL database built into the runtime, and provides a ready-to-use quickstart configuration.

The built-in HSQL database is, by default, only accessible from the runtime itself, and is NOT recommended for production use.

For production use, we recommend using the certified database servers.

Configure session logging

To configure the runtime log storage:

Edit the engineParameters.xml file.
Configure logging.
Set the log database parameters.
Restart the runtime.

Configuration example

The following configuration example uses a log database named logDatabase, configured to use the runtime built-in HSQL quickstart database.

The built-in HSQL database is, by default, only accessible from the runtime itself, and is NOT recommended for production use. See Built-in HSQL database.

<parameters>
  <...>
  <engineParameters>
    <...>
      <parameter name="userLogDefaultName"
        value="logDatabase"/>
      <parameter name="defaultSessionLogLevel"
        value="400"/>
      <...>
  </engineParameters>
  <...>
  <logs>
    <...>
      <log userLogName="logDatabase"
        autoUpdate="true"
        userLogClass="com.semarchy.xdi.runtime.sessionlog.RdbmsLogger">
        <parameter name="userLogRdbmsDriver"
          value="org.hsqldb.jdbcDriver"/>
        <parameter name="userLogRdbmsUrl"
          value="jdbc:hsqldb:file:internalDb/sessionLogs/sessionLogs"/>
        <parameter name="userLogRdbmsUser"
          value="backend-user"/>
        <parameter name="userLogRdbmsPassword"
          value="backend-password"/>
        <!-- Use the following paramater to specify an encrypted password. -->
        <!--<parameter name="userLogRdbmsEncryptedPassword"
        value=""/>-->
        <parameter name="userLogRdbmsModule"
          value="HSQL"/>
        <parameter name="userLogRdbmsVarcharType"
          value="varchar"/>
        <parameter name="userLogRdbmsVarcharMaxSize"
          value="1000"/>
        <parameter name="userLogRdbmsNumericType"
          value="numeric"/>
        <parameter name="userLogRdbmsClobType"
          value="clob"/>
        <parameter name="userLogRdbmsBlobType"
          value="blob"/>
        <parameter name="userLogRdbmsSchemaName"
          value="logs"/>
        <parameter name="userLogRdbmsUseSchemaNameForIndexCreation"
          value="true"/>
        <parameter name="userLogRdbmsDeleteSyntaxe"
          value="Delete from"/>
        <!--<parameter name="userLogRdbmsTablePrefix"
          value="SEM_RT_LOG_"/>-->
        <parameter name="userLogRdbmsCompressedLevel"
          value="bestCompression"/>
        <!-- bestCompression / bestSpeed / default-->
        <parameter name="userLogRdbmsDeliveryFormat"
          value="compressed"/>
        <!-- text / binary / compressed-->
        <parameter name="userLogRdbmsPropertyMaxVarcharSize"
          value="1000"/>
        <!-- -1 : infinite -->
        <parameter name="userLogRdbmsPropertyMaxClobSize"
          value="10000"/>
        <parameter name="userLogRdbmsPropertyBinaryFormat"
          value="compressed"/>
        <!-- binary / compressed-->
        <parameter name="userLogRdbmsTimestampQuery"
          value="SELECT DISTINCT NOW() FROM INFORMATION_SCHEMA.SYSTEM_SCHEMAS"/>
        <parameter name="userLogRdbmsInactivityDetectionPeriod"
          value="90000"/>
        <parameter name="userLogRdbmsActivityRefreshInterval"
          value="60000"/>
        <parameter name="userLogRdbmsDeletionWithNotExists"
          value="true"/>
        <!--<parameter name="userLogRdbmsIndexCreationOption"
          value=""/>-->
        <!--<parameter name="userLogRdbmsTableCreationOption"
          value=""/>-->
        <parameter name="userLogRdbmsPoolEnabled"
          value="true"/>
        <parameter name="userLogRdbmsPoolConnectionTimeout"
          value="30000"/>
        <parameter name="userLogRdbmsPoolIdleTimeout"
          value="600000"/>
        <parameter name="userLogRdbmsPoolKeepAliveTime"
          value="0"/>
        <parameter name="userLogRdbmsPoolMaxLifetime"
          value="1800000"/>
        <parameter name="userLogRdbmsPoolMinimumIdle"
          value="0"/>
        <parameter name="userLogRdbmsPoolMaximumSize"
          value="20"/>
        <parameter name="userLogRdbmsPoolValidationTimeout"
          value="5000"/>
      </log>
      <...>
  </logs>
  <...>
</parameters>

Configure logging

In the engineParameters node of the engineParameters.xml file, two parameters are used to configure logging:

The userLogDefaultName parameter defines a name for the database configuration you use for logging. A further log database configuration must exist under the <logs> node.
The defaultSessionLogLevel parameter defines what session information is logged by default.

Refer to the runtime parameters reference for detailed information parameters discussed in this section.

Log levels

The defaultSessionLogLevel parameter defines how much information will be logged for a session. It applies only if a given session does not already have a log level set elsewhere. Possible values are: –1, 0, 100, 200, 300, and 400. A higher value means that more information is logged, as follows:

Table 1. Logged information by level
logLevel	Running session	Successful session	Session with errors
-1	Not logged	Not logged	Not logged
0	Full details	Not logged	Full details
100	Full details	Session header and statistics	Full details
200	Full details	[logLevel 100] + Process information and statistics	Full details
300	Full details	[logLevel 200] + Actions information and statistics	Full details
400	Full details	Full details	Full details

Example 1. Sample logging configuration

  <engineParameters>
    <...>
    <parameter name="userLogDefaultName" value="logDatabase"/>
    <parameter name="defaultSessionLogLevel" value="400"/>
    <...>
  </engineParameters>

Log database configuration

A log database configuration contains the connection information to a database that hosts the log. It is found under the <logs> node.

To select the log database used by the runtime, the userLogName attribute of the <log> node needs to match the userLogDefaultName engine parameter defined previously.

The following example shows a runtime configuration which writes logs to an Oracle database named productionLogDatabase by specifying the Oracle JDBC driver and a JDBC URL.

<engineParameters>
  <...>
  <parameter name="userLogDefaultName" value="productionLogDatabase"/>
  <...>
</engineParameters>
  <...>
  <logs>
  <...>
    <log userLogName="productionLogDatabase" autoUpdate="true" userLogClass="com.semarchy.xdi.runtime.sessionlog.RdbmsLogger">
      <parameter name="userLogRdbmsDriver" value="oracle.jdbc.driver.OracleDriver"/>
      <parameter name="userLogRdbmsUrl" value="jdbc:oracle:thin:@prod001.acme.com:1521:ORA"/>
    <...>

You do not need to configure anything in a database ahead of time to prepare it for logging. All the necessary tables will automatically be created the first time the runtime connects to the database.

For the runtime to communicate with a database other than the built-in HSQL database, the log database configuration must contain a userLogRdbmsModule parameter with the name of the module containing the required JDBC database drivers.

External database configuration example

The following configuration example uses a log database named oracleLogDatabase using an Oracle storage.

The following parameters must be adapted to the database used.

userLogRdbmsUrl
userLogRdbmsUser
userLogRdbmsPassword
userLogSchemaName
userLogRdbmsModule

Refer to the runtime parameters reference for detailed information about each parameter.

<parameters>
  <...>
  <engineParameters>
  <...>
    <parameter name="userLogDefaultName"
      value="oracleLogDatabase"/>
    <parameter name="defaultSessionLogLevel"
      value="400"/>
    <...>
  </engineParameters>
  <...>
  <logs>
    <...>
    <log userLogName="oracleLogDatabase"
      autoUpdate="true"
      userLogClass="com.semarchy.xdi.runtime.sessionlog.RdbmsLogger">
      <parameter name="userLogRdbmsDriver"
        value="oracle.jdbc.driver.OracleDriver"/>
      <parameter name="userLogRdbmsUrl"
        value="jdbc:oracle:thin:@[host]:[port]:[sid]"/>
      <parameter name="userLogRdbmsUser"
        value="[USER]"/>
      <parameter name="userLogRdbmsPassword"
        value="[PASSWORD]"/>
      <parameter name="userLogRdbmsModule"
        value="[MODULE_NAME]"/>
      <parameter name="userLogRdbmsSchemaName"
        value="[SCHEMA_NAME]"/>
      <parameter name="userLogRdbmsVarcharType"
        value="varchar2"/>
      <parameter name="userLogRdbmsVarcharMaxSize"
        value="4000"/>
      <parameter name="userLogRdbmsClobType"
        value="clob"/>
      <parameter name="userLogRdbmsBlobType"
        value="blob"/>
      <parameter name="userLogRdbmsNumericType"
        value="number"/>
      <parameter name="userLogRdbmsDeleteSyntaxe"
        value="Delete from"/>
      <parameter name="userLogRdbmsDeliveryFormat"
        value="text"/>
      <parameter name="userLogRdbmsPropertyMaxVarcharSize"
        value="1000"/>
      <parameter name="userLogRdbmsPropertyMaxClobSize"
        value="10000"/>
      <parameter name="userLogRdbmsPropertyBinaryFormat"
        value="compressed"/>
      <parameter name="userLogRdbmsPoolEnabled"
        value="true"/>
      <parameter name="userLogRdbmsPoolConnectionTimeout"
        value="30000"/>
      <parameter name="userLogRdbmsPoolIdleTimeout"
        value="600000"/>
      <parameter name="userLogRdbmsPoolKeepAliveTime"
        value="0"/>
      <parameter name="userLogRdbmsPoolMaxLifetime"
        value="1800000"/>
      <parameter name="userLogRdbmsPoolMinimumIdle"
        value="0"/>
      <parameter name="userLogRdbmsPoolMaximumSize"
        value="20"/>
      <parameter name="userLogRdbmsPoolValidationTimeout"
        value="5000"/>
    </log>
    <...>
  </logs>
  <...>
</parameters>

The default engineParameters.xml installed with the runtime contains commented log database configuration samples. Uncomment and adapt these samples to configure to your log storage. Make sure to set the userLogDefaultName engine parameter to the right log database configuration.

Disable session logging

You can disable session logging in the runtime in two ways:

Explicitly define a special, non-logging database entry using the EmptyLogger class as shown in the further example. This method is recommended.
Remove or comment out all log database configurations under the <logs> node. Without a valid log database configuration, the runtime will not log any sessions.

Example 2. Configure the runtime for no logging

<parameters>
  <...>
  <engineParameters>
    <...>
    <parameter name="userLogDefaultName" value="noLogs"/>
    <...>
  </engineParameters>
  <...>
  <logs>
    <log
    userLogName="noLogs"
    autoUpdate="false"
    userLogClass="com.semarchy.xdi.runtime.sessionlog.EmptyLogger">
    </log>
  </logs>
  <...>
</parameters>

Scheduled log purges

You can schedule automatic purges of the session logs, in Semarchy xDI Analytics in the Purge tab of the Runtime Editor or using the command line as explained below.

To schedule a log purge:

Run the startCommand.bat|sh script.
Run the schedule purge command described below

schedule purge keep <number> <minute|hour|day|session>  cron <cronExpression> [sessionname <name,name2,...>] [status <done,error,killed>] [on host <hostname>] [port <hostport>]

For example:

schedule purge keep 90 day cron "0 0 23 * * ?"

Debug logs

The runtime also stores debug logs in the log/ subdirectory of its installation folder.
By default, the debug logs are rotating to keep a reasonable volume of debug logs.

These logs are configured using the log4j.xml file.

For help configuring debug logs to troubleshoot issues, contact the Semarchy support team.

Change the log directories

If they do not exist already, the runtime will create two directories in its installation folder to store logging information. You can change which directories are used to store the logs.

Session directory

In its default configuration, the runtime creates an HSQL database inside a sessions directory to hold logging information. The directory is defined in the userLogRdbmsUrl engine parameter. The following example is from a runtime that writes to the default sessions/internalDb/sessionLogs location:

<parameter name="userLogRdbmsUrl" value="jdbc:hsqldb:file:internalDb/sessionLogs/sessionLogs"/>

The parameter can take a different file path, including absolute file system paths. Make sure that the directories you specify have write permissions enabled.

For more information about the HSQL database, see the Appendix.

Debug log directory

The log directory is used for runtime debug logs, as previously mentioned in the Debug logs section. The directory is defined in the log4j.xml file as an attribute of the <RollingFile> node, as in the following example:

<RollingFile fileName="log/xdi-runtime.log" filePattern="log/xdi-runtime.log.%i" immediateFlush="true" name="runtimeAppender">

The node attributes can take a different folder and file path, including absolute file system paths. If the path points to different, pre-existing directories instead of the default log, make sure those directories have write permissions enabled.

Appendix

Built-in HSQL database

Summary

The runtime ships a built-in HSQL quickstart database that is used with the default configuration for the session log database and scheduler database.

The built-in HSQL database is, by default, only accessible from the runtime itself, and is NOT recommended for production use.

For production use, we recommend using the certified database servers.

On runtime versions prior to 2023.4.0, the built-in database was an H2 database. See Migrate the built-in database from H2 to HSQL to migrate your data to HSQL, if necessary.

Database configuration

The built-in HSQL database is created at the first runtime’s startup, with the user and password defined in the runtime’s configuration file. See Configuration example.

If you start the runtime with an already existing database, make sure that the user and password defined in the runtime configuration match the user and password expected by the database. This is most notably important when upgrading from a previous runtime version.

Open external access

As mentioned previously, the built-in HSQL database is, by default, only accessible from the runtime itself. The runtime logs into the database in memory, and it isn’t exposed as a server accessible from the outside.

To open access to the built-in HSQL database from the outside, it must be manually started as a server. This can be useful to access the sessions logs from xDI Analytics, for example. To do this:

Stop the runtime.
Update the runtime configuration
1. Log Database:
  1. Change the userLogRdbmsUrl parameter value from jdbc:hsqldb:file:internalDb/sessionLogs/sessionLogs to jdbc:hsqldb:tcp://localhost:42100/sessions/internalDb/sessionLogs
2. Scheduler:
  1. Change the org.quartz.dataSource.internal.URL parameter value from jdbc:hsqldb:file:internalDb/scheduler/scheduler to jdbc:hsql:tcp://localhost:42100/internalDb/scheduler/scheduler
Start the built-in HSQL database as a server.
1. According to your operating system, run the start-database.bat or start-database.sh from the runtime’s installation root folder.
Start the runtime.

After this change, the runtime requires the built-in HSQL database server to be running prior to being started. Make sure to start the built-in HSQL database prior to the runtime.