Configure Runtime logging
When using Semarchy xDI Runtime, you can log two kinds of information:
- Session information
-
Data and messages that a runtime generates when executing deliveries.
- Debug information
-
Supplementary technical data and messages that a runtime generates at any time while it is running.
A runtime stores these logs in two places.
First, a runtime always stores session information to a log database, which you can monitor from xDI Designer, xDI Analytics, or the command line. You can change which database to use. The default log database is an HSQL database built into xDI Runtime which provides a ready-to-use configuration.
xDI Runtime also uses the Log4j library to store both session and debug information to multiple locations. Configure Log4j settings to change storage locations, as well as how much information you want to log. You can monitor these logs with external monitoring tools
The built-in HSQL database is, by default, only accessible from the runtime. You should not use it for production use. Use a certified database server instead. |
Configure session logging to a database
The main session logging settings are in the main xDI Runtime configuration file engineParameters.xml
. This file is in the properties
subdirectory of the Runtime installation directory by default.
The main configuration sections are in a <parameters>
element, inside the engineParameters
and logs
nodes.
Here is an overview of how you configure runtime session logging to a database:
-
Open the
engineParameters.xml
configuration file. -
Under
<parameters>
, find theengineParameters
and thelogs
elements. -
Set the log database parameters.
-
Restart the runtime.
Configure initial logging engine parameters
In the engineParameters
node of the engineParameters.xml
file, set two parameters to configure logging:
-
The
userLogDefaultName
parameter defines a name for a database configuration. Create and edit that configuration further in the configuration file, under<logs>
. -
The
defaultSessionLogLevel
parameter defines how much information the runtime logs by default.
Refer to the runtime parameters reference for detailed information parameters.
Log levels
The defaultSessionLogLevel
parameter defines how much information the runtime saves in a session. It applies only if the session does not set its log level elsewhere. Possible values are as follows:
logLevel | Running session | Successful session | Session with errors |
---|---|---|---|
-1 |
Not logged |
Not logged |
Not logged |
0 |
Full details |
Not logged |
Full details |
100 |
Full details |
Session header and statistics |
Full details |
200 |
Full details |
[logLevel 100] + Process information and statistics |
Full details |
300 |
Full details |
[logLevel 200] + Actions information and statistics |
Full details |
400 |
Full details |
Full details |
Full details |
<engineParameters>
<...>
<parameter name="userLogDefaultName" value="logDatabase"/>
<parameter name="defaultSessionLogLevel" value="400"/>
<...>
</engineParameters>
Set an external logging engine database
If you want to use the default HSQL database instead of an external database, see the section about HSQL. |
A log database configuration contains the connection information to a database that hosts the log. It must be inside the <logs>
element, in its own <log>
node.
To set a log database used by the runtime, follow these steps:
-
Copy a
<log>
element and its contents, or find a<log>
entry to modify. -
Set the
userLogName
attribute of the<log>
node to match the initialuserLogDefaultName
engine parameter. -
Set the JDBC URL and driver in
userLogRdbmsUrl
anduserLogRdbmsDriver
parameters. -
Set the database username and password in
userLogRdbmsUser
anduserLogRdbmsPassword
parameters. -
Set the
userLogSchemaName
parameter to the database schema you want to use. -
Set the
userLogRdbmsModule
parameter to the name of an xDI module containing the required JDBC database drivers.
You do not need to configure a database schema ahead of time to prepare it for logging. The runtime creates the necessary tables the first time it connects to the database.
Example configuration for an external database
The following configuration example uses a log database named oracleLogDatabase using an Oracle storage.
Adapt the following parameters to your database.
-
userLogRdbmsUrl
-
userLogRdbmsUser
-
userLogRdbmsPassword
-
userLogSchemaName
-
userLogRdbmsModule
Refer to the runtime parameters reference for detailed information about each parameter. |
<parameters>
<...>
<engineParameters>
<...>
<parameter name="userLogDefaultName"
value="oracleLogDatabase"/>
<parameter name="defaultSessionLogLevel"
value="400"/>
<...>
</engineParameters>
<...>
<logs>
<...>
<log userLogName="oracleLogDatabase"
autoUpdate="true"
userLogClass="com.semarchy.xdi.runtime.sessionlog.RdbmsLogger">
<parameter name="userLogRdbmsDriver"
value="oracle.jdbc.driver.OracleDriver"/>
<parameter name="userLogRdbmsUrl"
value="jdbc:oracle:thin:@[host]:[port]:[sid]"/>
<parameter name="userLogRdbmsUser"
value="[USER]"/>
<parameter name="userLogRdbmsPassword"
value="[PASSWORD]"/>
<parameter name="userLogRdbmsModule"
value="[MODULE_NAME]"/>
<parameter name="userLogRdbmsSchemaName"
value="[SCHEMA_NAME]"/>
<parameter name="userLogRdbmsVarcharType"
value="varchar2"/>
<parameter name="userLogRdbmsVarcharMaxSize"
value="4000"/>
<parameter name="userLogRdbmsClobType"
value="clob"/>
<parameter name="userLogRdbmsBlobType"
value="blob"/>
<parameter name="userLogRdbmsNumericType"
value="number"/>
<parameter name="userLogRdbmsDeleteSyntaxe"
value="Delete from"/>
<parameter name="userLogRdbmsDeliveryFormat"
value="text"/>
<parameter name="userLogRdbmsPropertyMaxVarcharSize"
value="1000"/>
<parameter name="userLogRdbmsPropertyMaxClobSize"
value="10000"/>
<parameter name="userLogRdbmsPropertyBinaryFormat"
value="compressed"/>
<parameter name="userLogRdbmsPoolEnabled"
value="true"/>
<parameter name="userLogRdbmsPoolConnectionTimeout"
value="30000"/>
<parameter name="userLogRdbmsPoolIdleTimeout"
value="600000"/>
<parameter name="userLogRdbmsPoolKeepAliveTime"
value="0"/>
<parameter name="userLogRdbmsPoolMaxLifetime"
value="1800000"/>
<parameter name="userLogRdbmsPoolMinimumIdle"
value="0"/>
<parameter name="userLogRdbmsPoolMaximumSize"
value="20"/>
<parameter name="userLogRdbmsPoolValidationTimeout"
value="5000"/>
</log>
<...>
</logs>
<...>
</parameters>
The default engineParameters.xml
installed with the runtime contains commented log database configuration samples. Uncomment and adapt these samples to configure to your log storage. Make sure to set the userLogDefaultName
engine parameter to the appropriate log database configuration.
Set or configure the built-in HSQL database
If you want to use an external database instead of the default HSQL database, see the section about external databases. |
The runtime ships a built-in HSQL quick start database. The default configuration uses this database to store logs for delivery execution sessions and the scheduler.
The built-in HSQL database is intended for development environments. It is only accessible from the runtime itself by default. You should use a certified database server where possible.
On runtime versions prior to 2023.4.0, the built-in database was an H2 database. See Migrate the built-in database from H2 to HSQL to migrate your data to HSQL. |
Set the HSQL directory
In its default configuration, the runtime creates an HSQL database inside a sessions
directory to hold logging information. The runtime tries to create the directory if it does not exist already. If an HSQL database already exists, make sure that the user and password in the configuration file match the ones expected by the database. This is most important when upgrading from a previous runtime version.
The directory name and location are set by the userLogRdbmsUrl
engine parameter. The following example is from a runtime that writes to the default sessions/internalDb/sessionLogs
location:
<parameter name="userLogRdbmsUrl" value="jdbc:hsqldb:file:internalDb/sessionLogs/sessionLogs"/>
The parameter can take a different file path, including absolute filesystem paths. Make sure that the directories you specify have write permissions enabled.
Open external access
The built-in HSQL database is only accessible from the runtime itself by default. The runtime logs to the database in-memory, and does not expose the database as an external service.
To open access to the built-in HSQL database from the outside, you must manually start it as a server. This can be useful to access its logs from xDI Analytics, for example. To do this:
-
Stop the runtime.
-
Update the runtime configuration
-
Log Database:
-
Change the
userLogRdbmsUrl
parameter value fromjdbc:hsqldb:file:internalDb/sessionLogs/sessionLogs
tojdbc:hsqldb:tcp://localhost:42100/sessions/internalDb/sessionLogs
-
-
Scheduler:
-
Change the
org.quartz.dataSource.internal.URL
parameter value fromjdbc:hsqldb:file:internalDb/scheduler/scheduler
tojdbc:hsql:tcp://localhost:42100/internalDb/scheduler/scheduler
-
-
-
Start the built-in HSQL database as a server.
-
According to your operating system, run the
start-database.bat
orstart-database.sh
from the runtime’s installation root folder.
-
-
Start the runtime.
After this change, you must always start the HSQL database server before starting the runtime itself.
Disable session logging
You can disable delivery session logging in the runtime in two ways:
-
Explicitly define a special, non-logging database entry using the
EmptyLogger
class as shown in the further example. Use this method first. -
Remove or comment out all log database configurations under the
<logs>
node. This stops the runtime from logging delivery sessions.
<parameters>
<...>
<engineParameters>
<...>
<parameter name="userLogDefaultName" value="noLogs"/>
<...>
</engineParameters>
<...>
<logs>
<log
userLogName="noLogs"
autoUpdate="false"
userLogClass="com.semarchy.xdi.runtime.sessionlog.EmptyLogger">
</log>
</logs>
<...>
</parameters>
Schedule log purges
You can schedule automatic purging of logs in Semarchy xDI Analytics, in the Purge tab of the Runtime Editor. You can also use the command line:
-
Run the
startCommand.bat|sh
script. -
Run the
schedule purge
command described as follows:
schedule purge keep <number> <minute|hour|day|session> cron <cronExpression> [sessionname <name,name2,...>] [status <done,error,killed>] [on host <hostname>] [port <hostport>]
For example:
schedule purge keep 90 day cron "0 0 23 * * ?"
Configure session and debug logging with Log4j
When xDI Runtime saves logs using Log4j, it reads settings from the XML file log4j.xml
. This file is in the properties
subdirectory of the Runtime installation directory by default.
The configuration file starts with a Configuration
element. Inside it, you can find Appenders
and Loggers
.
Appenders send logging information to its final destination, like a log file or a console. Loggers set which elements log4j records, and need at least one reference to an appender.
<Configuration shutdownHook="disable">
<Appenders>
<RollingFile fileName="log/xdi-runtime.log" filePattern="log/xdi-runtime.log.%i" immediateFlush="true" name="runtimeAppender">
<PatternLayout>
<Pattern>%d{dd/MM/yyyy HH:mm:ss,SSS} [%t] %c{10} - %-5p - %m%n%throwable</Pattern>
</PatternLayout>
<SizeBasedTriggeringPolicy size="20 MB"/>
<DefaultRolloverStrategy fileIndex="min" max="6"/>
</RollingFile>
</Appenders>
<Logger name="com.indy.engine.user.console" level="all">
<AppenderRef ref="consoleAppender"/>
</Logger>
<Logger name="com.semarchy.xdi.runtime.httpserver" level="warn">
<AppenderRef ref="consoleAppender"/>
<AppenderRef ref="runtimeAppender"/>
</Logger>
</Configuration>
Aside from the loggers built into the Log4j library, xDI Runtime implements some of its own, among which:
-
Session start
-
Session end
-
Action start
-
Action end
-
Java compiler
-
Module service
-
HTTP server
The default configuration disables some of these loggers. You can re-enable them.
Change the log location
By default, the Log4j library writes to the xdi-runtime.log
file in a log
subdirectory of the runtime installation path. If this subdirectory does not exist, the runtime tries to create it. The parameters that set the directory are in the log4j.xml
file, as attributes of the RollingFile
appender:
<RollingFile fileName="log/xdi-runtime.log" filePattern="log/xdi-runtime.log.%i" immediateFlush="true" name="runtimeAppender">
The attributes can take a different folder and file path, including absolute filesystem paths. If the path points to external directories, make sure those directories have write permissions enabled.
Logs can also be sent to other places, such as the runtime console using the Console
appender.
Set appender patterns
When writing to disk, appenders determine how to write the log to disk from a Layout
. One such layout is the PatternLayout
, which sets string patterns to enrich log messages with additional information. The pattern strings have their own internal syntax and pattern parameters.
xDI has pattern parameters of its own. To use them, add them with the syntax %X{parameterName}
.
You can also use %X
on its own to get all the parameters.
<PatternLayout>
<Pattern>%d{dd/MM/yyyy HH:mm:ss,SSS} [%t] %c{10} - %-5p - %m%n%throwable</Pattern>
</PatternLayout>
List of xDI patterns
Parameter Name | Description |
---|---|
version |
Version of the logging system |
type |
Type of event |
message |
event message |
sessionId |
Id of the session |
sessionIteration |
Iteration of the session |
sessionName |
Name of the session |
Parameter Name | Description |
---|---|
sessionStatus |
Status of the session (always |
sessionParentId |
Id of the parent session |
sessionBeginDate |
Start date of the session |
processId |
Id of the process |
processName |
Name of the process |
deliveryId |
Id of the delivery |
deliveryName |
Name of the delivery |
deliveryBuildDate |
Build date of the delivery |
deliveryBuildUser |
Build user of the delivery |
configuration |
Configuration used for the execution |
runtimeHost |
Hostname of the runtime used for the execution |
runtimePort |
Port of the runtime used for the execution |
executionMode |
Mode of execution (MEMORY/AUTONOMOUS) |
launchMode |
Launch mode (RESTART/COMMAND_LINE/SCHEDULE/DESIGNER/ACTION/WEB_INTERACTIVE/WEB_SERVICE/ENGINE_COMMAND) |
sessionLogLevel |
Log level of the session |
maxparallelExecutions |
Max parallel executions setting |
autoRestartInterval |
Auto restart interval setting |
autoRestartMaxTries |
Auto restart max tries setting |
launchUserName |
Name of the user who launched the session |
Parameter Name | Description |
---|---|
sessionStatus |
Status of the session |
sessionErrorMessage |
Error message of the session |
sessionEndDate |
End date of the session |
sessionDuration |
Total duration of the session |
var.<varName> |
Variables |
stat.<aggregateName> |
Statistics |
Parameter Name | Description |
---|---|
actionId |
ID of the action |
actionIteration |
Iteration of the action |
actionType |
Type of the action (Code/Process) |
actionPath |
Path of the action |
actionPathWithIteration |
Unique identifier of the action (<path>_<iteration>) |
actionStatus |
Status of the action ( |
Parameter Name | Description |
---|---|
actionBeginDate |
Start date of the action |
actionParentID |
ID of the parent session |
actionParentIteration |
Iteration of the parent session |
actionIsBegin |
Is the action |
actionNumber |
Unique number of the action |
Parameter Name | Description |
---|---|
actionErrorMessage |
Error message of the Action [NOTE] This is not for errors that occur during action executions. It is for all other action errors, such as if actions did not generate. |
actionEndDate |
End date of the action |
actionDuration |
Total duration of the action |
actionBindIteration |
Final bind iteration of the action |
actionExecutionCount |
Execution count of the action (when used in a loop) |
var.<varName> |
Variables |
stat.<aggregateName> |
Statistics |
For more on patterns and layouts, see the official Log4j documentation.