Welcome to Semarchy xDM.
This guide contains information about installing Semarchy xDM to design and develop an MDM project.

Preface

Audience

This document is intended for administrators and project managers interested in installing the Semarchy xDM for their Enterprise Master Data Management Initiatives.

If you want to learn about MDM or discover Semarchy xDM, you can watch our tutorials.
The Semarchy xDM Documentation Library, including the development, administration and installation guides is available on-line.

Document Conventions

This document uses the following formatting conventions:

Convention Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or a product specific term or concept.

italic

Italic type indicates special emphasis or placeholder variable that you need to provide.

monospace

Monospace type indicates code example, text or commands that you enter.

Other Semarchy Resources

In addition to the product manuals, Semarchy provides other resources available on its web site: http://www.semarchy.com.

Obtaining Help

There are many ways to access the Semarchy Technical Support. You can call or email our global Technical Support Center (support@semarchy.com). For more information, see http://www.semarchy.com.

Feedback

We welcome your comments and suggestions on the quality and usefulness of this documentation.
If you find any error or have any suggestion for improvement, please mail support@semarchy.com and indicate the title of the documentation along with the chapter, section, and page number, if available. Please let us know if you want a reply.

Overview

Using this guide, you will learn how to:

  • Plan the configuration of Semarchy xDM for development and production environments.

  • Start and connect to Semarchy xDM.

Introduction to Semarchy xDM

What is Semarchy xDM?

Semarchy xDM is designed to support any kind of Enterprise Master Data Management initiative. It brings an extreme flexibility for defining and implementing master data models and releasing them to production. The platform can be used as the target deployment point for all master data of your enterprise or in conjunction with existing data hubs to contribute to data transparency and quality with federated governance processes. Its powerful and intuitive environment covers all use cases for setting up a successful master data governance strategy.

Semarchy xDM is based on a coherent set of features for all Master Data Management projects.

Architecture Overview

The Semarchy xDM architecture includes:

  • The Semarchy xDM Application: This JEE application is deployed and runs in an application server, and stores its information in a repository.

  • The Repository that stores all the MDM projects metadata and execution logs. This repository is hosted in a database schema. A given Semarchy xDM Application is attached to a single repository, and users connecting to this applications access only the content of this repository.

  • The Data Locations. Each data location contains a data hub. This hub contains the golden data, the source data pushed by publishers in the hub, and the various stages of this data in the certification process. A data location is hosted in a database schema. Several data locations can be attached to a single repository.

  • The Semarchy xDM Workbench: This web 2.0 application is served from the Semarchy xDM Application and runs in a web browser.

Preparing to Install

Review the information in this section before you begin your installation.

System Requirements and Certification

Before installing Semarchy xDM, you should read the system requirements and certification documents to ensure that your environment meets the minimum installation requirements.

Hardware Requirements

The Semarchy xDM server runs as an application in a supported application server. The hardware requirements are those of the application Server.
Refer to your application server documentation for more information about the required hardware requirements.

Semarchy xDM runs on physical machines or virtual machines (VM).

Software Requirements

This section contains a list of software requirements for this release of Semarchy xDM.

It is recommended to ensure that all software involved in the installation has current patch applied.
Server Requirements
Java

Supported Java Runtime Environment (JRE) or Development Kit (JDK) versions are:

  • Java 8 - version 1.8.0 or above.

The JAVA_HOME (for a JDK) or JRE_HOME (for a JRE) environment variable must be configured to point to this installation of Java.

Application Server

Semarchy xDM Server is a JEE6 web application certified with the following application servers:

  • Apache Tomcat 7.0.x, 8.0.x and 8.5.x

  • Eclipse Jetty 9.x

  • IBM Websphere Liberty Profile 17.x

  • Wildfly 8.x, 9.x, 10.x

  • GlassFish 4.x

  • Oracle WebLogic 12c Release 1 (12.1.1, 12.1.2, 12.1.3), Oracle WebLogic 12c Release 2 (12.2.1.3.0)

Database

Supported database versions for the repository and the data locations are:

  • Oracle Database 10g Release 2 - 10.2.0.1-10.2.0.5

  • Oracle Database 11g Release 1: 11.1.0.6–11.1.0.7

  • Oracle Database 11g Release 2: 11.2.0.1–11.2.0.4

  • Oracle Database 12c Release 1: 12.1.0.1-12.1.0.2

  • Oracle Database 12c Release 2: 12.2.0.1

In addition, the following databases are supported as a Preview Release (Not for Production) for the repository and data locations:

  • PostgreSQL version 9: 9.5.10-9.6.6 (Not for Production)

  • PostgreSQL version 10: 10.1.x (Not for Production)

PostgreSQL Preview Release

The support for PostgreSQL is available as a Preview. This means that the software is feature complete but likely to contain a number of known or unknown bugs, as well as speed/performance issues. The goal of the preview release is to enable feature testing among a wider user base likely to detect errors that regular tests might not raise.

With this preview release, PostgreSQL is not supported for production environments or with production data. Other non-production environements installed with the preview release on PostgreSQL are supported by our support team.

Getting the Oracle Database

Oracle Database can be downloaded for free for Linux and Windows at the following URL: http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index-092322.html

Semarchy xDM works with Oracle Express Edition. It also is possible to use with any other edition of Oracle (Standard or Enterprise). These are free to use for the purpose of developing, testing and prototyping. Note also that Amazon Web Services offers Oracle as part of Cloud Relational Database Service.
Getting the PostgreSQL Database

The PostgreSQL Database can be downloaded for free for Linux, Unix, Windows, MacOs and other platforms at the following URL: https://www.postgresql.org/download/

Client Requirements

Supported browsers for the Semarchy xDM Workbench are:

  • Internet Explorer 11 and later

  • Google Chrome 58 and later

  • Firefox 40 and later

  • Safari 10 and later (macOS)

  • Microsoft Edge 38 and later

Supported browsers for the MDM applications are:

  • Google Chrome 58 and later (incl. Chrome for Android)

  • Firefox 40 and later

  • Microsoft Edge 38 and later with known issues

  • Safari 10 and later (macOS and iOS), with known issues

For browsers indicated "with known issues", please check the release notes and contact support for an updated list of issues.

License

To complete the installation and configuration of Semarchy xDM, you will be requested for a license file.
Please contact support@semarchy.com if you run into licensing issues or connect to http://www.semarchy.com/download/ to request a trial license of Semarchy xDM.

Make sure you have the license file provided by Semarchy before starting the installation.

Planning the Installation

Architecture Details

This section details the various components of the Semarchy xDM architecture and their interactions.

Application

The Semarchy xDM application is a Java EE application deployed and running in a supported application server.

This application provides several access methods:

  • Users access it via their web browser using the Semarchy Workbench user interface or generated Applications.

  • Applications access the platform services and MDM hub data via integration points.

The Semarchy xDM application stores its information in a repository. One application is always attached to a single repository, and connects this repository using a JDBC datasource named SEMARCHY_REPOSITORY configured in the application server.

The Semarchy xDM application is used at design-time to design data models and deploy them. At run-time, it manages the processes involved to schedule and execute the certification process in the hub.

The processes are managed by:

  • The Integration Batch Poller that polls for new data batches pushed in the hub by the publishing applications.

  • The Execution Engine that orchestrates the certification process to generate the golden data.

The application uses role-based security for accessing Semarchy xDM features. The users and roles used to connect to the application must be defined in the security realm of the application server. Configuring the roles and users is part of the application configuration.

Repository

The repository stores the following information:

  • The models: entities, attributes, etc.

  • The models version control information: branches, editions, etc.

  • The configuration & security information: roles, privileges, notification servers, notification policies, preferences, etc.

  • Data locations information: deployed model, job definitions, etc.

  • Run-time information: queues, logs

A repository is stored in an database schema accessed from the application using a JDBC datasource named SEMARCHY_REPOSITORY.

The repository should never be accessed directly via SQL queries. Access to the Semarchy xDM information must be performed through the Semarchy Workbench user interface provided by the application.
Repository Types

There are two types of repositories. The repository type is selected at creation time and cannot be modified afterwards.

The repository types are:

  • Design: All design-time and run-time operations are possible in this repository.

  • Deployment: With this repository type, you can only import closed model editions and cannot edit them.

The deployment repositories are suitable for production sites. Model transfer from design to deployment repositories is handled via incremental export/import of closed model editions.

Data Locations

When a MDM hub must be available for run-time (for testing or production purposes), it is generated from a data model defined in the repository, and deployed in a data location. Data Locations contain the deployed data hubs. Each hub contains the golden data, the source data pushed by publishers in the hub, and the various stages of this data in the certification process.

The data location is hosted in an database schema and accessed via a JDBC datasource defined in the application server. A data location refers to the datasource via its JNDI URL.

Data Location Contents

A Data Location contains the hub data, stored in the schema accessed using the data location’s datasource. This schema contains database tables and other objects generated from the model edition.

The data location also refers three type of jobs (stored in the repository):

  • Installation Jobs: The jobs for creating or modifying in a non-destructive way the data structures in the schema.

  • Integration Jobs: The jobs for certifying data in these data structures, according to the model job definitions.

  • Purge Jobs: The jobs for purging the logs and data history according to the retention policies.

Data Locations, Repositories and Models

A data location is attached to a repository: You can declare as many data locations as you want in a repository, but a data location is always attached to a single repository. It is not possible to have a data location attached to two repositories at the same time.

You may deploy several model editions successively in a data location, but only one model edition is deployed and is active in the data location at a certain point in time.

Data Location Types

There are two types of data locations. The type is selected when the data location is created and cannot be changed afterwards:

The data location types are:

  • Development Data Locations: A data location of this type supports deploying open or closed model editions. This type of data location is suitable for testing models in development and quality assurance environments.

  • Production Data Location: A data location of this type supports deploying only closed model editions. This type of data location is suitable for deploying MDM hubs in production environments.

Installation Patterns

This section provides patterns for deploying Semarchy xDM in real-life environments.

Pattern #1: Development, QA/UAT, and Production in Different Sites

This pattern assumes that the development, QA/UAT (Quality Assurance, User Acceptance Tests), and production sites are located on different networks or sites.

For this pattern, three repositories are created instead:

  • A REPO_DEV Design repository for the development site. A DEV Development data location is attached to this repository. This location is used by the development team to test their work.

  • A REPO_QA Deployment repository for the QA site. A QA Production data location is attached to this repository. This location contains QA data and allows for limit values testing.

  • A REPO_PROD Deployment repository for the production site. A PROD Production data location is attached to this repository. Only closed and production-ready model editions can be deployed in this location. The real production data is here.

With this configuration:

  • The entire development phase is performed in the REPO_DEV repository, with closed and open model editions. Model editions are deployed to the DEV data location, for the developers' tests.

  • When the development phase is finished, Model editions are closed and exported to files from the REPO_DEV repository and imported in the REPO_QA repository. These closed model editions are deployed to the QA data location for testing.
    QA teams perform testing to ensure the model is bug-free and ready for production.

  • Once the user testing/QA process is finished, the closed model editions are exported from the REPO_QA to the REPO_PROD repository and then deployed to the PROD data location.

With this configuration, you need to deploy three instances of Semarchy xDM, one per repository. These three instances are located on three different networks with possibly different security, scalability and high availability requirements and settings.

Pattern #2: Development/QA/UAT and Production in Different Sites

This pattern is similar to Pattern #1 but assumes that development and QA/UAT are co-located in one site, and production is located in a remote location.

For this pattern, two repositories are created:

  • A REPO_DEV Design repository for the development and QA site. A DEV Development data location and a QA Development data locations are attached to this repository.

  • A REPO_PROD Deployment repository for the production site. The PROD Production data location is attached to this repository.

With this configuration:

  • Models are developed in the REPO_DEV repository and tested by developers in the DEV data location.

  • When a milestone is reached, models editions are closed. These closed model editions are deployed to the QA data location for testing.

  • Once the QA phase is finished, the closed model editions are exported to files from the REPO_DEV repository and imported in the REPO_PROD repository. From this repository, these closed model editions are deployed to the PROD data location.

With this configuration, you need to deploy two instances of Semarchy xDM, one per repository. These two instances are located on two different networks with possibly different security, scalability and high availability requirements and settings.

Pattern #3: Single Repository and Project

This pattern assumes that a single project is designed through a development/QA/Production lifecycle.

For this pattern:

  • A single Design repository is created.

  • Three data locations, DEV, QA and PROD are created:

    • DEV is a Development data location into which open model editions are deployed during the development phase.

    • QA is a Development data location into which the model editions closed by development are deployed for QA consumption. Possibly, open model editions can be deployed directly from development for immediate testing.

    • PROD is a Production data location.

In this pattern, a single repository contains the development, QA and production editions of the models. Model versioning allows freezing and delivering to the next stage (and next deployment location) a model as it moves along its lifecycle.

Patterns #1 and #2 are more suitable for real enterprise environement. Pattern #3 is more suitable for prototyping or evaluation environments.

Pattern #4: Single Repository, Multiple Projects.

This pattern is similar to the previous one but assumes that several projects/models are managed in the same repository.

For this pattern:

  • A single design repository is created.

  • Three data locations are created per project: DEV1, QA1 and PROD1, then DEV2, QA2, PROD2, etc.

The organization is the same as in pattern #3, but a set of data locations exists for each project managed in the single repository.

You can combine this pattern with patterns #1 and #2 to manage multiple projects across multiple development, QA and production environments.

High-Availability Configuration

Semarchy xDM can be configured to support enterprise-scale deployment and high-availability.

Semarchy xDM supports the clustered deployment of the Semarchy xDM web application for high-availability and failover. A clustered deployment can be set up for example to support a large number of concurrent users performing data access and authoring operations.

Reference Architecture for High-Availability

In a clustered deployment, only one instance of the Semarchy xDM application manages and runs the certification processes. This instance is the Active Instance. A set of clustered Semarchy xDM applications serves users accessing the Workbench (for modeling, administration or data stewardship) as well as applications accessing data locations via integration points. These are Passive Instances. The back-end databases hosting the repository and data locations are deployed in a database cluster, and an HTTP load balancer is used as a front-end for users and applications.

The reference architecture for such a configuration is described in the following figure: image

This architecture is composed of the following components:

  • HTTP Load Balancer: This component manages the sessions coming from within the enterprise network or from the Internet (typically via a Firewall). This component may be a dedicated hardware load balancer or a software solution, which distributes the incoming sessions on the passive instances running in the JEE application server cluster.

  • JEE Application Server Cluster + Passive Semarchy xDM Platforms: A Semarchy xDM application instance is deployed on each node of this cluster, which is scaled to manage the volume of incoming requests. In the case of a node failure, the other nodes remain able to serve the sessions. The Semarchy xDM applications deployed in the cluster are Passive Instances. Such instance is able to process user access to the Semarchy Workbench, or application access to the integration points, but is unable to manage the batches and the jobs.

  • JEE Server + Active Semarchy xDM Platform: This single JEE server hosts the only complete Semarchy xDM platform of the architecture. This Active Instance is not accessible to users or applications. Its sole purpose is to poll the submitted batches and process the Jobs.

  • Oracle RAC Cluster: This component hosts the Semarchy xDM Repository and the Data Locations schemas in a clustered environment. Both active and passive instances of the Semarchy xDM Platform connect to this cluster using Oracle RAC JDBC datasources. In a PostgreSQL environement, use a database cluster for an equivalent configuration.

In this architecture:

  • Design-time or administrative operations are processed by the passive instances in the JEE application server cluster.

  • Operations performed on the MDM Hubs (data access, steppers or external loads) are also processed by the passive instances, but the resulting batches and jobs are always processed by the single active instance.

In this architecture, only one Active Instance must be configured. Multiple active instances are not supported.

Load Balancing

In the architecture, load balancing ensures an optimal usage of the resources for a large number of users and applications accessing simultaneously Semarchy xDM.

Load balancing is performed at two levels:

  • The HTTP load balancer distributes the incoming requests on the nodes of the JEE application server cluster.

  • The Oracle RAC JDBC datasource configuration distributes database access to the repository and the data locations on the RAC cluster nodes. In a PostgreSQL environement, use the JDBC connection configuration to enable load balacing on the multiple nodes of the cluster.

Failure and Recovery

In the reference architecture, fail-over is managed for both user and application sessions. This section describes the behavior in case of a failure at the various points of the architecture.

Database Failure

In the event of a RAC node failure, other nodes are able to recover and process the incoming database requests.

Passive Instance Failure

If one of the nodes of the JEE application server cluster fails:

  • Application sessions are moved to the other active nodes.

  • User sessions to this node are automatically restarted on other active nodes. The only information not recovered is the content of the un-saved editors for the user sessions. All the other content is saved in the repository or the data locations. Transactions attached to steppers for example, are saved in the data locations and not lost.

Active Instance Failure

The purpose of the Active Instance is to process batches and jobs.

If this server fails:

  • Jobs running the queues are halted.

  • Queued jobs remain in their queue.

  • Incoming batches remain pending for the batch poller to process them.

The active instance must be restarted automatically or manually to recover from a failure.
When it is restarted, the platform resumes its normal course of activity with no user action required.

A Failure of the Active Instance does not impact the overall activity of users or applications, as these rely on the (clustered) Passive Instances. The only impact of such a failure may be a delay in the processing of data changes.

Configuring Semarchy xDM for High-Availability

The Semarchy xDM application comes in two flavors corresponding to two WAR (Web Application Archive) files. Both these files are in the semarchy-xdm-install-<version tag>.zip archive file, in the mdm-server folder:

  • semarchy-passive.war includes the passive application to deploy on all the passive nodes of the cluster. This war does not include the Batch Poller and Engine services, and is not able to trigger or process submitted batches.

  • semarchy.war includes the active application to deploy on the single active server. This war includes the Batch Poller and the Engine, and is able to trigger and process the submitted batches.

The overall installation process for a high-availability configuration is similar to the general installation process:

  1. Create the repository and data location schemas in the RAC cluster.

  2. Configure the application server security for both the cluster and the active node.

  3. Configure the Oracle RAC JDBC Datasources for the nodes/cluster. Refer to your Oracle Database and Application Server documentation for more information about configuring RAC JDBC Datasources. If using PostgreSQL, refer to the PostgreSQL JDBC driver documentation to configure high-availability and load balacing.

  4. Deploy the applications:

    • Deploy the active instance (semarchy.war) in the active node.

    • Deploy the passive instances (semarchy-passive.war) in the JEE cluster according to the recommendations given in the application server documentation.

HTTP Load Balancing Configuration
Semarchy xDM requires that you configure HTTP Load Balancing with Sticky Sessions (also known as session persistence or session affinity). In this mode, requests from existing sessions are consistently routed to the same server. This is mandatory for the Semarchy xDM user interfaces (the Workbench and the applications), but not for integration points.
For example, for Amazon Web Services (AWS) deployments, sticky sessions are configured in the Load Balancer.

Configuring the Database Schemas

This section explains how to configure the database schemas for the repository and data locations.

Configuring the Repository Schema

Before installing Semarchy xDM, you must create a database schema for the repository. You can create it manually or use your database administration interface for this purpose. In this section, we provide sample scripts for creating this schema. Make sure to adapt this script to your database configuration.

Creating the repository schema (Oracle)
CREATE USER <repository_user_name> IDENTIFIED BY <repository_user_password>
 DEFAULT TABLESPACE USERS TEMPORARY TABLESPACE TEMP;
GRANT CONNECT, RESOURCE TO <repository_user_name>;

-- The following command should be used for Oracle 12c and above
GRANT UNLIMITED TABLESPACE TO <repository_user_name>;
Creating a database and the repository schema (PostgreSQL)
/* Create a database for {mdm-full-product-name} */

CREATE DATABASE <postgresql_database_name>;

/* Disconnect and then reconnect using:
 *  the JDBC URL:  jdbc:postgresql://<host>:<port>/<postgresql_database_name>
 *  or using psql with the following command: psql -U postgres <postgresql_database_name>
 */

CREATE SCHEMA extensions;
GRANT USAGE ON SCHEMA extensions TO PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA extensions GRANT EXECUTE ON FUNCTIONS TO PUBLIC;
ALTER DATABASE <postgresql_database_name> SET SEARCH_PATH TO "$user",public,extensions;

CREATE EXTENSION IF NOT EXISTS "uuid-ossp"     with schema extensions;
CREATE EXTENSION IF NOT EXISTS "fuzzystrmatch" with schema extensions;

/* Create the repository schema */

CREATE USER <repository_user_name> WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE INHERIT NOREPLICATION CONNECTION LIMIT -1 UNENCRYPTED PASSWORD '<repository_user_password>';

CREATE SCHEMA <repository_user_name> AUTHORIZATION <repository_user_name>;

GRANT USAGE ON SCHEMA <repository_user_name> TO public;
Store the values of the <repository_user_name> and <repository_user_password> as you will need them later for creating the datasource to access the repository.

Configuring the Data Locations Schemas

You do not need to create the data locations schemas at installation time, but it is recommended to plan them as part of the installation and configuration effort. You can create them manually or use your database administration interface for this purpose. In this section, we provide a sample script for creating a data location schema. Make sure to adapt this script to your database configuration and duplicate it to create the schemas for all data locations.

Creating a data location schema (Oracle)
CREATE USER <data_location_user_name> IDENTIFIED BY <data_location_user_password>
 DEFAULT TABLESPACE USERS TEMPORARY TABLESPACE TEMP;
GRANT CONNECT, RESOURCE TO <data_location_user_name>;

-- The following command should be used for Oracle 12c and above
GRANT UNLIMITED TABLESPACE TO <data_location_user_name>;
Creating a data location schema (PostgreSQL)
CREATE USER <data_location_user_name> WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE INHERIT NOREPLICATION CONNECTION LIMIT -1 UNENCRYPTED PASSWORD '<data_location_user_password>';

CREATE SCHEMA <data_location_user_name> AUTHORIZATION <data_location_user_name>;

GRANT USAGE ON SCHEMA <data_location_user_name> TO public;
Store the values of the <data_location_user_name> and <data_location_user_password> as you will need them later for creating the datasource to access the data location.
The repository and data locations do not necessary need to be located in the same database. The same schema should not be used for the repository and a data location, and you cannot use the same schema for several data locations.

Sizing and Maintaining the Schemas

Repository Schema

The following considerations should be taken into account when sizing the repository schema:

  • The repository contains a small volume of information if you exclude the execution log. It can be sized by default from 100 to 200Mb depending on the number of models and number of model versions stored in it.

  • The execution log is the larger part of the data stored in the repository. The volume of data generated in the log depends on the number of job executions executed daily. It is recommended to monitor the job execution and resize the repository according to the log history you want to preserve.

  • In order to maintain the logs to a reasonable volume, it is recommended to regularly perform a purge of the log.

To purge the logs:

  1. Connect to the Semarchy Workbench.

  2. Select the Administration Console perspective.

  3. In the Administration view, double-click the Job Logs node. The Job Logs list editor opens.

  4. In the editor toolbar, select the Purge Using a Filter… button.

  5. Select and existing filter or define a new filter.

  6. Click the Finish button to purge using the selected filter.

Data Location Schemas

The following considerations should be taken into account when sizing the data location schema:

  • The data location schema contains for each entity of the deployed model several tables that correspond to the data in the various stages of the certification process. Some of these tables contain also the history of the previous iterations of the process.

  • The volume required in the data location depends on the following factors:

    • Number of entities

    • Number of source records pushed for these entities

    • Number of new or updated source records pushed for these entities.

A recommended original sizing is to add the source data volume pushed for each entity by all publishers plus one data authoring (the overall input) and multiply it by a factor of 10. It is recommended after the original sizing to monitor the size of the data location schema in the normal course of operations and adjust the sizing appropriately.

The same sizing considerations applies to both the data and temporary tablespaces in the case of the data locations, as the database engine is used for most of the processing effort in the certification process.

Data Retention Policies can be created to define the volume of data to retain in the data locations, and Data Purges can be scheduled to trigger the pruning of unnecessary data. Defining Retention Policies is covered in the Securing Data chapter of the Semarchy xDM Developer’s Guide. Data Purges are described in the Managing Execution chapter of the Semarchy xDM Administration Guide.

Deployment and Configuration Overview

The Semarchy xDM Application is a Java EE application that can be deployed to a number of environments. It requires a Java EE applications server (for example: Tomcat, Glassfish, JBoss/WildFly, etc). This section details the steps required to configure the application server and deploy the application in the application server.

The instructions provided in the following chapters detail one method and approach to configure the application server. Note that the method may vary depending on the application server version. Please refer to the application server documentation for the up-to-date instructions and instruction details. Approaches also differ depending on the best practices used in your information system. Please contact the application server administrator and review these instructions to adapt them to your practices.

Conventions

In the following chapters, the following variables names are used in the tasks:

  • The semarchy-xdm-install-xxx.zip file refers to the Semarchy xDM - Server Installation file you have downloaded to install Semarchy xDM. The name of this file varies as it includes the Semarchy xDM version and build number.

  • The semadmin user refers to the first user created for connecting to Semarchy xDM. This user is named by default semadmin. This name can be changed in the installation process.

  • <semadmin_password> refers to the password you want to set for the semadmin user. This password must comply with the password policy defined for your application server.

  • The temp folder refers to a temporary folder of your choice.

Security

The application configuration includes the security in the application server.
The goal of this task is to create:

  • A semarchyConnect role. This built-in role grants the privilege to connect to the application.

  • A semarchyAdmin role. This built-in role grants full privileges in the application.

  • A user called semadmin with semarchyConnect and semarchyAdmin roles. This user is the administrator for the Semarchy xDM application.

Depending on the application server, users are directly mapped to roles, or are mapped to roles via a Group concept. When an application server uses groups:

  • The semarchyConnectGroup and semarchyAdminGroup groups are created

  • The semarchyConnectGroup group is mapped to the built-in semarchyConnect role.

  • The semarchyAdminGroup group is mapped to the built-in semarchyAdmin role.

  • The semadmin user is added to both the semarchyConnectGroup and semarchyAdminGroup groups.

This basic configuration allows you to connect with the semadmin user and have full privileges for the application.

It is recommended to tune and enhance the security by:

  • Creating new users, roles and groups dedicated to Semarchy xDM in the application server security realm according to your organization. Refer to your application server’s security guide for more information.

  • Declaring these roles in Semarchy xDM and granting privileges to the application’s features to these roles. For more information, refer to the Semarchy xDM Administration Guide.

Datasources

The configuration of the application includes creating datasources to connect to the repository and data locations that will be used by your MDM projects.

To configure the repository datasource, make sure you have the following information:

  • For Oracle:

    • <oracle_instance_hostname>: host name or IP address of the database server

    • <oracle_listener_port>: number of the port where the server listens for requests.

    • <oracle_SID_name> or <oracle_service_name>: name of a database on the server. This is the SID or ServiceName in the Oracle terminology.
      Note that the JDBC URL used for oracle varies depending on the value used:

      • SID: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • Service Name: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>/<oracle_service_name>

    • <repository_user_name>: name for the user created when configuring the repository schema.

    • <repository_user_password>: this user’s password.

  • For PostgreSQL:

    • <postgresql_hostname>: host name or IP address of the database server

    • <postgresql_port>: number of the port where the server listens for requests.

    • <postgresql_database_name>: name of a database on the server.

    • <repository_user_name>: name for the user created when configuring the repository schema.

    • <repository_user_password>: this user’s password.

The JNDI Name of the repository datasource must always be jdbc/SEMARCHY_REPOSITORY. Make sure to set this name for the repository datasource as it is referred to with that name by the application.

To configure the data location datasources, make sure you have the following information for each data location schema:

  • <data_location_name>: name for the data location. You can use this name as the JNDI name for the datasource in which this data location will be hosted.

  • For Oracle:

    • <oracle_instance_hostname>: host name or IP address of the database server

    • <oracle_listener_port>: number of the port where the server listens for requests.

    • oracle_SID_name> or <oracle_service_name>: name of a database on the server. This is the SID or ServiceName in the Oracle terminology.

    • <data_location_user_name>: name for the user created when configuring the data location schema.

    • <data_location_user_password>: this user’s password.

  • For PostgreSQL:

    • <postgresql_hostname>: host name or IP address of the database server

    • <postgresql_port>: number of the port where the server listens for requests.

    • <postgresql_database_name>: name of a database on the server.

    • <data_location_user_name>: name for the user created when configuring the data location schema.

    • <data_location_user_password>: this user’s password.

Datasource Tuning Considerations

When configuring datasources for Semarchy xDM, more specifically in production environments, you must take into account several considerations listed below.

We provide below explanations for data source tuning and configuration, with samples for some of the supported application servers. Refer to your application server documentation for more detailed information.
Connection Pool Sizing

Datasources configured for Semarchy should use connection pools, sized according to the expected usage.

In the normal course of operation of Semarchy:

  • Each simultaneous user session connected to and interacting with an MDM application consumes one connection to the data location datasource and possibly one to the repository when working with steppers.

  • Each simultaneous integration job consumes one connection to the data location datasource, plus one to the repository datasource.

You must configure the pool size as a trade-off between two directions:

  • If the pool creates and keeps too many connections, it will be ready to serve any client immediately, but it will also overuse resources by pro-actively creating too many connections.

  • If the pool does not have enough connections available, then the pool may be exhausted at certain times, causing client requests to wait until a connection becomes available in the pool. In a nutshell, the more users try to access a small pool and the longer their queries, the more they will have to wait between each click.

To configure the pool size, the application servers provide a series of parameters.

Application Server Relevant Parameters

Apache Tomcat

  • initialSize: initial number of connections when the pool is created.

  • maxActive: the maximum number of active connections at the same time.

  • maxIdle: the maximum number of idle connections kept in the pool at all times.

  • minIdle: the minimum number of idle connections kept in the pool at all times.

Other parameters to consider: timeBetweenEvictionRunsMillis, minEvictableIdleTimeMillis and maxWait.

Oracle WebLogic

  • Initial Capacity: number of connections created when the pool is created.

  • Maximum Capacity: the maximum number of connections at the same time in the pool

  • Maximum Capacity: the minimum number of connections kept in the pool at all times.

Other parameters to consider: Pool Resize Quantity and Shrink Frequency.

WildFly

  • max-pool-size: the maximum number of connections at the same time in the pool

  • min-pool-size: the minimum number of connections kept in the pool at all times.

  • prefill: create max-pool-size connections when the pool is created.

Other parameter to consider: idle-timeout-minutes.

Connection Management

When Semarchy takes a connection from the connection pool, it assumes that this connection is a valid one. In certain situations, the connection may be invalid at the database side (for maintenance reasons, etc.). It is a good practice to configure the pool to test connections before serving them.

This capability is typically enabled in application servers using a query that is executed before serving the connection from the pool.

Similarly, connections in the pool should be configured not in auto-commit mode. When a connection is released to the pool, the configuration should ensure that this connection rollbacks any uncommitted statement.

Application Server Relevant Parameters

Apache Tomcat

  • testOnBorrow: boolean indicating whether connections are tested when borrowed from the pool.

  • validationQuery: query used to test the connection (e.g.: select 1 from dual for Oracle or select 1 for PostgreSQL).

  • validationInterval: minimum interval between two validation tests, to avoid excess validation.

  • defaultAutoCommit: default auto-commit state for the connections. It should be set to false.

  • rollbackOnReturn: rollbacks the transaction when the connection is released to the pool. It should be set to true.

Make sure to set defaultAutoCommit to false and rollbackOnReturn to `true in PostgreSQL connection pools.

Another parameter to consider: validationQueryTimeout.

Oracle WebLogic

  • Test Reserved Connections: indicates whether connections are tested when served from the pool.

  • Test Table Name: Name of a table onto which a select 1 from <table_name> is issued for testing.

  • Seconds to Trust an Idle Pool Connection: minimum interval between two validation tests, to avoid excess validation.

WildFly

  • validate-on-match: indicates whether connections are tested when served from the pool.

  • valid-connection-checker: class implementing an optimized method to validate the connection. Use org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker for Oracle or org.jboss.jca.adapters.jdbc.extensions.postgres.PostgreSQLValidConnectionChecker for PostgreSQL.

  • check-valid-connection-sql: (Alternate method to valid-connection-checker) query used to test the connection (e.g.: select 1 from dual for Oracle or select 1 for PostgreSQL).

Long-Running Queries

Semarchy xDM sometimes executes long-running queries to the data location schema. For example, an integration job that processes large data volumes may have queries that run for several minutes or hours.

Application servers have a mechanism to consider connection borrowed from the pool for a long time as "leaked" or "stale". When such connection is detected, the connection is reclaimed by the pool and the query failed. This mechanism must be disabled for Semarchy xDM long-running queries to work properly.

Application Server Relevant Parameters

Apache Tomcat

  • removeAbandoned: boolean indicating whether connections should be considered stale and removed after removeAbandonedTimeout seconds. This parameter should preferably be set to false (it is its default value).

  • removeAbandonedTimeout: time after which a connection is considered stale.

Oracle WebLogic

  • Inactive Connection Timeout: time after which a connection is considered leaked and is reclaimed by WebLogic for the pool. Set this parameter to '0' to disable this feature.

WildFly

  • stale-connection-checker: class implementing a method for properly checking stale connections. Use org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker.

JavaMail Session

Semarchy xDM uses email servers for example to send email notifications.
For the email features to work, you must configure an email notification server.

A notification server can be configured either:

  • By entering the mail server SMTP configuration the Semarchy xDM.

  • By referring Semarchy xDM to a JavaMail Session resource configured in the application server.

This latter option is described in the next chapters for each application server.

If a JavaMail Session resource is already configured in the application server, you can skip these steps. You will reuse the existing JavaMail Session resource when configuring the notification server.
The resource name used for the JavaMail session resource is set to mail/Session. This value can be changed when running the installation process, and the changed value must be used when configuring the notification server.

Application Deployment

The application is deployed by default with the semarchy context. Therefore it is accessible on the application server on the following URL: http://<application_server_host>:<application_server_port>/semarchy/.

During the application deployment, it is possible to use a different context than semarchy. If you use a different context, make sure to take it into account in the URL to test and connect to the application.

Downloading Semarchy xDM

The Server Installation files for Semarchy xDM can be downloaded from the Semarchy website, at the following URL: http://www.semarchy.com/get/semarchy-xdm-install/.
The Semarchy xDM Server Installation file you download is referred to as semarchy-xdm-install-xxx.zip.

  1. Uncompress this file in the temp folder.

The semarchy-xdm-install-xxx.zip archive contains the following files and folders:

File/Folder Description

README.txt

File describing the package

mdm-server/

This folder contains the installation files for Semarchy xDM.

mdm-server/semarchy.war

Semarchy xDM deployable WAR file.

mdm-server/semarchy-passive.war

Semarchy xDM deployable WAR file for Passive Instances. Use this version for deploying Semarchy xDM in an existing supported application server for High-Availability Configurations.

mdm-server/semarchy.xml

Sample configuration file deploying Semarchy xDM in Apache Tomcat. When deploying a passive instance, this file should be renamed semarchy-passive.xml and used for deployment.

mdm-server/additional-libraries/

This folder contains libraries used with Semarchy xDM, listed below.

mdm-server/additional-libraries/ojdbc6.jar

Oracle JDBC Driver

mdm-server/additional-libraries/org.postgresql.jdbc<version>.jar

PostgreSQL JDBC Driver

mdm-server/additional-libraries/com.sun.mail.javax.mail_xxx.jar

Library to install to enable JavaMail for Apache Tomcat Servers. Ignore this file if you are using a different application server.

mdm-server/additional-libraries/com.semarchy.tool.jee.tomcat_<version>.jar

Tomcat tools for authentication and role mapping. See Delegating Authentication and Authorization in Tomcat for more information.

Deploying and Configuring with Apache Tomcat

This section explains how to configure and deploy the Semarchy xDM Application with Apache Tomcat.

In this section, <tomcat> refers to the Apache Tomcat installation folder.

Refer to the Tomcat Documentation for more details about the deployment and configuration processes for Apache Tomcat.

Installing Additional Libraries

Before adding libraries, you must stop the Apache Tomcat server using <tomcat>/bin/shutdown.bat (Windows) or <tomcat>/bin/shutdown.sh (UNIX/Linux).
Similarly, after installing the libraries, restart the Apache Tomcat server using <tomcat>/bin/startup.bat (Windows) or <tomcat>/bin/startup.sh (UNIX/Linux)

Installing the Database JDBC Driver

To install the JDBC driver:

  1. Copy the database driver file (temp/mdm-server/additional-libraries/ojdbc6.jar for Oracle or temp/mdm-server/additional-libraries/org.postgresql.jdbc<version>.jar for PostgreSQL) to the <tomcat>/lib directory.

Installing the Mail Session Libraries

This configuration is required for mail notifications using JEE Mail Session.

To install the Java Mail Libraries:

  1. Copy the temp/mdm-server/additional-libraries/com.sun.mail.javax.mail_xxx.jar file to the <tomcat>/lib/ folder

Configuring the Security

To configure the Semarchy xDM administrator user:

  1. Stop the Apache Tomcat server.

  2. Edit the <tomcat>/conf/tomcat-users.xml file.

  3. In the <tomcat-users> section, add the following line:

 <user username="semadmin" password="<semadmin_password>" roles="semarchyConnect,semarchyAdmin"/>
  1. Save the file.

  2. Restart the Apache Tomcat server.

This operation adds to Apache Tomcat a semadmin user with its password. This user has full privileges to the Semarchy xDM application. Make sure to use a strong password for this user.

Configuring the Logging

It recommended to change the default configuration of the Tomcat server to benefit from the logging configuration directly from the Semarchy xDM administration console and prevent useless logging at server startup.

To configure the logging:

  1. Add Tomcat startup properties:

    1. Open the <tomcat>/bin/setenv.sh (UNIX/Linux) or <tomcat>/bin/setenv.bat (Windows) file with a text editor.

    2. Add the following properties to the CATALINA_OPTS variable: -Dorg.ops4j.pax.logging.DefaultServiceLog.level=WARN -Dorg.apache.cxf.Logger=org.apache.cxf.common.logging.Slf4jLogger

    3. Save the file.

  2. Edit the bootstrap logging configuration:

    1. Edit the <tomcat>/conf/logging.properties file with a text editor.

    2. Append the properties listed below to the file then save it.

Logging.properties configuration for Tomcat to prevent verbose bootstrap logging.
com.sun.xml.level = INFO
javax.xml.bind.level = INFO
org.apache.cxf.level = WARNING
org.ops4j.pax.logging.internal.Activator.level = WARNING
org.apache.aries.blueprint.level = WARNING
After this initial configuration, you can finely configure the logging . Refer to the Configuring the Logging chapter in the Semarchy xDM Administration Guide.

Deploying the Application

Deploying Passive Instances
The rest of the process describes the deployment of an active instance of Semarchy xDM.
The process for deploying a passive instance in a high-availability/load-balancing setup is similar.
For passive instances, semarchy-passive.war should be deployed instead of semarchy.war, and the semarchy.xml file should be renamed semarchy-passive.xml (to match the name of the deployed passive instance) and modified as described in the process.
See Configuring Semarchy xDM for High-Availability for more information about active and passive instances.

To deploy the application:

  1. Connect to the Apache Tomcat Manager (http://<tomcat_host>:<tomcat_port>/manager/).

  2. In the Deploy War File section, click the Select File button, then select the temp/mdm-server/semarchy.war file.

  3. Click the Deploy button.

The Semarchy xDM application is uploaded and deployed in the server.

Setting Up the Datasources

It is recommended to keep a backup copy of the semarchy.xml file. If you un-deploy the application, this file is removed and the changes performed are lost.

To configure the repository datasource:

  1. Edit the <tomcat>/conf/Catalina/localhost/semarchy.xml file.

  2. In the <context> configuration element, search the jdbc/SEMARCHY_REPOSITORY datasource and edit the following parameters:

    • url:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

    • username: <repository_user_name>

    • password: <repository_user_password>

    • If using PostgreSQL, replace the validationQuery value by SELECT 1.

  3. Save the semarchy.xml file.

Do not change the name of the SEMARCHY_REPOSITORY datasource. The application refers to a datasource with this name for the repository.

To configure a data location datasources:

  1. Edit the <tomcat>/conf/Catalina/localhost/semarchy.xml file.

  2. In the <context> configuration element, copy and un-comment the datasource sample definition called jdbc/DATA_LOCATION_1.

  3. Rename and edit the copy of the datasource settings with the following parameters:

    • name: jdbc/<data_location_datasource_name>

    • url:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

    • username: <data_location_user_name>

    • password: <data_location_user_password>

    • If using PostgreSQL, replace the validationQuery value by SELECT 1.

  4. Repeat the two previous steps for each data location’s datasource.

  5. Save the semarchy.xml file.

The repository and data location datasources are now configured, pointing to the schemas previously created.

Depending on the Tomcat server configuration, you may have to restart the Semarchy xDM application for refreshing the application configuration with your changes.

Configuring JavaMail Session

This configuration is required for mail notifications using JEE Mail Session.

To configure JavaMail Session:

  1. Edit the <tomcat>/conf/Catalina/localhost/semarchy.xml file.

  2. In the <context> configuration element add the entry given below and then save the semarchy.xml file. Change the entry below to match your SMTP server configuration. See the SMTP package documentation for a description of the properties.

<Resource name="mail/Session" auth="Container" type="javax.mail.Session"
 mail.smtp.host="<mail_server_host>"
 mail.port="<mail_server_port>"
 mail.smtp.user="<mail_user_name>"
 mail.transport.protocol="smtp"
 password="<mail_user_password>"
 mail.smtp.auth="true" />
 <!-- Add the following to the configuration in case of a SASL Authenticator error:
 mail.smtp.socketFactory.class="javax.net.ssl.SSLSocketFactory"
 mail.smtp.socketFactory.port="<mail_server_port>"
 mail.smtp.socketFactory.fallback="false"
 -->
Gmail Users may have to allow access to "Less Secure Apps" if facing connection issues.

Testing the Application

To test the application:

  1. Open a web browser.

  2. In the URL, enter: http://<tomcat_host>:<tomcat_port>/semarchy/.

The Semarchy xDM Login page appears.

Proceed directly to the Installing the Repository task.

Delegating Authentication and Authorization in Tomcat

This section provides advanced configuration information for delegating authentication and authorization in Apache Tomcat.

  • Delegating authentication lets an external system confirm who the user is (check the username and password). This system is, for example, an LDAP or Active Directory, or an OpenID provider. Note that the authenticating system may or may not provide the roles (or authorizations) of a given user. For example, OpenID is solely used for authentication and does not provide user’s authorizations.

  • Delegating authorization lets an external system provide the roles of a user that logs in. These roles must match those defined in Semarchy xDM to define what the user is authorized to do. Semarchy xDM provides a Role Mapper that lets you map external roles to Semarchy xDM roles.

A strong knowledge of Tomcat concepts and options is required to configure authentication and authorization according to each situation. Please read carefully the following documents to learn about these subjects:

LDAP

LDAP Authentication and Authorization

Semarchy xDM supports authenticating as well as roles retrieval from an external directory, such as LDAP or Active Directory. The information entered in the login form is passed to the external directory, which returns, if the user is valid one, this user’s roles.

To delegate the authentication to an LDAP directory, add to the semarchy.xml configuration file a JNDI Realm definition as shown in the example below. This configuration must be customized to match your LDAP directory configuration.

For more information about configuring LDAP with Tomcat, see the Tomcat Realm Documentation.

Sample Configuration: LDAP authentication and authorization
<Realm className="org.apache.catalina.realm.JNDIRealm"
        connectionURL="ldap://ldaphost.mydomain.com:389"                (1)
        userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"        (2)
        roleBase="ou=groups,ou=people,dc=myCompany,dc=com"                (3)
        roleName="cn"                                                        (3)
        roleSearch="(member={0})" />                                        (3)

The parameters of the realm must be customized to your configuration:

1 connectionURL: Connection URL to the LDAP server
2 Users log in to Semarchy xDM with their LDAP UID. The password passed in the login form must the one of the user found in the LDAP tree using the userPattern.
3 Roles are searched in the roleBase point in the LDAP tree. roleSearch defines the LDAP search filter used to search roles attached to a username (represented by {0}). The roles returned are the attribute identified by roleName.

OpenID and OpenID Connect

OpenID is a mechanism used to delegate authentication to another provider.

With OpenID configured, you can use Google, Yahoo, Facebook or Twitter accounts to authenticate to {mdm-regular-product-name, to replace or complement a Login Form.

There are two main "implementations" of OpenID supported by the authentication providers: OpenID 2.0 and OpenID Connect. OpenID Connect is a newer implementation, and authentication providers move over time from OpenID 2.0 to OpenID Connect.

This section explains how to configure OpenID 2.0 or OpenID Connect for Tomcat, and how to mix these authentication schemes with a Login Form.

OpenID is an authentication service. It does not support roles. A realm (LDAP, File, or other) must be defined to retrieve roles for the users identified with OpenID.
OpenID 2.0 Authentication and LDAP Authorization

The following configuration steps apply to Tomcat 7 and 8.

To configure authentication with OpenID:

  1. Download the Open ID Authenticator for Tomcat.

  2. Edit the semarchy.xml configuration file and create the OpenID valve and realm configuration.
    A sample OpenID configuration using Yahoo authentication is provided below. It uses LDAP for the authorization.

Sample Configuration: OpenID (Yahoo) authentication, LDAP authorization
<!-- OpenID Valve configuration for Authentication -->
<Valve className="org.bsworks.catalina.authenticator.openid.OpenIDAuthenticator"
        singleProviderURI="https://me.yahoo.com"
        loginNameAttributeType="http://axschema.org/contact/email"
        allowedClaimedIDPattern="https://me.yahoo.com/a/.+"
        hostBaseURI="http://mdm_host:port" />         (1)

<!--Realms configuration -->
<Realm className="org.bsworks.catalina.authenticator.openid.OpenIDRealm">
        <!--
        This realm provides authorizations for the OpenID-authenticated users.
        It uses LDAP with each username equal to the OpenID username.
        Any Tomcat realm can be used for this purpose.
        -->
        <Realm  className="org.apache.catalina.realm.JNDIRealm"
                connectionURL="ldap://ldaphost.mydomain.com:389"                        (2)
                userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"        (3)
                userPassword="uid"                                                (3)
                roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
                roleName="cn"
                roleSearch="(member={0})" />
</Realm>

The parameters of the valve and realm must be customized to your configuration:

1 The OpenID connector will return to the original server after authentication. In most cases (when the server uses HTTPS), this mechanism works. If the server does not use HTTPS, it is recommended to set the host and port of the Semarchy xDM instance in hostBaseURI.
2 This LDAP host corresponds to the LDAP server containing your user and roles mappings.
3 LDAP Realms configuration. see the Tomcat Realm Documentation for more information. Note that in this realm configuration, the userPassword must point to the LDAP uid (user id). The LDAP user ID must be equal to the OpenID Connect User ID (in that case, the email).
OpenID 2.0 and Form Authentication, LDAP Authorization

Mixing OpenID 2.0 with a form authentication provides the possibility to log in using an OpenID account or a login form. Authentication via the login form is made against an authentication provider, for example an LDAP directory or a database of users stored in the application server’s disk.

The following configuration steps apply to Tomcat 7 and 8.

To configure authentication with OpenID and a Login Form:

  1. Download the Open ID Authenticator for Tomcat.

  2. Edit the semarchy.xml configuration file.
    The following configuration sample can be used as a starting point.

Sample Configuration: OpenID (Yahoo) + Login Form (LDAP) authentication, LDAP authorization
<!-- OpenID Valve configuration for Authentication -->
<Valve className="org.bsworks.catalina.authenticator.openid.OpenIDAuthenticator"
        loginNameAttributeType="http://axschema.org/contact/email"
        singleProviderURI="https://me.yahoo.com"
        allowedClaimedIDPattern="https://me.yahoo.com/a/.+"
        hostBaseURI="http://mdm_host:myport" />

<!--
        Login form configuration, allowing to display a OpenID authentication button
        pointing to OpenID Provider (Yahoo) given by parameter value
-->
<Parameter name="SingleSignOn"
           value="openid_identifier=https://me.yahoo.com"
           override="true" />

<!--Realms configuration -->
<Realm className="org.bsworks.catalina.authenticator.openid.OpenIDRealm">
        <!--
        The first realm provides authorizations for the OpenID-authenticated users.
        This realm uses LDAP with each username equal to the OpenID username.
        Any Tomcat realm can be used for this purpose.
        -->
        <Realm  className="org.apache.catalina.realm.JNDIRealm"
                connectionURL="ldap://ldaphost.mydomain.com:389"
                userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"
                userPassword="uid"
                roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
                roleName="cn"
                roleSearch="(member={0})" />

        <!--
        The second realm (and subsequent ones) are used as fallbacks for
        authentication and authorization.
        Local realms can be used as local authentication providers,
        or fallback providers if the OpenID authentication fails.
        -->
        <Realm         className="org.apache.catalina.realm.JNDIRealm"
                connectionURL="ldap://ldaphost.mydomain.com:389"
                userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"
                roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
                roleName="cn"
                roleSearch="(member={0})" />

</Realm>
OpenID Connect Authentication, LDAP Authorization
The following configuration steps apply to Tomcat 8.5. OpenID Connect is not supported for Tomcat 7.

To configure authentication with OpenID Connect:

  1. Download the latest Open ID Authenticator for Tomcat. The file is named tomcat8-oidcauth-<version_number>.jar

  2. Copy the tomcat8-oidcauth-<version_number>.jar file to the $TOMCAT_HOME/lib directory.

  3. Edit the semarchy.xml configuration file and create the OpenID valve and realm configuration.
    A sample OpenID Connect configuration using Google authentication is provided below.

Sample Configuration: OpenID Connect (Google) authentication, LDAP authorization
<!-- Valve configuration for OpenID Connect authentication -->

<Valve className="org.bsworks.catalina.authenticator.oidc.OpenIDConnectAuthenticator"
        providers="https://accounts.google.com,GOOGLE_CLIENT_ID,GOOGLE_CLIENT_SECRET_KEY" (1)
        noForm="true" (2)
        usernameClaim="email"
        additionalScopes="email"
        hostBaseURI="http://mdm_host:port" (3)
        landingPage="/"/>

<!--Realms configuration -->

<!--
The following LDAP realm provides the authorizations of the OpenID Connect-authenticated users.
In this LDAP, each user has a name equal to the OpenID Connect username.
Any Tomcat realm can be used for this purpose.
-->
<Realm className="org.apache.catalina.realm.JNDIRealm"
        connectionURL="ldap://ldaphost.mydomain.com:389" (4)
        userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"
        roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
        roleName="cn"
        roleSearch="(member={0})"/>

The parameters of the valve and realm must be customized to your configuration:

1 The Google Client ID and Google Client Key are retrieved from the Google Developer Console. Refer to the Google Identity Platform site for detailed instructions.
2 This property indicates whether the login form should be available or not.
3 The OpenID Connect connector will return to the original server after authentication. In most cases (when the server uses HTTPS), this mechanism works. If the server does not use HTTPS, it is recommended to set the host and port of the Semarchy xDM instance in hostBaseURI. You must declare this redirect URI in the Google Developer Console. See the Google Identity Platform site for detailed instructions.
4 This LDAP host corresponds to the LDAP server containing the roles for your users. see the Tomcat Realm Documentation for more information about LDAP configuration. Note that the LDAP user ID must be equal to the OpenID Connect User ID (in that case, the email).
OpenID Connect and Form Authentication, LDAP Authorization

Mixing OpenID Connect with a form authentication provides the possibility to log in using an OpenID Connect account or a login form. Authentication via the login form is made against an authentication provider, for example an LDAP directory or a database of users stored in the application server’s disk.

The following configuration steps apply to Tomcat 8.5. OpenID Connect is not supported for Tomcat 7.

To configure mixed authentication with OpenID Connect:

  1. Download the latest Open ID Authenticator for Tomcat. The file is named tomcat8-oidcauth-<version_number>.jar

  2. Copy the tomcat8-oidcauth-<version_number>.jar file to the $TOMCAT_HOME/lib directory.

  3. Copy the temp/mdm-server/additional-libraries/com.semarchy.tool.jee.tomcat-<version>.jar file to the $TOMCAT_HOME/lib directory. This file is available from the semarchy-xdm-install-xxx.zip archive file you downloaded.

  4. Edit the semarchy.xml configuration file and create an the OpenID valve and realm configuration.
    Sample OpenID Connect configurations using Google authentication are provided below.

OpenID Connect and Form Authentication (LDAP), LDAP Authorization

This sample uses OpenID connect for authentication, plus an LDAP directory for login form authentication and for all users' authorizations.

Sample Configuration: OpenID Connect (Google) + Login Form (LDAP) authentication, LDAP authorization
<!-- Valve configuration for mixed OpenID Connect authentication -->

<Valve className="org.bsworks.catalina.authenticator.oidc.OpenIDConnectAuthenticator"
        providers="https://accounts.google.com,GOOGLE_CLIENT_ID,GOOGLE_CLIENT_SECRET_KEY" (1)
        noForm="false"
        usernameClaim="email"
        additionalScopes="email"
        hostBaseURI="http://mdm_host:port" (2)
        landingPage="/"/>

<!--Realms configuration -->

<!--
The realm provides authorizations for the OpenID Connect-users.
It also provides authentication for form users.
This realm uses LDAP with each username equal to the OpenID username.
Any Tomcat realm can be used for this purpose.
-->
<Realm className="org.apache.catalina.realm.JNDIRealm"
        connectionURL="ldap://ldaphost.mydomain.com:389" (3)
        userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"
        roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
        roleName="cn"
        roleSearch="(member={0})" />

The parameters of the valve and realm must be customized to your configuration:

1 The Google Client ID and Google Client Key are retrieved from the Google Developer Console. Refer to the Google Identity Platform site for detailed instructions.
2 The OpenID Connect connector will return to the original server after authentication. In most cases (when the server uses HTTPS), this mechanism works. If the server does not use HTTPS, it is recommended to set the host and port of the Semarchy xDM instance in hostBaseURI. You must declare this redirect URI in the Google Developer Console. See the Google Identity Platform site for detailed instructions.
3 This LDAP host corresponds to the LDAP server containing the roles for your OpenID Connect users, or the users to authenticate via the form. see the Tomcat Realm Documentation for more information about LDAP configuration.
OpenID Connect and Form Authentication (UserDatabase), UserDatabase Authorization

This sample uses:

  • The OpenIdConnectAuthenticator valve to authenticate used via OpenID Connect.

  • A Tomcat UserDataBase Realm that stores in the tomcat-users.xml file both authentications and authorizations for users using the login form.

  • Another Tomcat UserDataBase Realm (openid-users.xml file) stores the authorizations for OpenID Connect-authenticated users.

Sample Configuration: OpenID Connect (Google) + Login Form (UserDatabase) authentication, UserDatabase authorization
<!-- Valve configuration for mixed OpenID Connect authentication -->

<Valve className="org.bsworks.catalina.authenticator.oidc.OpenIDConnectAuthenticator"
        providers="https://accounts.google.com,GOOGLE_CLIENT_ID,GOOGLE_CLIENT_SECRET_KEY" (1)
        noForm="false"
        usernameClaim="email"
        additionalScopes="email"
        hostBaseURI="http://mdm_host:port" (2)
        landingPage="/"/>

<!--Realms configuration -->
<Realm className="org.apache.catalina.realm.CombinedRealm">
        <!--
        The first realm provides authorizations for the OpenID Connect-users.
        They are in an "OpenIDDatabase" file resource, declared in server.xml, into
        which each user has name and password equal to the OpenID Connect username.
        -->
        <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="OpenIDDatabase"/>
        <!--
        The next realm is used for authenticating users via the login form.
        It uses the default /conf/tomcat-users.xml file.
        -->
         <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                resourceName="UserDatabase"/>
</Realm>
Configuration in /conf/server.xml to declare the file resource containing the OpenID Roles.
<!--
The resource named "OpenIDDatabase" used in the realm configuration
must be declared as the /conf/openid-users.xml file stored in the
application server's file system.
-->
<Resource name="OpenIDDatabase" auth="Container"
          type="org.apache.catalina.UserDatabase"
          description="User database for OpenID Connect Roles"
          factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
          pathname="conf/openid-users.xml" />
Sample /conf/openid-users.xml file. Note that username and password are equal. This file should be used only for mapping roles to usernames, and not for authentication.
<user username="user01@domain.com" password="user01@domain.com" roles="semarchyConnect,semarchyAdmin"/>
<user username="user01@domain.com" password="user01@domain.com" roles="semarchyConnect,dataSteward"/>

Windows Authentication Using Waffle

Windows Authentication (SSO) is supported in Tomcat using the Waffle (Windows Authentication Framework) component.
Using this mechanism, the user connected to the windows machine is used to authenticate to Semarchy xDM.

The following configuration only works when the machine running the Semarchy xDM Server is a Windows server joined to the domain you want to use for authentication.

To configure Windows Authentication:

  1. Download and Uncompress Waffle.

  2. Copying the waffle-jna.jar, guava-*.jar, jna-*.jar, platform-*.jar, slf4j*.jar and waffle-tomcat7.jar files to the $TOMCAT_HOME/lib directory.

  3. Copy the temp/mdm-server/additional-libraries/com.semarchy.tool.jee.tomcat-<version>.jar file to the $TOMCAT_HOME/lib directory.

  4. Edit the semarchy.xml file and add the valve and realm configuration and then restart the server.

To enable Windows authentication, use the following configuration.

Sample Configuration: Windows authentication and authorization
<Valve className="com.semarchy.tool.jee.tomcat.RoleMappingNegotiateAuthenticator"
        principalFormat="fqn"
        roleFormat="fqn"
        protocols="NTLM"
        />

<Realm className="waffle.apache.WindowsRealm" />

To mix Login Form and Windows authentication use the following configuration:

Sample Configuration: Windows + Login Form Authentication
<parameter name="SingleSignOn"
           value="action=j_negotiate_check"
           override="true" />

<Valve className="com.semarchy.tool.jee.tomcat.RoleMappingMixedAuthenticator"
        principalFormat="fqn"
        roleFormat="fqn"
        protocols="NTLM"
        />

<Realm className="waffle.apache.WindowsRealm" />
Both these configurations use specific extensions to the Waffle authenticator valves in order to support role mapping. You can configure a role mapping file as described in the Using the Tomcat Role Mapper section. If you do not need the role mapping, you can use the Waffle NegotiateAuthenticator or MixedAuthenticator valves instead.
These configurations is valid for Waffle versions starting with version 1.7. For versions before 1.7, replace the protocols="NTML" element by disableNegociate=”true".

Browser Configuration

Browsers must be specifically configured to work with Windows Authentication:

  • Internet Explorer: The Tomcat server must be considered as Intranet host.

  • Google Chrome: The configuration is shared with Internet Explorer.

  • Firefox:

    1. Open a new Tab.

    2. Enter about:config in the address bar.

    3. Enter network.negotiate-auth.trusted-uris in the Filter box.

    4. Enter the Semarchy xDM server name as the value. If you have multiple servers, enter a comma separated list.

    5. Close the tab.

Using the Tomcat Role Mapper

When the service providing the authorizations returns groups of users (if it does not support the concept of Roles) or role names that cannot exactly match the roles declared in Semarchy xDM (for example, if they include spaces or special characters), you have to configure a mapping between these role names and the Semarchy xDM role names.
Semarchy xDM provides a specific component for this purpose, called the Role Mapper.

This component is available as a jar named com.semarchy.tool.jee.tomcat-<version>.jar in your Semarchy xDM installation package.

The role mapper is compabible with Tomcat 8 or above and Waffle 1.7 and above.

To configure the Role Mapper:

  1. Copy the temp/mdm-server/additional-libraries/com.semarchy.tool.jee.tomcat-<version>.jar file to the $TOMCAT_HOME/lib directory.

  2. Create a $TOMCAT_HOME/conf/roles-mapping.properties file to define the role mapping.

  3. Edit the semarchy.xml file and add the following valve and realm configuration:

<Realm className="com.semarchy.tool.jee.tomcat.RoleMappingRealm">
        <!-- This is the Realm to which role mapping is applied. -->
        <Realm ClassName="org.apache.catalina.realm.JNDIRealm"
        ...
        />
</Realm>
The role mapper realm must enclose the realm for which it does the role mapping.
If you use Windows Authentication Using Waffle, note that the RoleMappingNegotiateAuthenticator and RoleMappingMixedAuthenticator valves support natively role mapping and do not require additional realm configuration.
Role Mapping File format

The role mapping file contains one line per role mapping, in the format:

        <directory_group>=<semarchy_role_1>,<semarchy_role_2>,...

The group names may contain spaces. In this situation, it should be kept as is and NOT enclosed in single or double quotes.

Examples of Role Mapping
AdministratorsGroup=semarchyConnect,semarchyAdmin
UsersGroup=semarchyConnect
Data Stewards=demoDataStewards

In certain cases with Microsoft Active Directory, group names with special characters such as spaces or backslashes require that you replace them with their Unicode equivalent them as shown below:

Examples of Role Mapping with escaped spaces (\u0020) and backslashes (\u005C)
MDM\u0020Users=semarchyConnect
Global\u005CStewards=semarchySteward

The role mapper supports role mapping using regular expressions and match groups patterns to convert input roles into different roles. This requires setting the role mapping realm’s regexEnable attribute to true.

Example of a Role Mapping using Regular Expressions
SUPP_.* = Supplier 	(1)
SUPP_(.*) = $1		(2)
(.*) = $1		(3)

If the example above, an incoming role named SUPP_Premium would convert into the Supplier, Premium `and `SUPP_Premium roles, due to the following rules:

1 If a role starts with SUPP_, then the Supplier role is mapped as the output.
2 If a role starts with SUPP_, then the rest of the role string captured as a group is returned as a role. For example: SUPP_Premium as an input role would return Premium as the output.
3 The entire input role is captured as a group and returned as an output.
Advanced Role Mapping Configuration

You can use the following attributes to configure the role mapper:

  • regexEnable: Set to true to enable role replacement using regular expressions. This option defaults to "false".

  • keepMappedRoles: Set to true to preserve in the user roles list the original roles that have been mapped. Otherwise, these roles are replaced by their mapped value. This option defaults to "true". This option replaces the deprecated replaceRole option. If keepMappedRoles or keepUnmappedRoles is set, then replaceRole is ignored.

  • keepUnmappedRoles: Set to true to preserve in the user roles list the original roles that have not been mapped. Otherwise, these roles are removed from the list. This option defaults to "true".

  • rolesMappingPathName: Location of the role mapping file. By default this file is located in $TOMCAT_HOME/conf/roles-mapping.properties

Advanced Role Mapper configuration: Regular expressions are supported. All original roles are removed.
<Realm className="com.semarchy.tool.jee.tomcat.RoleMappingRealm"
       keepMappedRoles="false"
       keepUnmappedRoles="false"
       regexEnable="true"
       rolesMappingPathName="/home/user/map.properties">

Using the Tomcat Wrapper Realm

To simplify the configuration of user authentication and authorizations in Apache Tomcat, Semarchy xDM comes with a Wrapper realm that supports two nested realms:

  • The first realm defines user authentication, checking that the login/password combination is valid and connecting the user.

  • The second realm defines how to retrieve user authorizations, that is the list of roles associated with the connected user after authentication).

Wrapper Realm configuration: Authenticate with LDAP, retrieve roles from a database
<Realm className="com.semarchy.tool.jee.tomcat.AuthAndRolesRealm">

        <!-- First realm is for authentication -->
        <Realm className="org.apache.catalina.realm.JNDIRealm"
               connectionURL="ldap://ldaphost.mydomain.com:389"
               userPattern="uid={0},ou=users,ou=people,dc=myCompany,dc=com"
               roleBase="ou=groups,ou=people,dc=myCompany,dc=com"
               roleName="cn"
               roleSearch="(member={0})"
               />

        <!-- Second realm is for authorizations -->
        <Realm className="org.apache.catalina.realm.JDBCRealm"
               driverName="oracle.jdbc.driver.OracleDriver"
               connectionURL="jdbc:oracle:thin:@dbserver:1521:ora11"
               userRoleTable="user_roles"
               userNameCol="user_name"
               roleNameCol="role_name"
               />

</Realm>

This wrapper can be used in combination with other Tomcat realms features, such as combined realms to enable fallback.

Wrapper Realm configuration with Tomcat’s combined realm to enable fallback
<Realm className="org.apache.catalina.realm.CombinedRealm">

    <!-- default authentication and authorization realm -->
    <Realm className="com.semarchy.tool.jee.tomcat.AuthAndRolesRealm">
                <!-- First realm is for authentication -->
                <Realm className="org.apache.catalina.realm.JNDIRealm"
                       ...
                />
                <!-- Second realm is for authorizations -->
                <Realm className="org.apache.catalina.realm.JDBCRealm"
                       ...
                />
   </Realm>

    <!-- fallback realm #1: authentication and authorization realm -->
   <Realm className="com.semarchy.tool.jee.tomcat.AuthAndRolesRealm">
                <Realm className="org.apache.catalina.realm.JNDIRealm"
                       ...
                />
                <Realm className="org.apache.catalina.realm.JDBCRealm"
                       ...
                />
   </Realm>

   <!-- fallback #2: using tomcat users for both authentication and authorizations -->
   <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>

</Realm>

Configuring a JMS Destination in Tomcat

Semarchy xDM can use a JMS (Java Message Service) provider as a notification server, in order to send job completion notifications to other applications in the form of JMS messages.

In most cases, the resource definition is generic and follows the Tomcat generic guidelines.

Certain JMS servers do not precisely follow the JMS Standards and require specific mechanisms and configuration to connect and access their JMS destinations. For these cases, Semarchy xDM includes a generic JNDI lookup factory to request a JNDI Connections and Resources (Queue or Topic).

This component is available as a jar named com.semarchy.tool.jee.tomcat-<version>.jar in your Semarchy xDM installation package.

Configuring JMS Using the JNDI Lookup Factory

To configure the JNDI Lookup Factory:

  1. Copy the temp/mdm-server/additional-libraries/com.semarchy.tool.jee.tomcat-<version>.jar file to the $TOMCAT_HOME/lib directory.

  2. Copy the client libraries (.jar files) required for your JMS Server to the $TOMCAT_HOME/lib directory.

  3. Edit the semarchy.xml file and add the following resource declarations:

Connection Factory resource definition
<!-- The Connection Factory encapsulates a set of connection configuration parameters.
     It is used to create a connection with the JMS provider. -->

<Resource name="jms/<connection_factory_name>"                        (1)
        auth="Container"
        type="javax.jms.ConnectionFactory"
        factory="com.semarchy.tool.jee.tomcat.jndi.JndiLookupFactory"
        jndiKey="<connection_factory_jndi_location_in_provider>"(2)
        initialCtxFactory="<initial_context_factory>"                (3)
        providerUrl="<provider_url>"                                (4)
        username = "<jms_server_login>"                                (5)
        password = "<jms_server_password>"                        (5)
/>
<!-- Depending on the JMS provider, you must provide additional parameters.
     For example:
     java.naming.security.protocol = "ssl"
-->

The Connection Factory resource definition uses the following parameters:

1 The Tomcat resource name for your JMS connection factory.
2 The location of the connection factory JNDI resource in the remote JNDI provider.
3 The initial context factory class, specific to the JNDI provider.
4 The URL of the JNDI provider.
5 The login and password required to access the JNDI resource, if required.
JMS Destination resource definition
<!-- A destination is a JMS Queue or Topic into which
     {mdm-regular-product-name} posts notifications.
-->

<Resource name="jms/<destination_name>"                         (1)
        auth="Container"
        type="javax.jms.Queue"                                        (2)
        factory="com.semarchy.tool.jee.tomcat.jndi.JndiLookupFactory"
        jndiKey="<destination_jndi_location_in_provider>"         (3)
        initialCtxFactory="<initial_context_factory>"                 (4)
        providerUrl="<provider_url>"                                 (5)
        username = "<jms_server_login>"                                (6)
        password = "<jms_server_password>"                        (6)
/>
<!-- Depending on the JMS provider, you must provide additional parameters.
     For example:
     java.naming.security.protocol = "ssl"
-->

The JMS destination resource definition uses the following parameters:

1 The resource name of your JMS destination in Tomcat.
2 Type of the JMS destination. Can be a Queue or Topic.
3 The location of the JMS destination JNDI resource in the remote JNDI provider.
4 The initial context factory class, specific to the JNDI provider.
5 The URL of the JNDI provider.
6 The login and password required to access the JNDI resource, if required.

Using JMS Destinations from the JNDI Lookup Factory

When configuring the Notification Server and Job Notification Policy, or order to use the JMS destination defined:

  • Use the jms/<connection_factory_name> value in the notification server Connection Factory URL property.

  • Use the jms/<destination_name> value in the job notification JMS Destination property.

Both those values must be prefixed with java:comp/env/ when used.

Configuring and Branding the Login Page

You can configure the look and feel of the Semarchy xDM login, logout and error pages.

The following parameters can be passed in the semarchy.xml` file to configure this page:

  • SingleSignOnButtonLabel: Label used for the login method for SSO in the SSO Login button. For example, Google, Facebook, etc. This label is prefixed with a localized version of "Log in with ". This parameters is optional and defaults to SSO

  • SingleSignOnButtonIconURL: URL of the icon in the single sign-on button. Use a HTTP URL to a 24x24px PNG file. Defaults to account.png. We also include facebook-box.png, google.png, google-plus.png and twitter.png icons in the application, so you can refer to them directly.

  • SingleSignOnButtonColor: CSS background color of the SSO button.

  • SignOnImageURL: image that appears in the banner of the login/logout/error pages. Should be a 300x100px PNG file. If not specified the Semarchy xDM logo is used.

  • SignOnBottomMessage: HTML text that appears at the bottom the page (under the Log in button).

Login page branding & configuration example
<!-- Enable SSO with Google along with the login form -->
<Parameter name="SingleSignOn" value="openid_identifier=https://www.google.com/accounts/o8/id" override="true"/>

<!-- Google Login button branding -->
<Parameter name="SingleSignOnButtonLabel" value="Google" />
<Parameter name="SingleSignOnButtonIconURL" value="google.png" />
<Parameter name="SingleSignOnButtonColor" value="#4885ed" />

<!-- Login/Logout/Error pages image banner and footer text -->
<Parameter name="SignOnImageURL" value="http://localhost:80/staticAssets/banner.png" />
<Parameter name="SignOnBottomMessage" value="Please use this application <b>wisely</b>" />

Deploying and Configuring with JBoss/WildFly

This section explains how to configure and deploy the Semarchy xDM Application with WildFly Application Server (formerly JBoss AS).

In this section, <wildfly_home> refers to the Wildfly server installation folder.

Refer to the WildFly Documentation for your WildFly version for more details about the deployment and configuration processes in WildFly.

Installing Additional Libraries

Installing the Database JDBC Driver

To install the Database JDBC driver:

  1. Copy the database driver file (temp/mdm-server/additional-libraries/ojdbc6.jar for Oracle or temp/mdm-server/additional-libraries/org.postgresql.jdbc<version>.jar for PostgreSQL) to the <wildfly_home>/standalone/deployments directory.

Configuring the Security

The configuration in this section uses the UsersRolesLoginModule (properties file based login module) and may be changed to a stronger authentication mechanism.

The following example explains how to configure the semadmin user for Wildfly in the default configuration.

To configure the security realm:

  1. Go to the <wildfly_home>/bin folder and start the add-user.sh or add_user.bat script

  2. Selection option b) Application User.

  3. Select a realm or press Enter.

  4. Enter semadmin for the Username and then press Enter.

  5. Enter this user password and then press Enter.

  6. Re-enter the password and then press Enter.

  7. Enter the following list of roles: semarchyConnect,semarchyAdmin and then press Enter.

  8. Enter y to confirm the user creation.

  9. Enter n as this user is not used for one AS process to connect another AS process.

Setting up the Datasources

To configure the repository datasource:

  1. Connect to the Administration Console.

  2. In the Profile section, select Connector > Datasources.

  3. Configure a datasource with the following parameters:

    • Name: SEMARCHY_REPOSITORY

    • JNDI: jdbc/SEMARCHY_REPOSITORY

    • Driver: odjcb6.jar for Oracle or postgres for PostgreSQL

    • Username: <repository_user_name>

    • Password: <repository_user_password>

    • Connection URL:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

  4. Save this configuration and make sure it is enabled.

To configure a data location datasource:

  1. Connect to the Administration Console.

  2. In the Profile section, select Connector > Datasources.

  3. Configure a datasource with the following parameters:

    • Name: <data_location_datasource_name>

    • JNDI: jdbc/<data_location_datasource_name>

    • Driver: odjcb6.jar for Oracle or postgres for PostgreSQL

    • Username: <data_location_user_name>

    • Password: <data_location_user_password>

    • Connection URL:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

  4. Save this configuration and make sure it is enabled.

  5. Repeat this operation for each data location’s datasource.

The repository and data location datasources are now configured, pointing to the schemas previously created.

Deploying the Application

To deploy the application:

  1. Copy the temp/mdm-server/semarchy.war file to the <wildfly_home>/standalone/deployments/ folder.

The Semarchy xDM application is deployed in the server.

Configuring JavaMail Session

This configuration is required for mail notifications using JEE Mail Session.

To configure JavaMail Session:

  1. Edit the the domain.xml or standalone-full.xml configuration file and create the mail subsystem to match your configuration. See https://docs.jboss.org/author/display/AS71/Mail+Subsystem for more information.

Testing the Application

To test the application:

  1. Open a web browser.

  2. In the URL, enter: http:/<wildfly_host>:<wildfly_port>/semarchy/.

Proceed directly to the Installing the Repository task.

Deploying and Configuring with GlassFish

This section explains how to configure and deploy the Semarchy xDM Application with Glassfish Application Server.

In this section, <glassfish_home> refers to the Glassfish server installation folder.

Refer to the Glassfish Documentation for more details about the deployment and configuration processes in Glassfish.

Installing Additional Libraries

Installing the Database JDBC Driver

To install the Database JDBC driver:

  1. Copy the database driver file (temp/mdm-server/additional-libraries/ojdbc6.jar for Oracle or temp/mdm-server/additional-libraries/org.postgresql.jdbc<version>.jar for PostgreSQL) to the <glassfish_home>/glassfish/lib directory.

After installing the libraries, restart the Glassfish server.

Configuring the Security

The configuration in this section uses the default File Realm and may be changed to your enterprise’s type of realm.

Configuring the Security Realm and Semarchy xDM Administrator

To configure the security realm:

  1. Open the Glassfish WebAdmin interface (http://<glassfish_host>:4848).

  2. In the Common Tasks panel, select Configuration > Server-config > Security > Realms.

  3. Click the New button to create a new realm with the following properties:

    • Name: semarchyRealm

    • ClassName: com.sun.entreprise.security.auth.realm.file.FileRealm

    • JAAS Context: fileRealm

    • Key file: ${com.sun.aas.instanceRoot}/config/keyfile

  4. Click OK to save the new realm.

To configure the semadmin user:

  1. Click the new semarchyRealm and then select the Manage Users button.

  2. Click the New button to create a new user with the properties:

    • User ID: semadmin

    • Group List: semarchyAdminGroup,semarchyConnectGroup

    • Password: <semadmin_password>

  3. Click OK to save the new user.

Configuring Groups/Roles Mappings

The configured realm uses the default Java Authorization Contract for Containers (JACC) provider included in Glassfish. This JACC provider does not support dynamic roles, and mandates that the mappings between Groups and Roles are defined in the deployed application descriptor file.

To define the groups/roles mappings:

  1. Uncompress the /temp/mdm-server/semarchy.war file into the temp/semarchy_war/ folder.

  2. Edit the temp/semarchy_war/semarchy/WEB-INF/glassfish-web.xml file.

  3. Add the section given below in <glassfish-web-app> element and then save the file.

<security-role-mapping>
 <role-name>semarchyConnect</role-name>
 <group-name>semarchyConnectGroup</group-name>
</security-role-mapping>
<security-role-mapping>
 <role-name>semarchyAdmin</role-name>
 <group-name>semarchyAdminGroup</group-name>
</security-role-mapping>
To add new Semarchy xDM roles after the initial setup and map them to Glassfish groups, you must modify this file and redeploy the application.

Setting up the Datasources

To configure the repository datasource:

  1. Open the Glassfish WebAdmin interface (http://<glassfish_host>:4848).

  2. In the Common Tasks panel, select Resources > JDBC > JDBC Connection Pools.

  3. Click the New button to create a new connection pool with the following properties:

    • Pool Name: SEMARCHY_REPOSITORY

    • Resource Type: java.sql.ConnectionPoolDataSource

    • Database Driver Vendor: Oracle or PostgreSQL

  4. Click Next. Set the following additional properties:

    • User: <repository_user_name>

    • Password: <repository_user_password>

    • URL:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

    • MaxStatements: 50

  5. Click Finish.

  6. Select the new connection pool and click the Ping button to test it.

  7. In the Common Tasks panel, select Resources > JDBC > JDBC Resources.

  8. Click the New button to create a new JDBC resource with the following properties:

    • JNDI Name: jdbc/SEMARCHY_REPOSITORY

    • Pool Name: SEMARCHY_REPOSITORY

    • Status: Enabled

  9. Click OK.

In the Pool Settings, it is recommended to tune the Minimum Pool Size and Maximum Pool Size properties according to your needs. Having a pool size between 1 and 8 connections is typically sufficient for testing purposes.
Do not change the JNDI Name of the SEMARCHY_REPOSITORY datasource. The application refers to a datasource with this name for the repository.

To configure a data location datasource:

  1. Open the Glassfish WebAdmin interface (http://<glassfish_host>:4848).

  2. In the Common Tasks panel, select Resources > JDBC > JDBC Connection Pools.

  3. Click the New button to create a new connection pool with the following properties:

    • Pool Name: <data_location_datasource_name>

    • Resource Type: java.sql.ConnectionPoolDataSource

    • Database Driver Vendor: Oracle or PostgreSQL

  4. Click Next. Set the following additional properties:

    • User: <data_location_user_name>

    • Password: <data_location_user_password>

    • URL:

      • Oracle: jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name>

      • PostgreSQL: jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>

    • MaxStatements: 50

  5. Click Finish.

  6. Select the new connection pool and click the Ping button to test it.

  7. In the Common Tasks panel, select Resources > JDBC > JDBC Resources.

  8. Click the New button to create a new JDBC resource with the following properties:

    • JNDI Name: jdbc/<data_location_datasource_name>

    • Pool Name: <data_location_datasource_name>

    • Status: Enabled

  9. Click OK.

In the Pool Settings, it is recommended to tune the Minimum Pool Size and Maximum Pool Size properties according to your needs. Having a pool size between 1 and 8 connections is typically sufficient for testing purposes.

Repeat this operation for each data location’s datasource.

The repository and data location datasources are now configured, pointing to the schemas previously created.

Configuring JavaMail Session

This configuration is required for mail notifications using JEE Mail Session.

To configure JavaMail Session:

  1. Open the Glassfish WebAdmin interface (http://<glassfish_host>:4848).

  2. In the Common Tasks panel, select Resources > JavaMail Sessions.

  3. Click the New button to create a new JavaMail Session with the following properties:

    • JNDI Name: mail/Session

    • Mail Host: <mail_server_host>

    • Default User: <mail_user_name>

    • Transport Protocol: smtp

  4. Add the following additional property to enable SMTP authentication:

    • mail.smtp.auth: true

    • password: <mail_user_password>

  5. Click OK.

Deploying the Application

To deploy the application:

  1. Open the Glassfish WebAdmin interface (http://<glassfish_host>:4848).

  2. In the Common Tasks panel, select Applications.

  3. Click the Deploy… button.

  4. Select Local Packaged File or Directory …, and then click Browse Folders…

  5. Select the temp/semarchy_war/semarchy/ folder in the folder browser and click Choose Folder.

  6. Select Web Application for the Type and make sure the Status is Enabled.

  7. Click OK to deploy the application.

The Semarchy xDM application is deployed in the server.

Testing the Application

To test the application:

  1. Open a web browser.

  2. In the URL, enter: http:/<glassfish_host>:<glassfish_port>/semarchy/.

Proceed directly to the Installing the Repository task.

Deploying and Configuring with Jetty

This section explains how to configure and deploy the Semarchy xDM Application with the Eclipse Jetty Application Server.

In this section, <jetty_home> refers to the Jetty server installation folder.

Refer to the Jetty Documentation for more details about the deployment and configuration processes in Jetty.

Installing Additional Libraries

Installing the Database JDBC Driver

To install the Database JDBC driver:

  1. Copy the database driver file (temp/mdm-server/additional-libraries/ojdbc6.jar for Oracle or temp/mdm-server/additional-libraries/org.postgresql.jdbc<version>.jar for PostgreSQL) to the <jetty>/lib/ext directory.

Installing the Pooling Libraries

This configuration uses connection pooling available in DBCP.

To install the Connection Pooling Libraries:

  1. Download the Apache DBCP and Apache Common Pool.

  2. Uncompress both these files and then and copy the commons-dbcp-1.4.jar and commons-pool-1.6.jar files to the <jetty>/lib/ext directory.

Configuring the Security

To create the Semarchy xDM Realm:

  1. Edit the <jetty>/etc/jetty.xml file and add the realm definition as described in the example below.

<Call name="addBean">
  <Arg>
    <New class="org.eclipse.jetty.security.HashLoginService">
      <Set name="name">semarchyRealm</Set>
      <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
      <Set name="refreshInterval">0</Set>
    </New>
  </Arg>
</Call>

This realm uses file storage for the users and roles.

To add the semadmin user to the realm:

  1. Create or edit the <jetty>/etc/realm.properties file and add the users in the following format:

<user_name>: <password>[,<role_1>, <role_2>]

An example is given below:

Creating a semadmin user.
semadmin: <semadmin_password>, semarchyAdmin,semarchyConnect
myuser: <my_password>,semarchyConnect, dataSteward
All passwords configured in Jetty can be obfuscated. See the Secure Password Obfuscation chapter in the Jetty Documentation for more information.

Setting up the Datasources

To configure the repository and data location datasources using DBCP connection pooling:

  1. Edit the <jetty>/etc/jetty.xml file and add the new JNDI resources shown in the template below.

<!-- Repository Datasource -->
<New id="SEMARCHY_REPOSITORY" class="org.eclipse.jetty.plus.jndi.Resource">
   <Arg></Arg>
   <Arg>jdbc/SEMARCHY_REPOSITORY</Arg>
   <Arg>
    <New class="org.apache.commons.dbcp.BasicDataSource">
          <!-- Configuraton for Oracle -->
          <Set name="driverClassName">oracle.jdbc.OracleDriver</Set>
          <Set name="url">jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name></Set>

          <!-- Configuration for PostgreSQL -->
          <!-- <Set name="driverClassName">org.postgresql.Driver</Set> -->
          <!-- <Set name="url">jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>`</Set> -->

          <Set name="username"><repository_user_name></Set>
          <Set name="password"><repository_user_password></Set>
     </New>
   </Arg>
  </New>


<!-- Repeat the following element for each data location -->
<New id="<data_location_datasource_name>" class="org.eclipse.jetty.plus.jndi.Resource">
   <Arg></Arg>
   <Arg>jdbc/<data_location_datasource_name></Arg>
   <Arg>
    <New class="org.apache.commons.dbcp.BasicDataSource">
          <!-- Configuraton for Oracle -->
          <Set name="driverClassName">oracle.jdbc.OracleDriver</Set>
          <Set name="url">jdbc:oracle:thin:@<oracle_instance_hostname>:<oracle_listener_port>:<oracle_SID_name></Set>

          <!-- Configuration for PostgreSQL -->
          <!-- <Set name="driverClassName">org.postgresql.Driver</Set> -->
          <!-- <Set name="url">jdbc:postgresql://<postgresql_hostname>:<postgresql_port>/<postgresql_database_name>`</Set> -->

          <Set name="username"><data_location_user_name></Set>
          <Set name="password"><data_location_user_password></Set>
     </New>
   </Arg>
  </New>

Configuring JavaMail Session

  1. Edit the <jetty>/etc/jetty.xml file and add the new JavaMail resource as described in the template below.

<New id="mail" class="org.eclipse.jetty.plus.jndi.Resource">
     <Arg><Ref refid="wac"/></Arg>
     <Arg>mail/Session</Arg>
     <Arg>
       <New class="org.eclipse.jetty.jndi.factories.MailSessionReference">
         <Set name="user"><mail_user_name></Set>
         <Set name="password"><mail_user_password></Set>
         <Set name="properties">
           <New class="java.util.Properties">
             <Put name="mail.transport.protocol">smtp</Put>
             <Put name="mail.smtp.host"><mail_server_host></Put>
             <Put name="mail.smtp.port"><mail_server_port></Put>
             <Put name="mail.from"><mail_from_user_name></Put>
             <Put name="mail.debug">true</Put>
           </New>
          </Set>
       </New>
     </Arg>
</New>

Deploying the Application

To deploy the application:

  1. Copy the temp/mdm-server/semarchy.war file to the <jetty>/webapp/ folder.

  2. Open a command line in the <jetty> folder and then run java -jar start.jar to start the Jetty Server.

The Semarchy xDM application is deployed and the server started.

Testing the Application

To test the application:

  1. Open a web browser.

  2. In the URL, enter: http:/<jetty_host>:<jetty_port>/semarchy/.

Proceed directly to the Installing the Repository task.

Deploying and Configuring with Oracle WebLogic

This section explains how to configure and deploy the Semarchy xDM Application with Oracle WebLogic Application Server.

Refer to the WebLogic Documentation for more details about the deployment and configuration processes in WebLogic.
Prior to configuring Semarchy xDM, configure and start the WebLogic domain into which you plan to deploy and configure the Semarchy xDM application.

Installing Additional Libraries

Installing the Database JDBC Driver

Oracle WebLogic comes with a JDBC Driver for the Oracle database. For PostgreSQL, you must add the PostgreSQL driver file to the classpath variable of the WebLogic Server.

Configuring the Security

To create the groups:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Security Realms from the left pane and then click the realm you are configuring.

  3. Select the Users and Groups tab, then Groups.

  4. Click New.

  5. In the Create a New Group page provide the following information:

    • Name: semarchyConnectGroup

    • Provider: DefaultAuthenticator

  6. Click OK. The group is added to the Groups table.

  7. Click New.

  8. In the Create a New Group page provide the following information:

    • Name: semarchyAdminGroup

    • Provider: DefaultAuthenticator

  9. Click OK. The group is added to the Groups table.

To create the semadmin user:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Security Realms from the left pane and then click the realm you are configuring.

  3. Select the Users and Groups tab, then Users.

  4. Click New.

  5. In the Create a New User page provide the following information:

    • Name: semadmin

    • Provider: DefaultAuthenticator

    • Password: <semadmin_password>

  6. Click OK. The user is added to the Users table.

  7. Click the semadmin user from the Users table, then select the Groups tab.

  8. Select the semarchyAdminGroup and semarchyConnectGroup groups in the Available group list, then click the Add button to add them to the Chosen list.

  9. Click Save to save the group membership.

To configure the roles mappings:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Security Realms from the left pane and then click the realm you are configuring.

  3. Select Roles and Policies > Realm Roles.

  4. Expand Global Roles and then click on Roles.

  5. Select New to create a new role with the following properties:

    • Name: semarchyConnect

    • Provider Name: XACMLRoleMapper

  6. Click OK.

  7. Select the semarchyConnect role in the Roles table.

  8. Click Add Conditions.

  9. Select Group for the Predicate List and then click Next.

  10. In the Group Argument Name, enter semarchyConnectGroup and then click Add.

  11. Click Finish.

  12. Click Save.

  13. Select Security Realms from the left pane and then click the realm you are configuring.

  14. Select Roles and Policies > Realm Roles.

  15. Expand Global Roles and then click on Roles.

  16. Select New to create a new role with the following properties:

    • Name: semarchyAdmin

    • Provider Name: XACMLRoleMapper

  17. Click OK.

  18. Select the semarchyAdmin role in the Roles table.

  19. Click Add Conditions.

  20. Select Group for the Predicate List and then click Next.

  21. In the Group Argument Name, enter semarchyAdminGroup and then click Add.

  22. Click Finish.

  23. Click Save.

Setting up the Datasources

To configure the repository datasource:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Services > Data Sources from the left pane.

  3. Select New > Generic Data Source in the Data Sources table.

  4. Enter the following JDBC datasource properties:

    • Name: SEMARCHY_REPOSITORY

    • JNDI Name: jdbc/SEMARCHY_REPOSITORY

    • Database Type: Oracle or PostgreSQL

  5. Click Next.

  6. In the Database Driver select Oracle’s Driver (Thin) for Instance connections: Version:9.0.1 and later or PostgreSQL’s Driver (Type 4).

  7. Click Next.

  8. De-select the Supports Global Transaction option.

  9. Click Next.

  10. Enter the following connection properties:

    • For Oracle

      • Database Name: <oracle_SID_name>

      • Host Name: <oracle_instance_hostname>

      • Port: <oracle_listener_port>

      • Database User Name: <repository_user_name>

      • Password: <repository_user_password>

    • For PostgreSQL

      • Database Name: <postgresql_database_name>

      • Host Name: <postgresql_hostname>

      • Port: <postgresql_port>

      • Database User Name: <repository_user_name>

      • Password: <repository_user_password>

  11. Click Next.

  12. Click Test Configuration to validate the connection information.

  13. In the Properties field, add the following line: defaultAutoCommit=false

  14. Click Next.

  15. Select a deployment target (by default Admin Server) and then click Finish.

Do not change the JNDI Name of the SEMARCHY_REPOSITORY datasource. The application refers to a datasource with this name for the repository.

To configure the data location datasource:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Services > Data Sources from the left pane.

  3. Select New > Generic Data Source in the Data Sources table.

  4. Enter the following JDBC datasource properties:

    • Name: <data_location_datasource_name>

    • JNDI Name: jdbc/<data_location_datasource_name>

    • Database Type: Oracle or PostgreSQL

  5. Click Next.

  6. In the Database Driver select Oracle’s Driver (Thin) for Instance connections: Version:9.0.1 and later or PostgreSQL’s Driver (Type 4).

  7. Click Next.

  8. De-select the Supports Global Transaction option.

  9. Click Next.

  10. Enter the following connection properties:

    • For Oracle

      • Database Name: <oracle_SID_name>

      • Host Name: <oracle_instance_hostname>

      • Port: <oracle_listener_port>

      • Database User Name: <data_location_user_name>

      • Password: <data_location_user_password>

    • For PostgreSQL

      • Database Name: <postgresql_database_name>

      • Host Name: <postgresql_hostname>

      • Port: <postgresql_port>

      • Database User Name: <data_location_user_name>

      • Password: <data_location_user_password>

  11. Click Next.

  12. Click Test Configuration to validate the connection information.

  13. In the Properties field, add the following line: defaultAutoCommit=false

  14. Click Next.

  15. Select a deployment target (by default Admin Server) and then click Finish.

Repeat this operation for each data location’s datasource.

The repository and data location datasources are now configured, pointing to the schemas previously created.

Deploying the Application

To deploy the application:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Deployments from the left pane.

  3. Click the Install button.

  4. Click the Upload your file(s) link.

  5. Select for the Deployment Archive the temp/mdm-server/semarchy.war file on your local disk.

  6. Click Next and then Next.

  7. In the Choose target style page select Install this deployment as an application and then click Next.

  8. In the Optional Settings page, in the Security option group, select Custom Roles: Use roles that are defined in the Administration Console; use policies that are defined in the deployment descriptor.

  9. Click Next.

  10. In the Review your choices and click Finish page Select No, I will review the configuration later.

  11. Click Finish.

The Semarchy xDM application is deployed in the server.

Configuring JavaMail Session

This configuration is required for mail notifications using JEE Mail Session.

To configure JavaMail Session:

  1. Connect to the WebLogic Server Administration Console.

  2. Select Services > Mail Sessions from the left pane.

  3. Click the New button.

  4. In the Mail Session Properties page, enter the following properties:

    • Name: MailSession

    • JNDI Name: mail/Session

  5. In the JavaMail Properties, enter the following lines:

    • mail.transport.protocol=smtp

    • mail.smtp.host=<mail_server_host>

    • mail.port=<mail_server_port>

    • mail.smtp.auth=true

    • mail.smtp.user=<mail_user_name>

    • password=<mail_user_password>

  6. Click Save to save the mail session configuration.

Testing the Application

To test the application:

  1. Open a web browser.

  2. In the URL, enter: http:/<weblogic_host>:<weblogic_port>/semarchy/.

Proceed directly to the Installing the Repository task.

Installing the Repository

Make sure you have the license file provided by Semarchy before running the following steps.
The repository type cannot be modified after installation, make sure to choose a repository type adapted to your installation. Review carefully the description of the Repository Types.

Semarchy xDM holds all its information in a repository stored in a database schema. The first task when connecting to Semarchy xDM is to create this repository structure in the database schema previously created.

Repository installation is done the first time an administrator connects to the Semarchy xDM application. The repository installation process also performs the license registration.

  1. Open your web browser and connect to the following URL: http://<application_server_host>:<application_server_port>/semarchy/.

  2. In the login prompt, enter the following:

    • User: semadmin

    • Password: <semadmin_password>

  3. The Semarchy xDM Workbench opens with the license agreement. Review the End-User License Agreement.

  4. Check the I have read and accept Semarchy’s End-User License Agreement box and then click Next.

  5. In the License Key File page, select a valid license key by clicking the Load a License Key File button and then click Next.

  6. In the Repository Creation wizard, select the type of repository.

  7. Optionally enter a customized name for your repository.

  8. Click Finish to create the repository.

  9. Click OK when the Repository Successfully Created message appears.

The repository is created and Semarchy xDM is now up and running.

Next Steps

Semarchy xDM is now up and running.

This section describes the next recommended tasks for the Administrators and Project Managers.

Administrators

Administrators should proceed with the following tasks:

  • Setting up the Security:

    • Create users, groups and roles in the application server realm. Refer to the application server documentation for more information.

    • Declare the roles in Semarchy xDM and grant platform-level privileges to these roles. Refer to the Semarchy xDM Administration Guide for more information.

  • Configuring Notification Servers for sending emails and job notification. Refer to the Semarchy xDM Administration Guide for more information.

Project Managers

Project Managers should proceed to the following tasks:

  • Creating a new model. Refer to the Models Management chapter in the Semarchy xDM Developer’s Guide for more information.

  • Creating data locations using to the datasources configured when installing Semarchy xDM.