Semarchy xDM Architecture

This document details the various components of the Semarchy xDM architecture and their interactions.

Semarchy xDM Server

The Semarchy xDM Server is a Java EE application deployed and running in a supported application server.

This server provides several access methods:

  • The Application Builder, Dashboard Builder, Discovery, and Configuration user interfaces: These web applications are used by designers and administrators to create, manage and administer the models and applications designed in Semarchy xDM, as well as profile and measure data.

  • Data Management Applications and Dashboard Applications: These web applications are used by business users to browse and manage data and visualize metrics dashboards.

  • A Rest API to perform programmatically data integration, management, and administrative operations.

The Semarchy xDM server is used at design-time to design models and applications and deploy them. At run-time, it schedules and runs the certification process in the hub.

Active and Passive Servers

The Semarchy xDM server comes in two flavors corresponding to two WAR (Web Application Archive) files:

  • The Active server (semarchy.war) contains all the features of Semarchy xDM, including running the certification job. You should only have one active server in a typical configuration.

  • The Passive server (semarchy-passive.war) is unable to run the certification job. It is used in high-availability configurations to extend the active server with additional nodes to server users and API requests.

A single instance (either active or passive) of Semarchy xDM can be deployed in each application server instance. Performing two deployments on the same application server instance - for example: http://host:port/semarchy_production and http://host:port/semarchy_test - is not supported.

Repository

The Semarchy xDM server stores its information in a repository, hosted in an database/schema. A Semarchy xDM server is always attached to a single repository.

Repository Contents

The repository stores the following information:

  • Platform-level configuration elements: roles, server configuration, plugins, identity management, datasources, etc.

  • For Data Management applications and models:

    • The models and applications metadata and their versions

    • The configuration & security information: privileges, notification policies, preferences, etc.

    • Data locations information: deployed model, jobs, etc.

    • Run-time information: logs

  • For xDM Dashboard:

    • The dashboard applications and their artifacts (queries, charts, etc.)

  • For xDM Discovery:

    • The datasource definitions and the profiles.

Semarchy xDM uses role-based security to access the features applications. The users and their assigned roles used may be stored in the repository, or an external identity provider system.

Repository Type

There are two types of repositories. The repository type is selected at creation time and cannot be modified afterward.

The repository types are:

  • Design: All design-time and run-time operations are possible in this repository.

  • Deployment: With this repository type, you can only import closed model editions and cannot edit them.

The deployment repositories are suitable for production sites. Model transfer from design to deployment repositories is handled via incremental export/import of closed model editions.

Semarchy xDM Dashboard and Semarchy xDM Discovery work identically with Design and Deployment repositories. Both types of repositories support profiling and the design of dashboard applications.

Repository Datasources

The Semarchy xDM server connects the repository using two datasources:

  • The Repository Datasource, with credentials allowing to read from and write into the repository schema.

  • The Repository Read-Only Datasource, with credentials allowing to read a subset of the tables stored in the repository schema.
    This datasource is used mainly by xDM Discovery to browse the profiles.

Both these datasources are defined in the server startup configuration.

The repository data should never be modified directly via SQL queries. Access to the Semarchy xDM information must be performed through the Semarchy user interfaces.

Data Locations

Data managed in Semarchy applications is stored in a data location.

The data location:

  • Contains and serves the hub data, structured after the model entities and attributes.

  • Runs certification jobs generated after the rules (enricher, validation, matching, etc) defined in the model.

A model is deployed into a data location to host the hub data according to the model structure and rules. At run-time, the data location contains the golden, master, and source data, with all the lineage and history.

Data locations are hosted in a databases/schemas and accessed via platform datasource defined in Semarchy xDM.

Data Location Contents

A Data Location contains the hub data, stored in the database/schema accessed using the data location’s datasource. This schema contains database tables and other objects generated from the model edition.

The data location also refers to jobs stored in the repository:

  • Installation Jobs: The jobs for creating or modifying in a non-destructive way the data structures in the data location.

  • Integration Jobs: The jobs for certifying data in these data structures, according to the model job definitions.

  • Purge Jobs: The jobs for purging the logs and data history according to the retention policies.

Data Locations, Repositories and Models

A data location is attached to a repository: You can declare as many data locations as you want in a repository, but a data location is always attached to a single repository. It is not possible to have a data location attached to two repositories at the same time.

You may deploy several model editions successively in a data location, but only one model edition is deployed and is active in the data location at a certain point in time.

Data locations are only used to deploy Semarchy models designed in the Application Builder. Dashboard and charts created in Semarchy xDM Dashboard do not require data locations.

Data Location Types

There are two types of data locations. The type is selected when the data location is created and cannot be changed afterward:

The data location types are:

  • Development Data Locations: A data location of this type supports deploying open or closed model editions. This type of data location is suitable for testing models in development and quality assurance environments.

  • Production Data Location: A data location of this type supports deploying only closed model editions. This type of data location is suitable for deploying MDM hubs in production environments.

Datasources

Semarchy xDM connects to multiple databases and schemas: