Deployment

This process is the deployment in a run-time environment (for production or development) of a model designed with its integration process.

Introduction to Deployment

Deployment consist in creating or updating in a Data Location (a database schema accessed via a JDBC datasource) a Model Edition. Once this model edition is deployed, it is possible to create a Data Edition (version of the data) using this model edition (version of the metadata).

In this process, the following components are involved:

In the deployment process:

Data Location Types

There are two types of data location. This type is selected when the data location is created and cannot be changed afterwards:

The data location types are:

Important: Be cautious when choosing the data location type, at is will determine the type of deployment operations that can be done. It is recommended to use only Production Data locations for Production and User Acceptance Test environments.

Data Location Structures

A Data Edition is based on one of the Model Editions deployed in the location. Many data editions may coexist in the same data location.

The various data editions are stored in the same data structures. The data structures (tables) in a given Data Location are compatible with any of the Model Editions deployed in this data location. When a new model edition is installed, the structures are modified to support this new model edition in addition to the ones previously deployed.

In addition, data duplication is avoided as much as possible. For example: If a Golden Record exists but remains the same for 5 editions, it is only stored once golden data table, and not replicated five times.

Creating a Data Location

A Data Location is a database schema into which several Model Editions will be deployed and Data Editions will be created.

Note: A data location is a connection to a database schema via a JDBC datasource defined in the application server running Semarchy Convergence for MDM. Make sure to have the administrator of this application server create this datasource, and to have the database administrator create the schema for you before creating the data location in Semarchy Workbench.

To create a new data location:

  1. In the menu, select File > New > New Data Location. The Create New Data Location wizard opens.
  2. In the Create New Data Location wizard, check the Auto Fill option and then enter the following values:
  3. Click Finish to close the wizard. The Data Locations perspective opens and the Data Editions view displays the new data location.

Deploying a Model Edition

There are two ways of deploying a model edition:

Installing a Model Edition

To install a model edition:

  1. In the Data Editions view Right-click the data location node and select Install Model Edition.... The Install Model Edition wizard opens.
  2. In the Install Model Edition wizard, enter the following values:
  3. In the Description field, optionally enter a description for the Model Edition.
  4. Click Next. The changes to perform in the data location to support this new model edition are computed. A second page shows the SQL script to run on the schema to deploy this model edition. This script can be executed by Semarchy Convergence for MDM or you can download, customize and execute it separately.

In the future, when you perform changes to this model’s structure, you will be able to create new editions of this model (For example [0.2], [0.3], ...) and will be able to deploy these new editions in the same data location. These new deployments will generate patching scripts to bring the physical database objects to the level of the new model edition.

Note: Deploying a model edition does not modify data in the data editions already in place in the data location.

Updating a Model Edition

To update a model edition:

  1. In the Data Editions view, expand the data location node and the Deployed Model Edition node.
  2. Right-click the deployed model edition that you want to update and select Update Deployed Model Edition (Design-Time). The Update Deployed Model Edition wizard opens.
  3. In the Update Deployed Model Edition wizard, select the items that you want to update: Integration Job and/or Database schema.
  4. Click Next. The list of DDL instructions needed to update the data structures is displayed.
      1. Click Finish to run the script and close the wizard.

Warning: Updating the job definitions erases all the logs attached to the previous job definitions.

Note: Although it is recommended to update both the jobs and schemas at the same time, you may want to update the data structure first then the jobs later. For example, if the data that you have in the data editions using this model edition is not fit for the new version of the jobs. In that case, you may want to run some transformation on the data with the updated data structures before updating the jobs.
Another use case for not deploying the job definition is when you know that the new and old job definition are similar and you want to preserve the existing job logs.

Creating a Data Edition

Creating a Root Data Edition

To create a data edition:

  1. In the Data Editions view, select the data location node, right-click and select Create Root Data Edition. The Create New Root Data Edition wizard opens.
  2. In the Create New Root Data Edition wizard, check the Auto Fill option and then enter the following values:
  3. In the Description field, optionally enter a description for the Root Data Branch.
  4. Click Next.
  5. In the Data Edition screen, select the Deployed Model Edition that will support the data edition.
  6. In the Description field, optionally enter a description for the Root Data Edition.
  7. Click Finish to close the wizard. The RootBranch data branch is created and a first data edition is created under this root branch.

This data edition is open. It means that we can perform loads and changes to this edition.
Later on, you will be able to close this edition and open a new one. Closing an edition freezes its content, creating a snapshot of the MDM Hub content.

Closing and Creating a New Data Edition

To close and create a new data edition:

  1. In the Data Editions view, expand the data location node.
  2. Expand the Data Branches > RootBranch node.
  3. Right-click the latest data edition (open) and select Close and Create New Edition. The Close and Create New Data Edition wizard opens.
  4. In the Close and Create New Data Edition wizard, enter a Description for the new data edition.
  5. Select the Deployed Model Edition that will support the data edition.
  6. Click Finish to close the wizard. The new data edition under the RootBranch data branch is created. The previous data edition is closed.

Note: It is possible to close and create a new edition while data entry or duplicate management workflows are in progress in the data edition. Long running workflows can execute across several data editions. A warning is issued workflows are in progress.

Warning: When closing and opening a new data edition with a different model edition, there is no check performed on the new model to guarantee that the ongoing workflows are supported with the new model. The administrator should check that all ongoing workflows will be able to continue to completion with the new model edition.

Note: If integration jobs are currently running for this data edition, you may be prompted to move it to a Maintenance mode. This status allows current jobs to complete while rejecting new submitted jobs.

Note: You can automate this operation in production environements using the Administrative Web Service. See the Web Services chapter in the Semarchy Convergence for MDM Integration Guide for more information.

Advanced Deployment Techniques

Switching Model Edition

This operation allows you to change the deployed model edition supporting a given data edition without closing the data edition.
This prevent you from creating data snapshots for model changes.

To perform the switch, you must first move the data edition to Maintenance mode.

To swich a data edition to a different model edition:

  1. In the Data Editions view, expand the data location node.
  2. Expand the Data Branches > RootBranch node.
  3. Right-click the latest data edition (open) and select Switch Model Edition. The Switch Model Edition wizard opens.
  4. Select the Deployed Model Edition that will support the data edition from now on.
  5. Check the option if you want to automatically re-open the data edition after the switch.
  6. Click Finish to close the wizard. The data edition is now switched to the new model edition.

If you have not selected the option to automatically re-open the data edition, you must now move it back to Open.

Moving Models at Design-Time

At design-time, it is possible to move models from one repository to another design repository using Export/Import:

Exporting a Model Edition

To export a model edition:

  1. Open the Model Editions perspective.
  2. Expand the model branch containing the model edition that you want to export.
  3. Select the model edition that you want to export, right click and select Export Model Edition.
  4. In the Model Edition Export dialog, select an Encoding for the export file.
  5. Click the Download link to download the export file on your local file system.
  6. Click Close.

Importing to a New Model

To import and create a new model:

  1. In the menu, select File > New > New Model from import... The Import to a New Model wizard opens.
  2. Click the Open button and select the export file.
  3. Click Finish to perform the import.

Click the Refresh button in the Model Editions view. The new model appears in the list.

Importing on an Existing Model

To import and replace an existing model:

  1. Open the Model Editions perspective.
  2. Expand the model branch containing the model edition that you want to replace.
  3. Select the open model edition that you want to replace, right click and select Import Replace.... The Import-Replace Model Edition wizard opens.
  4. Click the Open button and select the export file.
  5. Click Finish to perform the import.
  6. Click OK to confirm the deletion of the existing model.

The existing model edition is replaced by the content of the export file.

Deploying to a Remote Location

Frequently, the deployment environment is separated from the development environment. For example, the development/QA and production sites are located on different networks or locations. In such cases, it is necessary to use export and import to transfer the model edition before performing the deployment in production.

A remote deployment consists in moving a closed model edition:

Remote Deployment Architecture

In this configuration, two repositories are created instead of one:

The process for deploying a model edition in this configuration is given below:

  1. The model edition is closed in the design repository.
  2. The closed model edition is exported from the design repository to an export file.
  3. The closed model edition is imported from the export file into the deployment repository.
  4. The closed model edition is deployed from the deployment repository into a production data location.

Exporting a Model Edition

To export a model edition:

  1. Open the Model Editions perspective.
  2. Expand the model branch containing the model edition that you want to export.
  3. Select the closed model edition that you want to export, right click and select Export Model Edition.
  4. In the Model Edition Export dialog, select an Encoding for the export file.
  5. Click the Download link to download the export file on your local file system.
  6. Click Close.

Importing a Model Edition in a Deployment Repository

To import a model edition in a deployment repository:

  1. Open the Model Editions perspective.
  2. Select File > New > Import Model Edition. The Import Model Editions wizard opens.
  3. Click the Open button and select the export file.
  4. Click Next.
  5. Review the content of the Import Preview page and then click Finish.

Note: When importing a model edition, the root model and the branches containing this model edition are automatically created as needed.

Warning: When importing successive versions of model editions in a deployment repository, it is not recommended to skip intermediate versions, as it is not possible later to import these intermediate versions. For example, if importing version 0.1 of a model, then importing version 0.4, the intermediate versions – 0.2, 0.3 – can longer be imported into this repository.

Data Edition Statuses

Understanding Data Edition Statuses

This section provided detailed information about the processes involved when closing a data edition.

A data edition can be in three statuses:

When moving a data edition to a closed status, no batch must be in progress. You must first wait for batches to complete before moving the data edition to a closed state. Loads submitted after the data edition is closed will fail as the closed data edition is read only.

When moving a data edition to a Maintenance status, the currently processed batches continue until completion. Loads submitted after the data edition is moved to Maintenance will fail. They can be kept open and submitted later, when the data edition is restored to an open status.

Moving a Data Edition to Maintenance Mode

To set a data edition status to maintenance mode:

  1. In the Data Editions view, expand the data location node.
  2. Expand the Data Branches > RootBranch node.
  3. Right-click the latest data edition (open) and select Set Data Edition Status to Maintenance.
  4. Click OK in the confirm dialog.

The data edition is moved to Maintenance mode.

To set a data edition status to open mode:

  1. In the Data Editions view, expand the data location node.
  2. Expand the Data Branches > RootBranch node.
  3. Right-click the latest data edition (open) and select Set Data Edition Status Back to Open.

The data edition is moved to an open state.

Using the Maintenance Mode

The Maintenance status can be used to perform maintenance tasks on the data edition.
For example, if you want to close the data edition and open a new data edition on a new model edition with data structure changes that mandate manual DML commands to be issued on the hub tables, you may perform the following sequence:

  1. Move the data edition to Maintenance mode.
  2. Let the currently running batches complete. No batch can be submitted to this edition.
  3. Close and open a new data edition, using the updated model edition.
  4. Move the new data open edition to a Maintenance mode.
  5. Perform your DML commands.
  6. Move the data edition from Maintenance to the Open status. Batches can now be submitted to this edition.

Using this sequence, you prevent batches be submitted while the hub is in Maintenance.

Maintenance status is also mandatory when switching a data edition to a different deployed model edition.