Welcome to Semarchy xDM.
This guide contains information about developing new plug-ins for Semarchy xDM.

Preface

Audience

This document is intended for developers who want to extend the capabilities of the Semarchy xDM platform using the Open Plug-in Architecture.
The Open Plug-in Architecture is based on the Java language. Plug-in developers should be comfortable with this language and with the Eclipse IDE.

If you want to learn about MDM or discover Semarchy xDM, you can watch our tutorials.
The Semarchy xDM Documentation Library, including the development, administration and installation guides is available online.

Document Conventions

This document uses the following formatting conventions:

Convention Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or a product specific term or concept.

italic

Italic type indicates special emphasis or placeholder variable that you need to provide.

monospace

Monospace type indicates code example, text or commands that you enter.

Other Semarchy Resources

In addition to the product manuals, Semarchy provides other resources available on its web site: http://www.semarchy.com.

Obtaining Help

There are many ways to access the Semarchy Technical Support. You can call or email our global Technical Support Center (support@semarchy.com). For more information, see http://www.semarchy.com.

Feedback

We welcome your comments and suggestions on the quality and usefulness of this documentation.
If you find any error or have any suggestion for improvement, please mail support@semarchy.com and indicate the title of the documentation along with the chapter, section, and page number, if available. Please let us know if you want a reply.

Overview

This guide contains information about developing new plug-ins for Semarchy xDM.

Using this guide, you will learn how to:

  • Set up a development environment with the Semarchy xDM Open Plug-in API,

  • Design your own plug-ins for Semarchy xDM

  • Implement tests for these plug-ins

  • Release these plug-ins to production

Introduction to the Semarchy xDM Plug-in Architecture

About the Plug-in Architecture

With the Open Plug-in Architecture, Semarchy xDM allows extending its capabilities using Java code and external APIs. Through the plug-ins, existing services or information systems can contribute to the master data processing and enrichment.

You can extend the following capabilities in Semarchy xDM:

  • Enrichers: By adding new enrichers, you can perform record-level enrichment to update, augment or standardize existing attribute values, or create new content in attributes. For example, you can connect to an external web service to retrieve stock ticker symbols from company names.

  • Validations: By adding new validations, you can check records against complex data quality rules. For example, you can connect to an external provider to check whether a billing or shipping address is accurate or not.

Understanding the Plug-ins

Plug-in Development

Plug-ins are developed in Java using the Semarchy xDM Open Plug-in API. This API exposes several Java interfaces that your code must implement to interact with Semarchy xDM.

The plug-ins are developed in plug-in projects into which you defined extension endpoints. Each endpoint appears as an enricher or validation plug-in available in Semarchy xDM. These endpoints are based on two predefined endpoints types:

  • rowTransformer for an Enricher.

  • rowValidator for a Validation.

Each extension endpoint has:

  • A set of Parameters to optionally configure the plug-in behavior.

  • One or more Input fields. In the case of an enricher, these fields are pushed to the enricher for transformation. In the case of a validation, these fields are checked against the data quality rules implemented in the plug-in.

  • One or more Output fields (only for enrichers). These are the values returned by the enricher. In the case of a validation, there is no output field, as the validation only returns a Boolean indicating whether the validation is passed or failed.

To design a plug-in, you define first the plug-in metadata, then you implement the code, based on this metadata. The metadata is used in the Semarchy Workbench to expose the plug-in information, parameters and input/output fields.

Plug-in Packaging and Deployment

Plug-ins are packaged as bundles that can be installed or updated from the Workbench in a running instance of Semarchy xDM. The server does not need to restart to take new or updated bundles into account.

Bundles are tagged with a version number. Installing a new version of an existing plug-in will automatically make the platform take into account and work with the new version of the plug-in.

Process for Creating a Plug-in

Run the following steps create a plug-in:

  1. Create a plug-in project to host one or more plug-ins.

  2. Define the plug-in metadata.

  3. Implement the plug-in code.

  4. Test the plug-in by implementing a test project and JUnit test cases.

  5. Generate the plug-in binaries.

  6. Deploy the plug-in in Semarchy xDM.

About the Example

In this book, we will design a sample enricher plug-in called International Phone Standardizer.
This plug-in:

  • Takes a phone number in an input field called INPUTPHONE.

  • Standardizes it as an output phone number called STANDARDIZEDPHONE.

  • Has a NULLIFYONERROR parameter. If this parameter is set to 1, phone numbers that cannot be standardized are transformed to a null value. Otherwise, the original phone number is returned.

To keep the code simple, the standardization algorithm provided in this sample plug-in only works for phone numbers from France. It standardizes them to the +33<9 digits> format. It is intended to work with the data provided in the Getting Started demonstration environment of Semarchy xDM, and not in production environments.

Setting up the Environment

The Semarchy xDM Open Plug-in API is available for download as an update site for Eclipse in the Semarchy xDM SDK archive (semarchy-xdm-sdk-<version>.zip). In this section, we will explain how to configure your Eclipse environment with this API.

Installing and Configuring Eclipse

Before installing the Semarchy xDM Open Plug-in API, download and install Eclipse for RCP and RAP Developers (Indigo or later). Eclipse is available for download from the Eclipse Website.

Installing the Semarchy xDM Open Plug-in API for Eclipse

To install the Semarchy xDM Open Plug-in API for Eclipse:

  1. Download the latest Semarchy xDM SDK archive (semarchy-xdm-sdk-<version>.zip) from the Semarchy web site.

  2. Uncompress this file into a local /temp directory.

  3. In Eclipse, choose Help > Install New Software….

  4. In the Work with section, click the Add… button. The Add Repository dialog box appears.

  5. Click Archive, browse for the /temp/plugins-sdk/semarchy-xdm-plugins-sdk-updatesite.zip and then click OK. The file location appears in the Location field. Leave the Name field empty.

  6. Click OK.

  7. Select Semarchy xDM in the available software components. The Semarchy SDK component should be selected too.

  8. Un-select the Contact all update sites during install to find required software option.

  9. Click Next to review the list of items to install. Click Next again to read and accept the terms of the license agreement and then click Finish. The installation process starts.

  10. During the installation process, Eclipse prompts you to trust the certificates signing the plug-in SDK. Select the certificates and then click OK to proceed with the installation.

  11. When asked, restart Eclipse.

The Semarchy xDM Open Plug-in API is now installed.

Creating the Plug-in

Creating the Plug-in Project

A Semarchy xDM plug-in is developed as part of a Plug-in Project, into which you declare extension points for each plug-in contained in this project.

Creating a New Plug-in Project

A plug-in project contains one or more Semarchy xDM plug-ins and can be deployed as a single jar.

To create a new plug-in project:

  1. In Eclipse, select File > New > Others….

  2. In the Wizards filter, enter Plug-in. The list shows the Plug-in Project wizard.

  3. Select the Plug-in Project wizard and then click Next

  4. In the Project Name, enter the name of your project. For example: com.acme.phoneStandardizer. Leave the other fields as is.

  5. Click Next. In the Content page, change the following fields:

    • Version: version of your plug-in (for example: 1.0.0)

    • Name: Visible Name for your plug-in (for Example: Phone Standardizer)

    • Provider: Your company’s name (for Example, ACME Corp.)

    • Un-select the Generate an activator, This Plug-in will make contributions to the UI and Enable API Analysis options.

    • Make sure the Would you like to create a rich client application? option is set to No.

  6. Click Next

  7. Un-select the Create a plug-in using one of the templates option.

  8. Click Finish

  9. If asked, click Yes to switch to the development perspective.

Adding the Extensions to the Project

In order to define Semarchy xDM extension points, you need to add these extensions points’ types to the project.

To add the Semarchy xDM extension points to the project:

  1. In the plug-in editor, select the Dependencies tab

  2. In the Required Plug-ins, click Add… and then select the com.semarchy.engine.extensionpoint.
    The extension point is added to the dependencies.

  3. The extension point is added with a minimum version required equal to the version of the plug-in SDK. For example, if you have installed the SDK in version 3.1.0, the minimum server version required is 3.1.0.
    To make your plug-in compatible with previous minor versions of the server (for example, to install the plug-in on a 3.0.0 server), you must modify this dependency.

    1. Select the extension point in the list and then click the Properties… button.

    2. In the com.semarchy.engine.extensionpoint dialog, edit the Minimum Version field. For example, replace 3.0.1 with 3.0.0 to make this plug-in dependent on any server version after 3.0.0.

    3. Click OK to close the dialog.

  4. Select the Overview tab.

  5. In the Overview tab, click the Extensions hyperlink. The editor switches to the Extensions tab.

  6. In the Extensions tab, click Add…, and select the extension point corresponding to the type of plug-in you want to deliver in this project:

    • com.semarchy.integration.extensionpoint.rowTransformer: This extension point type is used to create enrichers.

    • com.semarchy.integration.extensionpoint.rowValidator: This extension point type is used to create validations.

  7. Repeat the previous step to add both types of extensions if you want to mix enrichers and validations in your project.

Defining the Plug-in

A project can contain several Semarchy xDM plug-ins (for example, a plug-in to enrich telephone numbers, and another one to validate email addresses). Each plug-in is declared as an extension endpoint of the rowTransformer or rowValidator endpoint type.

An extension endpoint is declared with all the plug-in metadata: The name, version, provider, as well as the parameters, input and output fields definition.

Declaring an Extension EndPoint

For the rest of this book, we will focus on creating an Enricher (Row Transformer). Validation plug-in follow the same development path.

To create an extension endpoint:

  1. Right-click the com.semarchy.integration.extensions.rowTransformer extension you just added, right-click and select New > Row Transformer.

  2. In the Extension Element Details section, enter the following values:

    • Id: ID of the enricher/validation. For example: com.acme.phoneStandardizer.intlPhoneStandardizer. Note this ID as it will be used later in the test cases.

    • Name: Text name of the enricher or validation. For example: International Phone Standardizer

    • Provider Name: Your company’s name (for Example, ACME Corp.)

    • isThreadSafe: Set this option to true if you know that your code is thread-safe. Otherwise, set it to false. See the Parallel Execution and Thread Safety for more information about this option.

  3. Click the Class link.

  4. In the New Java Class wizard, enter the following:

    • Package: Name of the package containing the class for your enricher/validation. For example: com.acme.phonestandardizer

    • Name: Name of the class for your enricher/validation. For example: IntlPhoneStandardizer

  5. Click Finish. The code editor for your class opens.

Parallel Execution and Thread Safety

Semarchy xDM supports parallel processing of records in plug-ins, provided that the plug-in is designed and declared as thread-safe. By setting the isThreadSafe option to true in a rowTransformer or rowValidator extension point, you declare that the code of the plug-in is thread safe and can be executed with several parallel threads.

It is the plug-in developer responsibility to review and test the plug-in to make sure of its thread safety before setting the isThreadSafe option to true.

Declaring the Parameters and Input/Output Fields

Parameters and Input/Output Types

Plug-ins use declared types for the parameters, inputs and outputs. When setting the values for the parameters and inputs, and when mapping the outputs, appropriate typing allows design-time validation and prevent run-time issues.

In the plug-in code, when managing inputs and parameters, each declared type will correspond to a given java type. Make sure to define and cast your variable according to the type of the input/output or parameter that they will map to.

The table below provide the correspondance between the plug-in types and Java types.

Plug-in Type Corresponding Java Type

BOOLEAN

java.lang.Boolean

BYTE_INTEGER

java.lang.Byte

DATETIME

java.util.Date

TIMESTAMP

java.util.Date

DOUBLE

java.lang.Double

FLOAT

java.lang.Float

INTEGER

java.lang.Integer

LONG_INTEGER

java.lang.Long

SHORT_INTEGER

java.lang.Short

Declaring the Parameters

Parameters allow customizing the behavior of the plug-in.

To declare the plug-in parameters:

  1. In the plug-in editor, select the Extensions tab

  2. Select the parameters node under your Extension Point, right-click and select New > param.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the parameter. For example: NULLIFYONERROR

    • Type: Type of the parameter. For example: BOOLEAN

    • Label: User-facing label for this parameter. For example: Nullify on Error

    • shortDescription: User-facing description for this parameter. For example: If set to 'true', returns null for a phone number that cannot be processed. Otherwise returns the original phone number.

    • isMandatory: Set to true to make this parameter mandatory.

  4. Repeat the two previous steps to add more parameters.

Declaring the Input Fields

Input fields are the values received by an enricher or validation plug-in.

To declare the plug-in input fields:

  1. In the plug-in editor, select the Extensions tab

  2. Select the inputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the field. For example: INPUTPHONE. This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: Type of the field. For example: STRING

    • Label: User-facing label for this field. For example: Input Phone

    • shortDescription: User-facing description for this field. For example: Phone Number to be standardized.

    • isMandatory: Set to true to make this field is mandatory.

  4. Repeat the two previous steps to add more input fields.

Declaring the Output Fields

Output fields are the values returned by an enricher plug-in.

To declare the plug-in output fields:

  1. In the plug-in editor, select the Extensions tab

  2. Select the outputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the field. For example: STANDARDIZEDPHONE. This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: Type of the field. For example: STRING

    • Label: User-facing label for this field. For example: Standardized Phone

    • shortDescription: User-facing description for this field. For example: Standardized Phone Number.

    • isMandatory: Set to true to make this field mandatory (a value is always returned).

  4. Repeat the two previous steps to add more output fields.

Make sure that the input and output fields names only contain uppercased alphanumerics and are no longer than 30 characters.
Make sure to use uppercase for the parameter and field names. These names are not visible to the end-users.

Implementing the Plug-in Code

Now that the plug-in is declared, it is possible to implement the specified behavior.

To implement the plug-in code:

  1. Switch to the class editor.

  2. If needed, Use the Eclipse quick fix feature to add the interface methods currently unimplemented in the code.

  3. Implement the required methods for your plug-in:

    • For a rowTransformer: the setUp, tearDown and transform Methods.

    • For a rowValidator: the setUp, tearDown and isValid Methods.

The full code of the IntlPhoneStandardizer.java class rowTransformer is provided in Appendix A. You can copy and paste the code from the appendix in the class editor.

The following sections describe the methods implemented in the plug-in code.

SetUp/TearDown Methods

These methods are respectively used to initialize the plug-in - typically by setting the value of the parameters - and to release the resources used by the plug-in.

In this case, the only parameter is NULLIFYONERROR. This parameter’s value is stored in a private Boolean variable called nullifyOnError.

Note the use of the getParameterAsBoolean method to retreive a boolean corresponding to the type set while Declaring the Parameters and Input/Output Fields. The same methods exist for other supported data types.

private boolean nullifyOnError;
...
@Override
public void setUp(IRowTransformerSetupInfo pSetupInfo) {
        nullifyOnError = pSetupInfo.getParameterAsBoolean("NULLIFYONERROR");
}

@Override
public void tearDown() {

}
String referring to parameter names should always be in uppercase in the Java code.
The IRowTransformerSetupInfo object can be used to retreive the data location’s JDBC datasource, using the getTargetDataSource() method.

Transform Method (Enricher Plug-ins)

Enrichers plug-ins enrich a list of data rows by calling transform method. This method returns the list of enriched data rows.
In our example, this method simply calls a private method called transformOneRow to process one of these data rows.

@Override
public List<IDataRow> transform(List<IDataRow> pDataRowsBatch) {
        ArrayList<IDataRow> outputDataRowList = new ArrayList<IDataRow>();
        for (IDataRow inputRow : pDataRowsBatch) {
                outputDataRowList.add(transformOneRow(inputRow));
        }
        return outputDataRowList;
}

The transformOneRow method transforms an input data row (IDataRow) into an output data row.

  • The input data row is passed to the transform method in the pDataRow variable, and contains the input fields passed to the enricher. You access each field value using the getValue method.

  • The output data row is created and returned by this method, with the output fields set using the setValue method.

In the example below, the input data row only contains the INPUTPHONE field and the output data row is built only with the STANDARDIZEDPHONE field.
The core of the transformation takes place in the private method normalizePhoneNumber. The code of this method can be reviewed in Appendix A.

private IDataRow transformOneRow(IDataRow pDataRow) {
        // First, create the returned IDataRow.
        DataRow outputDataRow = new DataRow();

        // Make sure to set a null value for each output field in the
        // outputDataRow.
        outputDataRow.setValue("STANDARDIZEDPHONE", null);

        // The transformation is done below. It uses the normalizePhoneNumber
        // method defined in the class.
        if (pDataRow.getValue("INPUTPHONE") != null) {
                outputDataRow.setValue("STANDARDIZEDPHONE",
                                normalizePhoneNumber(pDataRow.getValue("INPUTPHONE")
                                                .toString()));
        }
        return outputDataRow;
}
It is recommended to set all the mandatory output field values to null immediately after creating the output DataRow. This avoids returning partially complete data row as the output. This is done by issuing the following code for each output field outputDataRow.setValue("<output_field_name>", null);.
String referring to input/output field names should always be in uppercase in the Java code.

isValid Method (Validation Plug-ins)

The isValid method is required for a validation plug-in.
This method is similar to the transform method. It takes as an input a list of data rows and returns a list of boolean values indicating whether the input data rows comply with the validation specified in the plug-in. Typically, this method uses an isValidOneRow private method that validates a single data row.

Testing the Plug-in

It is recommended to implement tests to validate the plug-in behavior before deploying it to a Semarchy xDM instance.
In this section, we will explain how to create a JUnit test case for the enricher plug-in created in the previous chapter.

Creating a Test Project

To create a test project:

  1. Select File > New > Other

  2. In the Wizards filter, enter fragment. The list shows the Fragment Project wizard.

  3. Select the Fragment Project wizard and then click Next.

  4. In the Project Name, enter the name for your project. For example: com.acme.phoneStandardizerTests. Leave the other fields as is.

  5. Click Next. In the Content page, modify the following:

    • Version: Version of your test plug-in fragment (for example: 1.0.0)

    • Name: Visible Name for your test plug-in fragment (for Example: Phone Standardizer Test)

    • Provider: Your company’s name (for Example, ACME Corp.)

  6. In the Host Plug-in section, use the Browse… button to select the Plug-in ID.

  7. In the Select a Plug-in filter, enter the name of the Semarchy xDM plug-in you want to validate with your tests. For example com.acme.phoneStandardizer.

  8. Select the Semarchy xDM plug-in from the list. For example, select com.acme.phoneStandardizer

  9. Click Finish. A new project is created.

  10. In the project editor, select the Dependencies Tab.

  11. In the Required Plug-ins section, click the Add… button.

  12. In the Select a Plug-in filter, enter org.junit4. Select the org.junit4 plug-in in the list of matching items and then click OK.

  13. Press CTRL+S to save the editor.

Creating a Test Case

To create a test case:

  1. In the Package Explorer, select the test project (com.acme.phoneStandardizerTests), right click and then select File > New > Other.

  2. In the Wizards filter, enter junit. The list shows the JUnit Test Case wizard. Select it from the list and then click Next.

  3. In the JUnit Test Case wizard, enter the following:

    • Package: Name of the package containing the class for your test. For example: com.acme.phonestandardizer.tests

    • Name: Name of the test class. For example: TestPhoneFrance.

    • Class under test: Use the browse button to select the class you want to test with this test case. For example: com.acme.phonestandardizer.IntlPhoneStandardizer

  4. Click Next. Select the methods you want to test. Typically, the transform method for an enricher or the isValid method for a validation.

  5. Click Finish. The code editor for your test case opens.

In addition to the transform and isValid methods, you may also test other methods such as setUp and tearDown to verify the plug-in behavior during its entire lifecycle.

Implementing the Test Case

The test case checks the transformation of several phone numbers. It checks that the transformation provides expected results with valid, invalid or null phone numbers.

The full code of the TestPhoneFrance.java class is provided in Appendix B. You can copy and paste the code from the appendix in the class editor.

The rest of this section describes the key methods implemented in the test code.

The checkPhoneTransform method tests the behavior of the transformer. It takes three parameters:

  • The NULLIFYONERROR parameter

  • an INPUTPHONE input phone number

  • an expected STANDARDIZEDPHONE output phone number

The method:

  1. Defines the mapped inputs, outputs and parameters for the test case.

  2. Instantiates an rowTransformerExecutor object. This object enables testing a row transformer.
    Note that this object is created using:

    • The list of mapped inputs and outputs.

    • The ID of the enricher/validation set when declaring the endpoint, that is com.acme.phoneStandardizer.intlPhoneStandardizer.

  3. Sets up the plug-in using the mapped parameters.

  4. Creates a list containing one input DataRow containing an INPUTPHONE input field value.

  5. Runs the transform method for the rowTransformerExecutor to execute the enricher.

  6. Compares (using the JUnit assertEquals method) the value of the STANDARDIZEDPHONE field of the output IDataRow with the expected standardized phone.

  7. Finally, uses the tearDown method on the rowTransformerExecutor.

private void checkPhoneTransform(Boolean nullifyOnError, String inputPhone, String expectedOutputPhone) {

        // Define the mapped inputs.
        Set<String> mappedInputs = new HashSet<String>();
        mappedInputs.add("INPUTPHONE");

        // Define the mapped outputs.
        Set<String> mappedOuputs = new HashSet<String>();
        mappedOuputs.add("STANDARDIZEDPHONE");

        // Define the mapped parameters
        Map<String,Object> parameters = new HashMap<String, Object>();
        parameters.put("NULLIFYONERROR", nullifyOnError);

        // Create the rowTransformerExecutor
        RowTransformerExecutor rowTransformerExecutor = new RowTransformerExecutor("com.acme.phoneStandardizer.intlPhoneStandardizer",null,mappedInputs,mappedOuputs);

        // Setting up the plug-in.
        rowTransformerExecutor.setUp(parameters);

        try{

                // Creating a new data row with the test phone number
                IDataRow inDataRow = new DataRow();
                ((DataRow) inDataRow).setValue("INPUTPHONE", inputPhone);

                // Inserting this data row in a list of data rows.
                List<IDataRow> inDataRowList = new ArrayList<IDataRow>();
                inDataRowList.add(inDataRow);

                // Enriching the list of input data rows.
                List<IDataRow> outDataRowList = rowTransformerExecutor.transform(inDataRowList);

                // Testing the resulting phone number.
                assertEquals(expectedOutputPhone, outDataRowList.get(0).getValue("STANDARDIZEDPHONE"));

        } finally {

                // Tearing down the plug-in.
                rowTransformerExecutor.tearDown();
        }
}

This method is used in various test cases, as shown below.

@Test
public void testTransformNullPhones() {
        checkPhoneTransform(true, null, null);
        checkPhoneTransform(false, null, null);
}

@Test
public void testTransformBadPhones() {
        checkPhoneTransform(true, "abcd", null);
        checkPhoneTransform(false, "abcd", "abcd");
        checkPhoneTransform(true, "64169710", null);
        checkPhoneTransform(false, "64169710", "64169710");
        checkPhoneTransform(true, "111164169710", null);
        checkPhoneTransform(false, "111164169710", "111164169710");
        checkPhoneTransform(true, "+44664169710", null);
        checkPhoneTransform(false, "+44664169710", "+44664169710");
        checkPhoneTransform(true, "1664169710", null);
        checkPhoneTransform(false, "1664169710", "1664169710");
}
The unit tests only include French phone numbers, as it is what the transformer supports. Valid French Phone numbers have 10 digits, starting with a zero. They may be written with non-numeric characters that will be ignored. They may be prefixed with the +33 country code.

Running the Test Case

To run the test case:

  1. Save the test case code.

  2. Select the test project, right-click and select Run As > JUnit Plug-in Test. The JUnit view opens and show the execution results for the test case.

The test cases should display no error.

Releasing the Plug-in

Generating the Plug-in Binary

The plug-in binary is a Jar file that will be deployed in a Semarchy xDM instance. Once deployed, the plug-in becomes available for design and run-time.

Generate the Plug-in Binary

When the plug-in is tested, or when a new version is available, you can generate the plug-in binary.

To generate the Plug-in Binary:

  1. Select the plug-in project, right-click and select Export.

  2. Select Plug-in Development > Deployable plug-ins and fragments.

  3. In the Destination tab, use the Browse button to select a target directory for your plug-in. For example: c:\temp.

  4. Select the Plug-in to deploy. For example: com.acme.phoneStandardizer.

  5. Click Finish.

The jar for the plug-in is created in a plugins folder under the selected destination directory. The generated jar file is named after the plug-in ID and version number.

When selecting the plug-ins and fragments to deploy, make sure to select only the plug-in, and not the JUnit test case fragments as they are not needed for running the plug-in.

Incrementing the Plug-in Version

If the plug-in was modified or fixed, before generating the new version of the plug-in binary, increment the plug-in version

To increment the Plug-in version:

  1. In the plug-in project, double click the plugin.xml file to edit it.

  2. In the overview tab, increment the Version number. For example, if the original plug-in version is 1.0.0, then the first patch for this plug-in would be 1.0.1.

  3. Press CTRL+S to save the editor.

As the generated jar file is named after the plug-in ID and its version number, you can preserve in this directory the successive versions of the plug-in.

Deploying the Plug-in

To deploy a plug-in in a Semarchy xDM instance, you must connect with a user that has the Plug-ins Administration Read/Write privileges, or with an administrator user (semarchyAdmin role).

To deploy the plug-in binary:

  1. Connect to the Semarchy Workbench.

  2. Open the Administration Console perspective.

  3. Double-click the Plug-ins node in the Administration view.

  4. Click the Install or Update Plug-in button in the upper right corner of the Plug-ins editor. The Install/Update Plug-ins dialog opens.

  5. Click the Browse button and select the plug-in binary file. For example: com.acme.phoneStandardizer_1.0.0.jar.

  6. Click OK. A Status window shows the number of plug-ins installed or updated.

The plug-in now appears in the list, and can be used in the models and the integration jobs.

Using Your Plug-ins

Using Plug-ins

Plug-ins designed with the Plug-in Architecture can be used as regular plug-ins.
See the Certification Process Design chapter Semarchy xDM Developer’s Guide for more information.

Using the Plug-in Example in the Getting Started Environment

You can use the sample plug-in example created with this guide in the demonstration environment provided for the Semarchy xDM Getting Started Guide, or in your own projects.

Before using your plug-in in the demonstration environment, it is recommended that you first get comfortable with the Workbench and the demonstration environment by running the entire Getting Started.

Setting up the Getting Started Environment

To set up a Getting Started environment for testing the plug-in:

  1. Follow the instructions in the Semarchy xDM Getting Started Guide until the Create the Demo Model section. In this section, select Full Setup instead of selecting Partial Setup. This will create the entire model.

  2. Deploy the Plug-in to the platform as explained in the Deploying the Plug-in section.

Using the Enricher in the Model.

In this example, we will use the International Phone Standardizer enricher to standardize the phone numbers from the Phone1 attribute in the Contact entity. Note that we set the Nullify On Error parameter to false in order to preserve the original phone numbers that cannot be standardized.

To use the enricher in the model:

  1. Go to the Overview perspective.

  2. Open the CustomerAndFinancialMDM [0.0] model edition.

  3. Expand the Entities > Contact > Enrichers node.

  4. Right-click the Enrichers node and then select Add Plug-in Enricher.

  5. In the Create New Plug-in Enricher wizard, enter StandardizePhone for the Name field.

  6. In the Plug-in ID, select International Phone Standardizer.

  7. Click Finish. The Plug-in Enricher editor opens.

  8. In the Plug-in Params section, click the Value cell on the Nullify on Error line, and enter in the cell false.

  9. In the Plug-in Inputs section, click the Value cell on the InputPhone line, and enter in the cell Phone1.

  10. In the Plug-in Outputs section, click the Define Plug-in Output button in the upper-right corner. The Define Output Bindings dialog appears. Double-click the Phone1 attribute and then click Finish.

  11. In the Plug-in Inputs section, select the Output Name cell on the Phone1 line, and select Standardized Phone in the selection list.

  12. Press CTRL-S to save the editor.

Deploying the Updated Model

To deploy the updated model:

  1. Go back to the Overview perspective, and select the data location into which the model is currently deployed.

  2. right-click and select Deploy Model Edition.

  3. Click Finish in the Deploy Model Edition wizard.

In a development environment only, it is possible to update existing deployment without creating a version of the model edition. In addition, only the certification processes need to be updated, as the data structure is not modified by the addition of the enricher.

Running the Enricher

To run the enricher:

  1. In the menu, select Help > Demo Environment > Open Demo Application….

  2. Click the Reset Sample Data button to restore the sample data to its original state.

  3. Click the Publish Data to MDM… button in the editor toolbar.

  4. Click Finish to start the loading process.

  5. A Data Submit Succeeded dialog appears. Click OK to close it.

  6. View the log and expand the Contact task. You will now see your StandardizePhone enricher task and the number of records processed.

  7. Review the Contact golden records. Certain values of Phone1 have been standardized and appear with a +33 prefix.

Troubleshooting Plug-ins

Troubleshoot Deployment

When you deploy a plug-in, it appears in the list of plug-ins with an Available status and a green icon.
If your plug-in appears with a warning icon, click the Diagnose button in the Plug-ins editor toolbar. A dialog appears showing the plug-in issues.

Troubleshoot Execution

When a plug-in fails to execute in an integration job, the job is suspended, and the error raised by the plug-in is displayed in the job’s execution log.
See the Managing Execution chapter in the Semarchy xDM Administration Guide for more information about jobs and logs.

To troubleshoot with more details a plug-in behavior at run-time, it is possible activate a logger that traces plug-in execution in the application server log files.

To activate this logger:

  1. Open the Logging Configuration in the Administration view (in the Administration Console perspective).

  2. Add or modify the log4j.logger.com.semarchy.platform.engine.PluginExecution logger configuration and set it temporarily to DEBUG as needed (an example is given below).

  3. Press CTRL-S to save the new configuration.

log4j.logger.com.semarchy.platform.engine.PluginExecution=DEBUG

The DEBUG mode is very verbose. Make sure to revert the configuration after the troubleshooting phase.

Disabling Type Checking

With the explicit declaration of the Parameters and Input/Output Types, checks are enforced on the data types, including in Data Entry forms. To disable such checks, set the following system property for the application server environment:

	com.semarchy.engine.disableDataRowenforceType = true

Appendices

Appendix A - Source Code of the Plug-in

The following sample is the source code for the plug-in class described in this book.

package com.acme.phonestandardizer;

import java.util.ArrayList;
import java.util.List;
import com.semarchy.engine.extensionpoint.DataRow;
import com.semarchy.engine.extensionpoint.IDataRow;
import com.semarchy.engine.extensionpoint.IRowTransformer;
import com.semarchy.engine.extensionpoint.IRowTransformerSetupInfo;

public class IntlPhoneStandardizer implements IRowTransformer {

        private boolean nullifyOnError;

        public IntlPhoneStandardizer() {
        }

        /**
         * Normalize a French phone number to E164 international format.
         *
         * @param pText
         *            Input phone number.
         * @return Standardized phone number. If phone number cannot be
         *         standardized, returns null (if nullifyOnError is true) or the
         *         input phone number.
         */
        private String normalizePhoneNumber(String pText) {

                StringBuilder normalized = new StringBuilder(pText.length());
                for (char c : pText.toCharArray()) {
                        int digit = Character.digit(c, 10);
                        if (digit != -1) {
                                normalized.append(digit);
                        }
                }

                switch (normalized.length()) {
                case 9:
                        normalized.insert(0, "+33");
                        break;
                case 10:
                        if (normalized.charAt(0) == '0') {
                                normalized.replace(0, 1, "+33");
                        } else {
                                return invalidPhoneNumber(pText);
                        }
                        break;
                case 11:
                        if (normalized.charAt(0) == '3' && normalized.charAt(1) == '3') {
                                normalized.insert(0, '+');
                        } else {
                                return invalidPhoneNumber(pText);
                        }
                        break;
                default:
                        return invalidPhoneNumber(pText);
                }

                return normalized.toString();
        }

        /**
         * Utility method: Returns null or pText depending on the value of the
         * nullifyOnError variable value.
         *
         * @param pText
         *            Input text.
         * @return Null or pText depending on the value of the nullifyOnError
         *         variable value.
         */
        private String invalidPhoneNumber(String pText) {
                // This method is used to return null of the original phone number
                // depending on the value of the nullify On Error parameter.
                if (nullifyOnError)
                        return null;
                else
                        return pText;
        }

        /**
         * This method transforms an input IDataRow into an output IDataRow. When
         * implementing this method, use: - the getValue method to retrieve the
         * input field from the inputIDataRow - the setValue method to set the
         * output field value of the output IDataRow
         */
        private IDataRow transformOneRow(IDataRow pDataRow) {
                // First, create the returned IDataRow.
                DataRow outputDataRow = new DataRow();
                // Make sure to set a null value for each output field in the
                // outputDataRow.
                outputDataRow.setValue("STANDARDIZEDPHONE", null);
                // The transformation is done below. It uses the normalizePhoneNumber
                // method defined in the class.
                if (pDataRow.getValue("INPUTPHONE") != null) {
                        outputDataRow.setValue("STANDARDIZEDPHONE",
                                        normalizePhoneNumber(pDataRow.getValue("INPUTPHONE")
                                                        .toString()));
                }
                return outputDataRow;
        }

        /**
         * (non-Javadoc)
         *
         * @see com.semarchy.engine.extensionpoint.IRowTransformer#transform(java.util.List)
         *      This method transforms a list of rows. In this case, it is a basic
         *      loop calling the transformOneRow method.
         */
        @Override
        public List<IDataRow> transform(List<IDataRow> pDataRowsBatch) {
                ArrayList<IDataRow> outputDataRowList = new ArrayList<IDataRow>();
                for (IDataRow inputRow : pDataRowsBatch) {
                        outputDataRowList.add(transformOneRow(inputRow));
                }
                return outputDataRowList;
        }

        /**
         * (non-Javadoc)
         *
         * @see com.semarchy.engine.extensionpoint.IRowTransformer#setUp(com.semarchy.engine.extensionpoint.IRowTransformerSetupInfo)
         *      This method performs the setup of the enricher. It retrieves
         *      parameters through a SetupInfo object.
         */

        @Override
        public void setUp(IRowTransformerSetupInfo pSetupInfo) {
                    nullifyOnError = pSetupInfo.getParameterAsBoolean("NULLIFYONERROR");
        }

        @Override
        public void tearDown() {

        }

}
        public void tearDown() {

        }

}

Appendix B - Source Code of the JUnit Test Class

The following sample is the source code for the JUnit test class described in this book.

package com.acme.phonestandardizer.tests;

import static org.junit.Assert.*;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.junit.Test;

import com.semarchy.engine.extensionpoint.DataRow;
import com.semarchy.engine.extensionpoint.IDataRow;
import com.semarchy.engine.extensionpoint.RowTransformerExecutor;

public class TestPhoneFrance {

        @Test
        public void testTransformNullPhones() {
                checkPhoneTransform(true, null, null);
                checkPhoneTransform(false, null, null);
        }

        @Test
        public void testTransformBadPhones() {
                checkPhoneTransform(true, "abcd", null);
                checkPhoneTransform(false, "abcd", "abcd");
                checkPhoneTransform(true, "64169710", null);
                checkPhoneTransform(false, "64169710", "64169710");
                checkPhoneTransform(true, "111164169710", null);
                checkPhoneTransform(false, "111164169710", "111164169710");
                checkPhoneTransform(true, "+44664169710", null);
                checkPhoneTransform(false, "+44664169710", "+44664169710");
                checkPhoneTransform(true, "1664169710", null);
                checkPhoneTransform(false, "1664169710", "1664169710");
        }

        @Test
        public void testTransformGoodPhones() {
                checkPhoneTransform(true, "0664169710", "+33664169710");
                checkPhoneTransform(true, "664169710", "+33664169710");
                checkPhoneTransform(true, "+33664169710", "+33664169710");
                checkPhoneTransform(true, "0478339229", "+33478339229");
                checkPhoneTransform(true, "(0)4.78.33.92.29...", "+33478339229");
        }

        /**
         *
         * Tests the behavior of the phone transformer. It takes three parameters
         * corresponding to the parameter, the input and output string of the
         * transformeR.
         *
         *
         * @param nullifyOnError
         * @param inputPhone
         * @param expectedOutputPhone
         */
        private void checkPhoneTransform(Boolean nullifyOnError, String inputPhone, String expectedOutputPhone) {

                // Define the mapped inputs.
                Set<String> mappedInputs = new HashSet<String>();
                mappedInputs.add("INPUTPHONE");

                // Define the mapped outputs.
                Set<String> mappedOuputs = new HashSet<String>();
                mappedOuputs.add("STANDARDIZEDPHONE");

                // Define the mapped parameters
                Map<String,Object> parameters = new HashMap<String, Object>();
                parameters.put("NULLIFYONERROR", nullifyOnError);

                // Create the rowTransformerExecutor
                RowTransformerExecutor rowTransformerExecutor = new RowTransformerExecutor("com.acme.phoneStandardizer.intlPhoneStandardizer",null,mappedInputs,mappedOuputs);

                // Setting up the plug-in.
                rowTransformerExecutor.setUp(parameters);

                try{

                        // Creating a new data row with the test phone number
                        IDataRow inDataRow = new DataRow();
                        ((DataRow) inDataRow).setValue("INPUTPHONE", inputPhone);

                        // Inserting this data row in a list of data rows.
                        List<IDataRow> inDataRowList = new ArrayList<IDataRow>();
                        inDataRowList.add(inDataRow);

                        // Enriching the list of input data rows.
                        List<IDataRow> outDataRowList = rowTransformerExecutor.transform(inDataRowList);

                        // Testing the resulting phone number.
                        assertEquals(expectedOutputPhone, outDataRowList.get(0).getValue("STANDARDIZEDPHONE"));

                } finally {

                        // Tearing down the plug-in.
                        rowTransformerExecutor.tearDown();
                }
        }

}