Develop your plug-in

This page provides step-by-step instructions to develop a Semarchy xDM plug-in.

Create the plug-in project

A Semarchy xDM plug-in is developed as part of a plug-in project, into which you declare extension points for each plug-in contained in this project.

Create a new plug-in project

A plug-in project contains one or more Semarchy xDM plug-ins and can be deployed as a single jar.

To create a new plug-in project:

  1. In Eclipse, select File > New > Others….

  2. In the Wizards filter, enter Plug-in. The list shows the Plug-in Project wizard.

  3. Select the Plug-in Project wizard and then click Next

  4. In the Project Name, enter the name of your project (e.g., com.acme.phoneStandardizer). Leave the other fields as is.

  5. Click Next. In the Content page, change the following fields:

    • Version: version of your plug-in (e.g., 1.0.0)

    • Name: Visible Name for your plug-in (e.g., Phone Standardizer)

    • Provider: Your company’s name (e.g., ACME Corp.)

    • Un-select the Generate an activator, This Plug-in will make contributions to the UI and Enable API Analysis options.

    • Make sure the Would you like to create a rich client application? option is set to No.

  6. Click Next

  7. Un-select the Create a plug-in using one of the templates option.

  8. Click Finish

  9. If asked, click Yes to switch to the development perspective.

Add the extensions to the project

To define Semarchy xDM extension points, you need to add these extension points' types to the project.

To add the Semarchy xDM extension points to the project:

  1. In the plug-in editor, select the Dependencies tab

  2. In the Required Plug-ins, click Add… and then select the com.semarchy.engine.extensionpoint.
    The extension point is added to the dependencies.

  3. The extension point is added with a minimum version required equal to the version of the plug-in SDK. For example, if you have installed the SDK in version 3.1.0, the minimum server version required is 3.1.0.
    To make your plug-in compatible with previous minor versions of the server (for example, to install the plug-in on a 3.0.0 server), you must modify this dependency.

    1. Select the extension point in the list and then click the Properties… button.

    2. In the com.semarchy.engine.extensionpoint dialog, edit the Minimum Version field (e.g., replace 3.0.1 with 3.0.0 to make this plug-in dependent on any server version after 3.0.0).

    3. Click OK to close the dialog.

  4. Select the Overview tab.

  5. In the Overview tab, click the Extensions hyperlink. The editor switches to the Extensions tab.

  6. In the Extensions tab, click Add…, and select the extension point corresponding to the type of plug-in you want to deliver in this project:

    • com.semarchy.integration.extensionpoint.rowTransformer: This extension point type is used to create enrichers.

    • com.semarchy.integration.extensionpoint.rowValidator: This extension point type is used to create validations.

  7. Repeat the previous step to add both types of extensions if you want to mix enrichers and validations in your project.

Define the plug-in

A project can contain several Semarchy xDM plug-ins (for example, a plug-in to enrich telephone numbers, and another one to validate email addresses). Each plug-in is declared as an extension endpoint of the rowTransformer or rowValidator endpoint type.

An extension endpoint is declared with all the plug-in metadata: The name, version, provider, as well as the parameters, input and output fields definition.

Declare an extension endpoint

For the rest of this example, you will focus on creating an Enricher (Row Transformer). Validation plug-ins follow the same development path.

To create an extension endpoint:

  1. Right-click the com.semarchy.integration.extensions.rowTransformer extension you just added, right-click and select New > Row Transformer.

  2. In the Extension Element Details section, enter the following values:

    • Id: ID of the enricher/validation (e.g., com.acme.phoneStandardizer.intlPhoneStandardizer). Note this ID as it will be used later in the test cases.

    • Name: Text name of the enricher or validation (e.g., International Phone Standardizer)

    • Provider Name: Your company’s name (e.g., ACME Corp.)

    • isThreadSafe: Set this option to true if you know that your code is thread-safe. Otherwise, set it to false. See the Parallel execution and thread safety for more information about this option.

  3. Click the Class link.

  4. In the New Java Class wizard, enter the following:

    • Package: Name of the package containing the class for your enricher/validation (e.g., com.acme.phonestandardizer)

    • Name: Name of the class for your enricher/validation (e.g., IntlPhoneStandardizer)

  5. Click Finish. The code editor for your class opens.

Parallel execution and thread safety

Semarchy xDM supports parallel processing of records in plug-ins, provided that the plug-in is designed and declared as thread-safe. By setting the isThreadSafe option to true in a rowTransformer or rowValidator extension point, you declare that the code of the plug-in is thread-safe and can be executed with several parallel threads.

It is the plug-in developer’s responsibility to review and test the plug-in to make sure of its thread safety before setting the isThreadSafe option to true.

Declare the parameters and input/output fields

Parameters and input/output types

Plug-ins use declared types for the parameters, inputs and outputs. When setting the values for the parameters and inputs, and when mapping the outputs, appropriate typing allows design-time validation and prevents run-time issues.

In the plug-in code, when managing inputs and parameters, each declared type will correspond to a given Java type. Make sure to define and cast your variable according to the type of the input/output or parameter that they will map to.

The table below provides the correspondence between the plug-in types and Java types.

Plug-in Type Corresponding Java Type

BINARY

java.lang.Byte[]

BOOLEAN

java.lang.Boolean

BYTE_INTEGER

java.lang.Byte

DATE

java.util.LocalDate

DATETIME (deprecated, use DATE instead)

java.util.Date

TIMESTAMP

java.util.Date

DOUBLE

java.lang.Double

FLOAT

java.lang.Float

INTEGER

java.lang.Integer

LONG_INTEGER

java.lang.Long

SHORT_INTEGER

java.lang.Short

STRING

java.lang.String

Declare the parameters

Parameters allow customizing the behavior of the plug-in.

To declare the plug-in parameters:

  1. In the plug-in editor, select the Extensions tab

  2. Select the parameters node under your Extension Point, right-click and select New > param.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the parameter (e.g., NULLIFYONERROR)

    • Type: Type of the parameter (e.g., BOOLEAN)

    • Label: User-facing label for this parameter (e.g., Nullify on Error)

    • shortDescription: User-facing description for this parameter (e.g., If set to 'true', returns null for a phone number that cannot be processed. Otherwise returns the original phone number.)

    • isMandatory: Set to true to make this parameter mandatory.

  4. Repeat the two previous steps to add more parameters.

Declare the input fields

Input fields are the values received by an enricher or validation plug-in.

To declare the plug-in input fields:

  1. In the plug-in editor, select the Extensions tab

  2. Select the inputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the field (e.g., INPUTPHONE). This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: Type of the field (e.g., STRING)

    • Label: User-facing label for this field (e.g., Input Phone)

    • shortDescription: User-facing description for this field (e.g., Phone Number to be standardized)

    • isMandatory: Set to true to make this field mandatory.

  4. Repeat the two previous steps to add more input fields.

Declare the output fields

Output fields are the values returned by an enricher plug-in.

To declare the plug-in output fields:

  1. In the plug-in editor, select the Extensions tab

  2. Select the outputFields node under your Extension Point, right-click and select New > field.

  3. In the Extension Element Details section, enter the following values:

    • Name: Name of the field (e.g., STANDARDIZEDPHONE). This name should only contain uppercased alphanumerics and should not be longer than 30 characters.

    • Type: Type of the field (e.g., STRING)

    • Label: User-facing label for this field (e.g., Standardized Phone)

    • shortDescription: User-facing description for this field (e.g., Standardized Phone Number)

    • isMandatory: Set to true to make this field mandatory (a value is always returned).

  4. Repeat the two previous steps to add more output fields.

Make sure that the input and output field names only contain uppercased alphanumerics and are no longer than 30 characters.
Make sure to use uppercase for the parameter and field names. These names are not visible to the end-users.

Implement the plug-in code

Now that the plug-in is declared, it is possible to implement the specified behavior.

To implement the plug-in code:

  1. Switch to the class editor.

  2. If needed, Use the Eclipse quick fix feature to add the interface methods currently unimplemented in the code.

  3. Implement the required methods for your plug-in:

    • For a rowTransformer: the setUp, tearDown and transform Methods.

    • For a rowValidator: the setUp, tearDown and isValid Methods.

Download the plug-in source code: IntlPhoneStandardizer.java.

The following sections describe the methods implemented in the plug-in code.

Setup/teardown methods

These methods are respectively used to initialize the plug-in - typically by setting the value of the parameters - and to release the resources used by the plug-in.

When a plug-in is used in Semarchy xDM, the SetUp and TearDown methods are not called if no record is passed to the plug-in for processing.

In this case, the only parameter is NULLIFYONERROR. This parameter’s value is stored in a private Boolean variable called nullifyOnError.

Note the use of the getParameterAsBoolean method to retrieve a boolean corresponding to the type set while [_declare_the_parameters_and_input_output_fields]. The same methods exist for other supported data types.

private boolean nullifyOnError;
...
@Override public void setUp(IRowTransformerSetupInfo pSetupInfo) {
	nullifyOnError = pSetupInfo.getParameterAsBoolean("NULLIFYONERROR");
}

@Override public void tearDown() {

}
String referring to parameter names should always be in uppercase in the Java code.
The IRowTransformerSetupInfo object can be used to retrieve the data location’s datasource, using the getTargetDataSource() method.

Transform method (enricher plug-ins)

Enrichers plug-ins enrich a list of data rows by calling the transform method. This method returns the list of enriched data rows.
In our example, this method simply calls a private method called transformOneRow to process one of these data rows.

@Override public List<IDataRow> transform(List<IDataRow> pDataRowsBatch) {
        ArrayList<IDataRow> outputDataRowList = new ArrayList<IDataRow>();
        for (IDataRow inputRow : pDataRowsBatch) {
                outputDataRowList.add(transformOneRow(inputRow));
        }
        return outputDataRowList;
}

The transformOneRow method transforms an input data row (IDataRow) into an output data row.

  • The input data row is passed to the transform method in the pDataRow variable, and contains the input fields passed to the enricher. You access each field value using the getValue method.

  • The output data row is created and returned by this method, with the output fields set using the setValue method.

In the example below, the input data row only contains the INPUTPHONE field and the output data row is built only with the STANDARDIZEDPHONE field.
The core of the transformation takes place in the private method normalizePhoneNumber.

An example of this method is provided below. Review the complete code of this method in the IntlPhoneStandardizer.java source code.

private IDataRow transformOneRow(IDataRow pDataRow) {
        // First, create the returned IDataRow.
        DataRow outputDataRow = new DataRow();

        // Make sure to set a null value for each output field in the
        // outputDataRow.
        outputDataRow.setValue("STANDARDIZEDPHONE", null);

        // The transformation is done below. It uses the normalizePhoneNumber
        // method defined in the class.
        if (pDataRow.getValue("INPUTPHONE") != null) {
                outputDataRow.setValue("STANDARDIZEDPHONE",
                                normalizePhoneNumber(pDataRow.getValue("INPUTPHONE")
                                                .toString()));
        }
        return outputDataRow;
}
It is recommended to set all the mandatory output field values to null immediately after creating the output DataRow. This avoids returning partially complete data row as the output. This is done by issuing the following code for each output field outputDataRow.setValue("<output_field_name>", null);.
String referring to input/output field names should always be in uppercase in the Java code.

Isvalid method (validation plug-ins)

The isValid method is required for a validation plug-in.
This method is similar to the transform method. It takes as an input a list of data rows and returns a list of boolean values indicating whether the input data rows comply with the validation specified in the plug-in. Typically, this method uses an isValidOneRow private method that validates a single data row.