Semarchy xDM 3.2 release notes

Feature highlights

Semarchy xDM 3.2 introduces new features enabling data stewards and business users to easily manage their master data. It includes customized search forms and a new match and merge technology supporting multiple match rules, suggestions and automated merge and confirmation of duplicates groups based on scoring.

Multiple match rules with automated validation and scoring

This version includes an extended configuration for matching records. Matchers now support multiple match rules, each rule defining a specific condition for detecting possible duplicates, with a confidence score. Depending on confidence scores, actions are automated triggered on match groups, to suggest or perform a group merge, and optionally confirm the consolidated golden record.

Improved duplicate management editor

To complement the new matching mechanism, the duplicate management editor has been improved and simplified to support matching suggestions and scoring. Duplicates are displayed using grouped view and customized tables to facilitate analysis and decisions on duplicates.

Customized search forms

Designers can now create customized forms to search data in the hub. These search forms support simple and complex search patterns as well as display types for the search parameters. Designers can also choose the search methods and search forms available to the business users in the business object views of the MDM applications.

Lineage navigation in SemQL

The SemQL language is extended to support lineage navigation. Using the updated syntax, designers can define rules and filters that seamlessly leverage golden, master or source records, as well as the errors.

Post-consolidation enrichers

Enrichers can now run not only on source data but also on consolidated records. Using these enrichers, matched and merged data passes through additional augmentation and standardization transformations before being published into golden records.

Semarchy Pulse dashboard enhancements

Designers can configure the Semarchy Pulse dashboards to show metrics for the current application only or for all applications. Each dashboard has been localized and now displays additional description and tooltips explaining the dashboard and the metrics. The color schemes used in the dashboards have also be enhanced for better consistency and readability.

Database functions

Designers can declare in the data model database functions and packages developed separately. By declaring these functions, they make them available in the SemQL editor and make possible syntactic validation while editing expressions.

Auto-refresh

Designers can configure applications and business object views to support auto-refresh. With auto-refresh, workflow lists as well as data tables, automatically refresh to reflect in the application user interface the most recent information from the hub.

Folders and business object view customized order

Designers can explicitly define the order of the folders and business objects displayed in the applications, using alphabetical or positional ordering.

Other notable changes

Design time

  • Workflow transitions can be configured to skip the complete task dialog.

  • Integration job batch ID (V_BATCHID) is available for enrichers and validators

  • New Download Hyperlink display type for binary content that should be downloaded by default.

MDM applications

  • The record edition dialog no longer displays the Enrich and Validate buttons if no enricher or validation is configured for the task.

  • The record edition dialog now supports immediate data save and task completion.

  • Transitions no longer show the complete task dialog depending on the workflow configuration.

  • Data entry workflows now support Select All then Remove action.

  • Duplicate manage workflows support Select All actions.

  • Tables in the application now save column width in the user preferences.

  • Pulse dashboard for duplicates now exposes the confirmation status for the golden records.

  • Workflow overview and workflow lists now display the workflow submit/cancel date and time for a completed task.

  • Claim task is checked by default in the complete task dialog if the user can claim the next task.

  • Notify Assignee is checked by default in the complete task dialog if notification is possible.

  • The Suggest Mapping feature in the data import wizard is enhanced to support reference fields (e.g., Column named like FID_…​, )

  • Number form fields raise better warning for invalid or overflowing numbers.

Administration

  • Model edition deployment uses a non-modal dialog during job execution.

  • OpenID Connect Connector is provided for Tomcat

Integration

  • New Web Service exposes ID generation.

Changelog

Version 3.2.5

Bug fixes

  • MDM-4306: "Align merged Exclusion Groups Ids" task merges too many exclusion groups.
    These exclusion groups cause large number of records to be automatically added to duplicate management workflows. This fix changes the integration jobs generation to prevent excessive merges of exclusion groups. For this fix to be effective, you must re-deploy the model in order to force the re-generation of the integration jobs.

Version 3.2.4

Bug fixes

  • MDM-4054: Singleton are not matched after an upgrade to 3.2.

This version only fixes the upgrade process from 3.1 to 3.2. Customers who have already upgraded to version 3.2 should refer to the Knowledge Base Article that explains how to fix an existing upgrade.

Version 3.2.3

Bug fixes

  • MDM-3955: Data edition on LongText fields fails with a No trespass RuntimeException

  • MDM-3935: The SemQL editor button on the Input Email Address field of the Email Validator Plugin pops up a blocking error

  • MDM-3957: NullPointerException (fitScrollableSizeToForm) issued when opening a Custom Search Form when other search methods are available.

  • MDM-3952: The integration load specific web service persists incorrect values when multiple complex attributes have definition attributes with the same name

New features

  • MDM-3931: Golden integration records are navigable from the post integration (golden error) records in SemQL.

  • MDM-3945: Additional information provided in the upgrade process for users having defined attributes with reserved physical names (starting with B_).

Version 3.2.2

New features

  • MDM-4291 - Default value for PARAM_ANALYZE_STATS job parameter is now 1.

Known issues

  • MDM-3883 - Error when discarding a record when creating more than one record in a data entry workflow

Bug fixes

  • MDM-2480 - Form fields are too small in a Filter by Form.

  • MDM-3857 - The PARAM_ANALYZE_STATS in existing jobs switches to 1 in all jobs after an upgrade.

  • MDM-3871 - "Select all" ignores the Business Object root filter when checking out records for duplicates management.

  • MDM-3872 - Email enricher causes load job to fail with javax.naming.NameNotFoundException: Name [null] is not bound in this Context. Unable to find [null] error after an upgrade to 3.2.1. This appears when the Datasource parameter is left empty in the job definition.

  • MDM-3876 - On an application, if you use required role for an entity view, the semadmin user is no longer be able to see the entity.

  • MDM-3878 - Workflow notification email inverts the Application and Workflow names, and provides obsolete steps to claim a task.

  • MDM-3881 - Chart are not displayed and the UI freeze when opening Pulse Dashboards with Internet Explorer, Chrome versions prior to 39 and Firefox versions prior to 34.

  • MDM-3901 - A "None" Privilege Grant causes Data Entry or import to fail with a com.semarchy.mdm.runtime.data.DataAccessPermissionDeniedException even if the secured attributes are not used in import or data entry.

  • MDM-3903 - Cannot start engine after upgrade to 3.2.1 due to integration batches pending without job instances.

  • MDM-3904 - A Search Form with a large number of parameter does not appear with a scrollbar.

  • MDM-3913 - Browser shows a cryptic {"head":{},"operations":[]} error message after logging in.

Version 3.2.1

Known issues

  • MDM-3857 - The PARAM_ANALYZE_STATS switches to 1 in all jobs after an upgrade to this version. As a workaround, customers should edit their jobs and set the PARAM_ANALYZE_STATS parameter value to 0.

  • MDM-3872 - Email enricher causes load job to fail with javax.naming.NameNotFoundException: Name [null] is not bound in this Context. Unable to find [null] error after an upgrade to 3.2.1. This appears when the Datasource parameter is left empty in the job definition. As a workaround customer should remove the empty parameter from the enricher and redeploy their jobs.

New features

  • The default schema name for Semarchy Pulse Metrics is now SEMARCHY_DEMO_PULSE.

  • MDM-3671 - Logout page made more user-friendly.

Bug fixes

  • MDM-3621 - Confusing validation message reported by data entry for an incorrect data type attribute value

  • MDM-3645 - Semarchy xDM application does not start on Oracle WebLogic Application Server 12C. This issue is due to a WebLogic bug.

  • MDM-3678 - Semarchy xDM does now work with the Microsoft Edge Browser.

  • MDM-3683 - Using "any Children" in a business object filter or a privilege grant condition on fuzzy matching entities causes a validation error

  • MDM-3684 - Data location upgrade fails when entities have been converted from fuzzy to exact matching in previous model editions

  • MDM-3688 - A display name containing a '&' character is not consistent between hierarchy and form view

  • MDM-3690 - Invalid thread access error when uploading plug-in in the administration view

  • MDM-3701 - HTTP headers are incorrectly set in HTTP notification servers

  • MDM-3707 - Unhandled event loop exception appears when re-opening the search view in a workflow.

  • MDM-3712 - The type of the ID attribute of an UDPK entity is automatically changed to string(128) after an 'Alter Entity' action when opening the ID column, even if the action is not saved.

  • MDM-3751 - Generated SQL diverge due to lax SemQL validation (many path expressions inside any/all)

  • MDM-3752 - NullPointerException when opening a Business Object View with a filter created "By Form" saved in the user preferences

Version 3.2.0

Known issues

  • MDM-3645 - Semarchy xDM application does not start on WebLogic Application Server 12C. This issue is due to a WebLogic bug.

New features

  • MDM-1087 - Multiple Match Rules with Automatic Validation and Scoring

  • MDM-1328 - The Claim Task option is checked by default in activities transitions

  • MDM-1688 - Pulse schemas upgrade like data locations and repository

  • MDM-1791 - Configurable match condition to rematch on previous group

  • MDM-1947 - Use a non-blocking dialog when deploying a model edition to a data location

  • MDM-1960 - Support MastersCount in the Duplicate management workflows forms, tables, and filters

  • MDM-1961 - Support "Remove" action with select all in data entry workflows

  • MDM-1963 - Identify group change in duplicate management workflows

  • MDM-1966 - Support attributes from referenced entities in duplicate management workflow tables

  • MDM-1980 - Support lineage relationship in SemQL Expressions and Conditions

  • MDM-2067 - Use literal values instead of SQL binding for some variables of the integration job

  • MDM-2138 - Browse duplicate management workflow content grouped by Current Golden

  • MDM-2159 - Dashboard should support properly measure with zero metrics

  • MDM-2199 - Port the Pulse "Workflow - Duration" dashboard in the MDM Application

  • MDM-2020 - Add filtering and Select All option in duplicate management workflows

  • MDM-2248 - Support duplicate management actions with "Select All"

  • MDM-2306 - Add the golden records' Confirmation Status metric to Pulse

  • MDM-2379 - Localization of the pivot field and measures in Pulse

  • MDM-2399 - Document the configuration of Jetty 9 application server

  • MDM-2406 - Actions in duplicate management workflows should not shuffle records

  • MDM-2408 - Reorganized Duplicate Statistics in duplicate management workflows

  • MDM-2465 - Support user defined PL/SQL functions to prevent SemQL warnings

  • MDM-2497 - Improvements to the "Suggest Mappings" feature in data import to handle references

  • MDM-2649 - Preserve table columns width in applications as part of the user preferences

  • MDM-2769 - Limit embedded Pulse Dashboards content to the content related to application

  • MDM-2771 - New "Download Hyperlink" display type for binary attributes

  • MDM-2815 - OpenID Connect 1.0 connector for Tomcat

  • MDM-2860 - Pulse dashboard axis should not display the decimal places when a metric is an integer value

  • MDM-2866 - Navigation Link to the golden record when golden ID is used in master data forms

  • MDM-2873 - Configurable auto-refresh for activities lists and business object views

  • MDM-2980 - Support "Remove All" action in duplicate management workflows

  • MDM-3000 - Customized search forms for business users

  • MDM-3019 - Configure the search types available for each node of a business object view

  • MDM-3031 - Localization of the values in Pulse

  • MDM-3063 - Workflow overview and list should display workflow submit/cancel date and time for completed task

  • MDM-3102 - Show Information and Tooltip on Pulse Dashboards

  • MDM-3138 - Notify assignee is checked by default when notification is possible

  • MDM-3162 - Support design time ordering of folders and business object views in applications

  • MDM-3181 - Browse duplicate management workflow content using a flat view

  • MDM-3184 - Support sorting when in duplicate management workflows

  • MDM-3204 - Complete a task from the data editing dialog and make the task completion dialog optional

  • MDM-3205 - Hide "Validate" and "Enrich" buttons if no validations or enrichments are defined for the task

  • MDM-3245 - "Demo Applications" in the Getting Started renamed "Sample Source Data"

  • MDM-3253 - OpenID Connect Connector should support web services authentication

  • MDM-3331 - Numeric form fields should raise warnings for invalid numbers and datatypes errors

  • MDM-3343 - Post-consolidation enrichers

  • MDM-3355 - Select the table view used in duplicate management workflow at design time

  • MDM-3367 - Browse duplicates management workflow content grouped by Original Golden, Latest Confirmed Golden and Suggested Group

  • MDM-3394 - SemQL supports :V_BATCHID in enrichers and validators running only in integration job

  • MDM-3401 - Predictable colors scheme in Pulse Dashboards

  • MDM-3433 - Remove in duplicate management workflows

  • MDM-3514 - New SuggestedMergeConfidenceScore, SuggestedMergeMastersCount attributes computed by the matching process

Bug fixes

  • MDM-229 - Installing and Updating a Model Edition containing SemQL errors updates the DDL before failing during Integration Job creation

  • MDM-1687 - Column D_BATCH.JOB_DEPRECTAED should be D_BATCH.JOB_DEPRECATED

  • MDM-1984 - Failed Model Deployment keeps the model marked as deployed

  • MDM-2109 - Null file when exporting (XLS, XLSX, CSV) data from a business object view using Firefox

  • MDM-2284 - User interface main window does not always resize to browser’s window size

  • MDM-2790 - Semarchy AddressEnricher OnDemand does not raise an error if the on demand server is down

  • MDM-2950 - Single-click on an editor tab has no effect in Internet Explorer 11. A double-click is required

  • MDM-2994 - Keyboard tab order in Grid Layout does not work properly

  • MDM-3055 - Duration computed by Pulse Load Job are wrong

  • MDM-3062 - Invalid assigned date in Workflow List

  • MDM-3065 - Scrollbars do not scroll as expected when clicking the track

  • MDM-3072 - StackOverflow error when enabling a quick filter after browsing a reference

  • MDM-3079 - Form attribute type is not re-evaluated after modifying the SemQL expression

  • MDM-3082 - Workflow notification fails with a "No recipient" error even when emails are configured for the roles

  • MDM-3083 - Error on one notification prevents the execution of subsequent notifications

  • MDM-3084 - ConcurrentModificationException while engine sends notifications

  • MDM-3089 - Pulse slicer labels should be shorter

  • MDM-3093 - Handling of acronyms in physical names generation should follow what is done for labels generation

  • MDM-3101 - Incorrect exception logged in console when using LOV Sort Mode = LABEL

  • MDM-3104 - Random http 500 crash when browser zoom is activated

  • MDM-3105 - Editing a record disables the filter currently activated

  • MDM-3110 - Thread Pool Size defined in enrichers and validators is ignored at execution time

  • MDM-3134 - Model name is unexpectedly altered during model import

  • MDM-3148 - The Checkout wizard for data entry workflows does not list user filters

  • MDM-3150 - SemQL parsing fails with "Unexpected error while rewriting AST" when using a case when (case x when y syntax)

  • MDM-3154 - The workflows' "Allow Properties Edition" property renamed to "Editable Properties"

  • MDM-3158 - Opening an existing purge schedule takes the current time instead of the configured time

  • MDM-3159 - New sections in a flow layout are added above instead of below the selected section

  • MDM-3168 - Logging using the log4j JMS appender fails with a javax.naming.NoInitialContextException

  • MDM-3201 - Duplicate Search Menu when enabling quick filter created by reference browsing

  • MDM-3208 - Default values in Pulse Cron schedule should reflect the current schedule

  • MDM-3256 - Excel export renders the attributes' SemQL expressions instead of the names

  • MDM-3257 - Incorrect field size check for string primary keys causes unhandled event loop exception in data entry

  • MDM-3261 - Columns are not properly sized after changing displayed columns when auto-fit is on

  • MDM-3273 - ID Matching entity with a manual ID should not allow editing of ID after initial record creation

  • MDM-3314 - Missing Consolidation Strategy button when switching consolidator from row-level to field-level

  • MDM-3323 - ".ForeignPublisherID" reference name suffix renamed to ".PublisherID" for consistency in data export

  • MDM-3333 - Guess Mapping button should be enabled only if "1st Line as Header" is checked

  • MDM-3337 - Integer Field using a number format allows entering overflowing values

  • MDM-3340 - Unable to order in descending mode in the form search value picker

  • MDM-3345 - "Master duplicates within a publisher" metric does not appear in Pulse dashboards

  • MDM-3375 - A whitespace at the end of a form attribute’s SemQL expression prevents data import in the attribute

  • MDM-3378 - Google Maps Enricher does not support Proxy authentication. The http.proxyUser and http.proxyPassword system properties are now supported

  • MDM-3393 - Export takes all table view attributes and ignores the "Export Displayed Table Columns" option

  • MDM-3404 - Inaccurate privileges error message when submitting data with an invalid payload to the web services

  • MDM-3416 - The person name enricher data file contains incorrect names, which leads to incorrect name detection

  • MDM-3481 - System information dialog crashes with permanent license Keys (License key without expiration date)

  • MDM-3500 - Model Privilege Grants missing "Allow Checkout"

  • MDM-3576 - Purge jobs are not updated on deployment data locations

  • MDM-3593 - "Unhandled Event Loop Exception" Error on import when the imported file size is greater than 2Mb.

  • MDM-3601 - Right-aligned label incorrectly aligns to the left when wrapping due to its length.

  • MDM-3595 - "Unhandled Event Loop Exception" Error displayed for a plugin when an input or output field name is longer than 30 characters.

  • MDM-3600 - Email enricher returns addresses containing apostrophe’s (single quotes and other special characters) as invalid. The enricher now adheres more strictly to RFC5322: It allows authorized special characters and supports quoted strings.