Semarchy xDM 2024.1 release notes

This page provides the latest information about Semarchy xDM version 2024.1, including new features and bug fixes.

Support information

Semarchy xDM version 2024.1 is a long-term support (LTS) release.

LTS product versions are released annually, and include all changes introduced since the last LTS release. They are maintained and supported for an extended period with patches and fixes.

Alternatively, mainstream support (MS) versions offer the latest improvements and features, and are released more frequently.

For more details about our releases and support policies, see our support and maintenance terms.

Feature highlights

Summary of 2023.x features

The following is a summary of the feature highlights from the 2023.2, 2023.3, and 2023.4 MS releases. More details on changes that have been made to the xDM software throughout the year are available in the documentation.

Since the introduction of Semarchy xDM data-driven workflows, previous releases have prioritized user collaboration and effective data management. Key enhancements include the workflow administration interface for monitoring and managing workflows, initiating multi-record workflows from different record selections, automating customizable workflow email notifications, scheduling workflow data and metadata purges, and starting workflows from erroneous records or via the REST API.

Mainstream releases have also introduced a wait dialog and automatic screen refresh for data authoring and duplication management, the Microsoft Purview connector for data visibility, and enhanced governance tools like per-data-type retention policies.

New REST APIs are also available to aid in automation of administrative and other tasks.

User-friendly validation messages in error views

Effectively communicating when an error occurs while authoring or editing a record is the first step in detecting and then correcting those errors.

When browsing records with errors, the Error Details tab in a business view now displays meaningful and detailed error messages for each listed error—​in addition to the constraint type and constraint name for built-in errors—​and provides the ability to switch between list and table views for comprehensive error details.

When querying error views using the REST API, the API response now includes a ConstraintErrorMessage attribute that contains either the customized or default error message associated with the error records.

These improvements to error messages reinforce the capacity of data stewards to design effective error communication. The increased clarity augments the ability of business users to act on them, facilitating the correction of errors in records.

User workflow and task history

Workflows allow collaboration across the organization for managing data. The introduction of the History feature provides users with visibility of workflows that they have initiated or completed.

Within My Tasks, the My Workflows section has been replaced with a new History feature that incorporates two distinct views:

  • The Workflows I Initiated view shows all workflows initiated by the current user. Detailed information—​including workflow description, status, dates, assignees, and more—​is presented in a structured table.

  • The My Completed Workflow Tasks view provides insights into tasks performed by the current user. Users can access comprehensive details such as a task description and completion date, or its related workflow status and progress, as well as records processed during the task.

Both views offer advanced search filters, enabling users to efficiently explore completed tasks and initiated workflows based on criteria like workflow ID, keywords found in the description, workflow name, status, date or time frame, and assignee.

User preferences and custom view options persist across sessions, ensuring a consistently personalized experience every time users access the History section.

REST API documentation

The REST API documentation has been updated to better handle large models, now organizing data location sections by entity. This improvement introduces a submenu for entity selection with enhanced search functionality, and allows downloading the API documentation for specific entities.

Other notable changes

Azure Database for PostgreSQL - Flexible Server

Semarchy xDM supports Azure Database for PostgreSQL - Flexible Server.

PostgreSQL 15 certification

Semarchy xDM supports PostgreSQL 15 across on-premises installations and cloud platforms, including Azure Database for PostgreSQL - Flexible Server, Amazon RDS for PostgreSQL, Cloud SQL for PostgreSQL, and Amazon Aurora PostgreSQL.

Authentication mechanism enhancement

For security purposes, xDM enforces a temporary block on the user’s IP address for 24 hours following five consecutive failed login attempts. This security measure applies solely to users connecting to xDM through the built-in identity provider (i.e., internal IDP).

Platform administrators can either modify the maximum number of attempts by adjusting the value of the xdm.idm.maxloginattempts property, or deactivate this safeguard by setting the xdm.idm.maxloginattempts.enable property to false using the appropriate startup configuration method.

Changelog

Version 2024.1.3

Known issues

  • MDM-7254: With PostgreSQL, the application server hosting the Semarchy xDM instance and the database server must be in the same timezone to avoid inconsistent dates in the integration loads.

  • MDM-7917: A boolean field default value is set to null when creating a record and to false when editing it.

  • MDM-7949: A model with a physical column name using a reserved name raises a validation error but can be deployed.

  • MDM-8034: When stopping Semarchy xDM using the Tomcat Manager, a Severe message is logged with org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks…​. This message is a known Tomcat issue that can be ignored.

  • MDM-8703: Data import from Excel does not properly handle dates before February 28, 1900.

  • MDM-8977: Changing a reference to a fuzzy-matching entity while authoring a master record is not taken into account.

  • MDM-9184: Existing FS_ and FP_ reference column values in a basic entity are overwritten by an enricher when this record is modified by the API or by authoring.

  • MDM-10193: When a record in an entity is updated, records with the same ID in inheriting entities are also updated.

  • MDM-10269: Using the SEM_NUMBER_TO_CHAR function with a string argument does not raise a model validation error but fails with an invalid number error in the log.

  • MDM-10319: Authoring the same record in two different browser windows makes the second authoring fail with a java.lang.IllegalStateException: Current Step (products) should be a form step error.

  • MDM-10738: Duplicate flexElement error occurs at runtime when using a transition named like a form element (e.g., "field", "container") in a business view.

  • MDM-12415: Long-integer values higher than 9223372036854700 raise errors when authoring (Invalid ID for data type (longInteger)) or browsing (<entity name> not found) records.

  • MDM-13099: When moving an attribute from one group to another governed by a different survivorship rule, any overridden value within this attribute causes all other attributes in the new group to be considered overridden as well. Consequently, the values of these attributes may be nullified during the subsequent execution of the certification process, unless override values have previously been defined. To minimize potential negative impacts on data consistency and accuracy, designers should consider moving attributes with pre-existing overridden values to a group that does not include any other attributes.

Bug fixes

  • MDM-14604: An unexpected backslash character (\) is observed at the end of the line just before a bullet-point list in bold formatting when inputting content into a long text field using a Markdown component.

  • MDM-14714: Attempting to apply row filter privileges to an entity with a SemQL condition involving an attribute of a parent entity and the IN operator fails with a java.lang.IllegalArgumentException error in the PDE log.

  • MDM-14800: X-Forwarded-For headers are available despite the allowXForwardedHeaders system property being set to false, potentially causing undesired header handling. For enhanced control and security, the default (false) behavior of the allowXForwardedHeaders property now removes X-Forwarded headers from requests.

  • MDM-14805: In the Configuration module, the Identity Provider Test side sheet fails to display logs when testing a SAML connection, regardless of whether the connection is made over HTTP or HTTPS.

  • MDM-14824: Security enhancement: REST API documentation.

  • MDM-15064: Applying row filter privileges to an entity, specifically when the condition references a child entity through a SemQL lookup expression, raises a java.lang.IllegalArgumentException error in the PDE log and makes the application inaccessible to users with restricted privileges.

  • MDM-15159: The execution engine’s performance has been optimized by reducing the number of database queries required to identify running loads in the continuous load process. For enhanced performance, platform administrators should create a custom database index on MTA_INTEG_LOAD(LOADID) to optimize queries that involve filtering or sorting by load ID.

  • MDM-15196: Third-party library upgrade: Apache Commons Compress.

  • MDM-15204: Third-party component upgrade: PostgreSQL JDBC driver.

  • MDM-15214: Third-party library upgrade: Apache Commons Compress.

  • MDM-15226: Attempting to create a replica datasource through the user interface when the original datasource was initially created using the REST API fails with the following JavaScript error message: Error: Minified React error #31.

  • MDM-15302: Within the Workflow Builder, the attribute list in the SemQL editor’s Assignment Condition view remains empty when the related model includes entities that inherit from other entities.

  • MDM-15308: When attempting to access a workflow user task from the My Tasks interface, a generic unexpected error occurs if the associated collection configured to display in My Tasks references a model variable, and the following error message is logged: com.semarchy.datahub.runtime.workflow.service.WorkflowEngineException: Missing named parameters.

  • MDM-15312: Security enhancement: datasource configuration.

  • MDM-15314: Security enhancement: login page content.

  • MDM-15317: Third-party component upgrade: Apache Tomcat for xDM preconfigured, Docker image, and cloud images (QuickStart VM and Azure Solution Template).

Version 2024.1.2

Known issues

  • MDM-7254: With PostgreSQL, the application server hosting the Semarchy xDM instance and the database server must be in the same timezone to avoid inconsistent dates in the integration loads.

  • MDM-7917: A boolean field default value is set to null when creating a record and to false when editing it.

  • MDM-7949: A model with a physical column name using a reserved name raises a validation error but can be deployed.

  • MDM-8034: When stopping Semarchy xDM using the Tomcat Manager, a Severe message is logged with org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks…​. This message is a known Tomcat issue that can be ignored.

  • MDM-8703: Data import from Excel does not properly handle dates before February 28, 1900.

  • MDM-8977: Changing a reference to a fuzzy-matching entity while authoring a master record is not taken into account.

  • MDM-9184: Existing FS_ and FP_ reference column values in a basic entity are overwritten by an enricher when this record is modified by the API or by authoring.

  • MDM-10193: When a record in an entity is updated, records with the same ID in inheriting entities are also updated.

  • MDM-10269: Using the SEM_NUMBER_TO_CHAR function with a string argument does not raise a model validation error but fails with an invalid number error in the log.

  • MDM-10319: Authoring the same record in two different browser windows makes the second authoring fail with a java.lang.IllegalStateException: Current Step (products) should be a form step error.

  • MDM-10738: Duplicate flexElement error occurs at runtime when using a transition named like a form element (e.g., "field", "container") in a business view.

  • MDM-12415: Long-integer values higher than 9223372036854700 raise errors when authoring (Invalid ID for data type (longInteger)) or browsing (<entity name> not found) records.

  • MDM-13099: When moving an attribute from one group to another governed by a different survivorship rule, any overridden value within this attribute causes all other attributes in the new group to be considered overridden as well. Consequently, the values of these attributes may be nullified during the subsequent execution of the certification process, unless override values have previously been defined. To minimize potential negative impacts on data consistency and accuracy, designers should consider moving attributes with pre-existing overridden values to a group that does not include any other attributes.

New features

  • MDM-14723: Semarchy xDM supports PostgreSQL 15 across on-premises installations and cloud platforms, including Azure Database for PostgreSQL - Flexible Server, Amazon RDS for PostgreSQL, Cloud SQL for PostgreSQL, and Amazon Aurora PostgreSQL.

  • MDM-15092: Platform administrators can deactivate the default security enforcement that temporarily blocks users after a certain number of unsuccessful login attempts by setting the xdm.idm.maxloginattempts.enable startup property to false.

Bug fixes

  • MDM-14970: Despite setting a display text for hyperlinks in form fields, the full URL is displayed instead of the specified text on browsing forms.

  • MDM-14987: Custom translations are not supported for the My Task interface.

  • MDM-14988: Special characters are not encoded correctly when incorporated through a SemQL custom parameter in an email subject line configured within the properties of the Send Email workflow automation.

  • MDM-15091: When attempting to edit the value of a hyperlink field in an authoring form and intervene in the middle of a URL, the cursor unexpectedly jumps to the end of the edit box upon typing.

  • MDM-15105: When validations are disabled for a reference relationship and a record contains a broken reference, the corresponding form field’s Value tab displays <null> instead of the expected value indicating the broken reference.

  • MDM-15151: Attempting to override a reference on a record that originates from a fuzzy- or ID-matched entity within a workflow’s user task causes the subsequent Submit Data automation step to fail with a PreparedStatementCallback; bad SQL grammar error.

  • MDM-15152: When testing REST APIs set up in Semarchy xDM REST clients to receive data from an external API, an unexpected error occurs due to misconfigurations in the JSONPath library.

  • MDM-15167: When creating a dashboard, despite successful completion, an unexpected error page appears within the Dashboard Builder with no corresponding entry in the PDE log.

Version 2024.1.1

Known issues

  • MDM-7254: With PostgreSQL, the application server hosting the Semarchy xDM instance and the database server must be in the same timezone to avoid inconsistent dates in the integration loads.

  • MDM-7917: A boolean field default value is set to null when creating a record and to false when editing it.

  • MDM-7949: A model with a physical column name using a reserved name raises a validation error but can be deployed.

  • MDM-8034: When stopping Semarchy xDM using the Tomcat Manager, a Severe message is logged with org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks…​. This message is a known Tomcat issue that can be ignored.

  • MDM-8703: Data import from Excel does not properly handle dates before February 28, 1900.

  • MDM-8977: Changing a reference to a fuzzy-matching entity while authoring a master record is not taken into account.

  • MDM-9184: Existing FS_ and FP_ reference column values in a basic entity are overwritten by an enricher when this record is modified by the API or by authoring.

  • MDM-10193: When a record in an entity is updated, records with the same ID in inheriting entities are also updated.

  • MDM-10269: Using the SEM_NUMBER_TO_CHAR function with a string argument does not raise a model validation error but fails with an invalid number error in the log.

  • MDM-10319: Authoring the same record in two different browser windows makes the second authoring fail with a java.lang.IllegalStateException: Current Step (products) should be a form step error.

  • MDM-10738: Duplicate flexElement error occurs at runtime when using a transition named like a form element (e.g., "field", "container") in a business view.

  • MDM-12415: Long-integer values higher than 9223372036854700 raise errors when authoring (Invalid ID for data type (longInteger)) or browsing (<entity name> not found) records.

  • MDM-13099: When moving an attribute from one group to another governed by a different survivorship rule, any overridden value within this attribute causes all other attributes in the new group to be considered overridden as well. Consequently, the values of these attributes may be nullified during the subsequent execution of the certification process, unless override values have previously been defined. To minimize potential negative impacts on data consistency and accuracy, designers should consider moving attributes with pre-existing overridden values to a group that does not include any other attributes.

Bug fixes

  • MDM-14989: When attempting to open a dashboard application from the welcome page, a blank screen is displayed in the browser.

  • MDM-14997: Attempting to open the Configuration module results in an unexpected error in the browser, along with a 404 not found message when opening the Dashboard Builder or Workflow Builder.

  • MDM-15025: Attempting to view profiles associated with datasources in the Discovery module or to open a profiling application from the welcome page results in an unexpected application error and a 404 not found message.

  • MDM-15066: When creating a new user and enabling the Ask for a password change at the next login setting, subsequent user attempts to log in result in an application error with a 404 not found message.

Version 2024.1.0

Known issues

  • MDM-7254: With PostgreSQL, the application server hosting the Semarchy xDM instance and the database server must be in the same timezone to avoid inconsistent dates in the integration loads.

  • MDM-7917: A boolean field default value is set to null when creating a record and to false when editing it.

  • MDM-7949: A model with a physical column name using a reserved name raises a validation error but can be deployed.

  • MDM-8034: When stopping Semarchy xDM using the Tomcat Manager, a Severe message is logged with org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks…​. This message is a known Tomcat issue that can be ignored.

  • MDM-8703: Data import from Excel does not properly handle dates before February 28, 1900.

  • MDM-8977: Changing a reference to a fuzzy-matching entity while authoring a master record is not taken into account.

  • MDM-9184: Existing FS_ and FP_ reference column values in a basic entity are overwritten by an enricher when this record is modified by the API or by authoring.

  • MDM-10193: When a record in an entity is updated, records with the same ID in inheriting entities are also updated.

  • MDM-10269: Using the SEM_NUMBER_TO_CHAR function with a string argument does not raise a model validation error but fails with an invalid number error in the log.

  • MDM-10319: Authoring the same record in two different browser windows makes the second authoring fail with a java.lang.IllegalStateException: Current Step (products) should be a form step error.

  • MDM-10738: Duplicate flexElement error occurs at runtime when using a transition named like a form element (e.g., "field", "container") in a business view.

  • MDM-12415: Long-integer values higher than 9223372036854700 raise errors when authoring (Invalid ID for data type (longInteger)) or browsing (<entity name> not found) records.

  • MDM-13099: When moving an attribute from one group to another governed by a different survivorship rule, any overridden value within this attribute causes all other attributes in the new group to be considered overridden as well. Consequently, the values of these attributes may be nullified during the subsequent execution of the certification process, unless override values have previously been defined. To minimize potential negative impacts on data consistency and accuracy, designers should consider moving attributes with pre-existing overridden values to a group that does not include any other attributes.

New features

  • MDM-9782: When browsing records with errors, the Error Details tab in a business view now displays meaningful and detailed error messages for each listed error—​in addition to the constraint type and constraint name for built-in errors—​and provides the ability to switch between list and table views for comprehensive error details.

  • MDM-9988: SEM_CAST_NUMERIC SemQL function available on SQL Server for converting numeric strings to integers.

  • MDM-9993: When querying error views using the REST API, the API response now includes a ConstraintErrorMessage attribute that contains either the customized or default error message associated with the error records.

  • MDM-12576: SEM_CAST_INTEGER SemQL function available on PostgreSQL for converting numeric types to integers.

  • MDM-13980: The REST API documentation explicitly specifies golden record IDs as 64-bit integers instead of numbers.

  • MDM-13998: In the workflow administration view, a new Workflow Instance Details panel presents general information on a selected workflow instance (including its start, end, and due date, progress status, initiator’s name, and more) to complement the existing step details panel.

  • MDM-14173: When executing invalid SQL queries in the Dashboard Builder, the query editor prompts detailed error messages to help pinpoint and rectify syntax mistakes or logical errors.

  • MDM-14217: The REST API documentation provides more specific summaries and descriptions for each endpoint, incorporating details such as entity name, for improved clarity in external API management tools.

  • MDM-14303: In the xDM official Docker image, the default Log4j configuration has been updated with the addition of the Console appender to the root logger, allowing platform administrators to retrieve logs effectively through the command line in Kubernetes environments.

  • MDM-14536: The REST API documentation has been updated to better handle large models, now organizing data location sections by entity. This improvement introduces a submenu for entity selection with enhanced search functionality and facilitates YAML downloads for specific entities.

  • MDM-14585: The toast message that is displayed after closing a duplicate manager now acknowledges situations where no changes have been made.

  • MDM-14611: The contrast ratio in read-only fields has been heightened, enhancing text visibility across all authoring fields in the user interface of xDM applications.

  • MDM-14729: The date picker in workflow startup and transition dialogs now restricts users from choosing a date prior to the current date when defining the due date.

  • MDM-14738: Support added for the Microsoft Build of OpenJDK.

Breaking changes

  • MDM-14822: An allowlist mechanism now restricts the classes permitted in Groovy conditions or scripts to enhance security within Semarchy xDM. To modify the default allowlist without compromising security, platform administrators are invited to contact our Technical Support team.

Bug fixes

  • MDM-12278: During a Review and Confirm Matches operation using a duplicate manager, the table view presenting details of the master records suggestions for the golden record is not scrollable—​both horizontally and vertically—​unless the scrollbars are clicked first.

  • MDM-13911: Missing translation keys result in java.util.MissingResourceException errors in the PDE log when exporting model documentation.

  • MDM-14269: Within the Application Builder, selecting the Directed Graph mode and clicking Auto Layout in the diagram of a model that contains a self-referencing inherited entity throws a java.lang.RuntimeException: Cycle detected in graph error.

  • MDM-14297: Processing entities inheriting from the same parent entity in separate queues may cause the certification jobs to produce discrepancies or errors in the data, as jobs performed on one child entity can incorrectly update the batch IDs of another child entity’s records and disrupt job completion.

  • MDM-14338: Configuring the Authoring Mode property of a form field using a SemQL expression while the Visible property is unselected causes a display error on authoring forms.

  • MDM-14348: In a workflow startup or transition dialog, users can input workflow instance descriptions without length limits despite the 128-character database restriction. This results in a value too long for type character varying(128) error after closing the dialog.

  • MDM-14386: Values copied from dashboard charts and from the Users and Roles tables in Configuration using the Control+C (or Command+C on macOS) shortcut are not available in the clipboard for pasting.

  • MDM-14388: In the Application Builder, configuring the value of a new form field with a primary-key or list-of-values attribute returns a non-blocking error with the following message: java.lang.NullPointerException.

  • MDM-14443: When using the responsePayloadRecordsExpressions parameter in a model’s REST API, the expected name for the defined expression is not reflected in the response payload; instead, the calculation expression itself is used as the attribute name.

  • MDM-14613: After customizing the header border color in an xDM application, the default color theme reverts upon refreshing the application.

  • MDM-14614: An ix_integ_load_loadid native index has been added to the loadid column in the MTA_INTEG_LOAD repository table to alleviate contention and prevent deadlocks in SQL Server.

  • MDM-14623: Security enhancement: authentication mechanism.

  • MDM-14625: Security enhancement: login process.

  • MDM-14626: Security enhancement: error page content.

  • MDM-14627: Security enhancement: logout page URL.

  • MDM-14629: Security enhancement: configuration module.

  • MDM-14630: Security enhancement: Docker Compose sample file.

  • MDM-14641: When setting a date and time in an authoring form using a date-and-time picker, the value displayed in the field does not reflect the current user’s timezone settings.

  • MDM-14642: Using invalid decimal values such as ,,, or . in an authoring form field does not trigger an error message, and results in the attribute value being processed as null.

  • MDM-14643: When using a SemQL condition to configure the Visible property of a section within a horizontal form container, the label of the attributes included in the browsing form section is not displayed as defined by the Position form field property.

  • MDM-14649: When exporting records, the Attribute Name and Label option can be selected in the advanced formatting options so that attribute names and labels are concatenated in the column headers of the exported file.

  • MDM-14679: When generating application components and opting for the default display card instead of creating a new one, an extra display card is systematically generated.

  • MDM-14685: Workflow user tasks based on entities having a UUID primary key cause the My Tasks screen to turn gray and unresponsive.

  • MDM-14686: External application links configured in object form fields cannot be previewed in xDM’s applications and fail to display with the following error message: Access to fetch at '<target URL>' from origin '<origin URL>' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

  • MDM-14708: Users who have been assigned a workflow administrator role cannot access the workflow administration view in My Tasks unless they have been granted full access to the model.

  • MDM-14717: Typo in a label in the French localized version.

  • MDM-14719: Third-party library upgrade: Apache Commons Compress.

  • MDM-14734: Multi-step steppers do not hinder users from entering pre-existing primary keys while creating new records, resulting in the potential overwrite of existing golden records. This behavior exclusively affects golden records created after opening such steppers.

  • MDM-14754: Embedded Javascript content—​such as Google Maps—​is not displayed in an object form field when initially opening a browsing form, and appears after navigating to another tab and returning.

  • MDM-14756: In the Datasources view of xDM Configuration, the chips in the Datasource Usages column are not horizontally aligned with the content of the adjacent columns.

  • MDM-14757: In the Workflow Builder, the value of a drop-down menu cannot be reset to None after a different value has been selected.

  • MDM-14776: When nullifying child references to a record from a self-referencing entity that is soft-deleted in a load, the certification process may revive the record previously flagged for deletion. This behavior occurs regardless of the child records' level in the record hierarchy. To fix this issue, the model must be redeployed after applying this patch.

  • MDM-14791: Resizing columns in the My Tasks view causes the application’s web page to become unresponsive.

  • MDM-14793: Third-party library upgrade: Azure Identity.

  • MDM-14794: Third-party component upgrade: Apache Tomcat (xDM preconfigured).

  • MDM-14823: Security enhancement: notification server.

  • MDM-14834: Workflow user tasks that have been configured to display task items from a collection containing a binary attribute column cause the My Tasks view to throw an exception without showing the list of task items.

  • MDM-14870: Third-party library upgrade: io.projectreactor.netty.

  • MDM-14898: File uploads in the Application Builder and in the Configuration module fail in AWS environments, specifically with Semarchy xDM versions containing the MDM-14622 fix.

  • MDM-14907: When using a lookup SemQL expression in the Condition on Selection property of a workflow start event, triggering it prompts an unexpected error message and writes an error to the PDE log (com.semarchy.datahub.runtime.workflow.service.WorkflowEngineException: This attribute builder should not be called), disrupting workflow execution.

  • MDM-14908: Third-party library upgrade: io.projectreactor.netty.

  • MDM-14909: Third-party module upgrade: Apache Tomcat (xDM preconfigured).