Entities

Overview

Entities are the key components of the logical modeling. Entities are not database tables. They represent Business Entities of the domain being implemented in the Data Hub. Example of entities: Customers, Contacts, Parties, etc.

Attributes

Entities have a set of properties, called Attributes. Each attribute has a given datatype.

For example, the Contact entity may have the following attributes:

  • FirstName and LastName: Simple attributes using the user-defined type called GenericNameType

  • Comments: Simple attribute using the built-in type LongText.

  • Gender: Simple attributes based on the GenderLov list of values.

  • Address: Complex Attribute using the GeocodedAddress complex type.

Entity types

Each entity has a given entity type. This type expresses the entity capabilities for match/merge and authoring. Entity types are:

  • Basic: Basic entities do not support match and merge of records. They assume that data comes from a single datasource. This entity type is suitable for entities for which data is authored (or imported) exclusively in the hub, or for simple reference data entities.

  • ID Matched (formerly known as UDPK): ID-matched entities support match and merge of records. They assume that data comes from many data sources, and similar records share an identifier common to all sources. Records in entities using ID Matching are matched if they have the same ID and then merged into golden records. This entity type is well suited when there is a true unique identifier for all the applications communicating with the MDM hub.

  • Fuzzy Matched (formerly known as SDPK): Fuzzy matched entities support match and merge of records. They assume that records comes from many data sources that do not share a common identifier. Records need to be matched using their data content. Records in entities using Fuzzy Matching are matched using a set of match rules defined in a Matcher.

The choice of an Entity Type is important. Please take into account the following differentiators when creating an entity.

Basic entity

  • Basic means that this entity does not support match and merge.

  • With this type of entity, you must assume that all data comes from a single (de-duplicated) source or is authored exclusively in the MDM hub.

  • When authoring or loading data in a basic entity, you simply overwrite the existing golden record with the new data, possibly keeping track of the changes. There is no notion of multiple master records merging into a golden record.

  • This type of entity is particularly suitable for reference data or classification data, which are typically managed only in the hub.

  • Due to the simple nature of these entities, the certification process is simpler and faster for basic entity records than for ID or fuzzy matched entity records.

  • A Matcher can be defined for the entity, for detecting potential duplicates when manually creating records in the hub.

Use Basic Entities for data that only exist in the hub, or for data coming from a single datasource into which there are no duplicates.

ID-matched entity

  • ID Matching assumes that the multiple applications providing data to this entity share a common ID. This ID can be used as the unique identifier, even for the golden records.

  • This ID is stored into a single attribute which will be the golden data Primary Key. If the ID in the information system is composed of several columns, you must concatenate these values into the PK column.

  • As this ID is common to all systems, similar records are always matched using the ID. A Survivorship Rule defines how they are consolidated into a single golden records, and how users can override the consolidated values.

  • Although ID matching is faster than fuzzy matching, it still requires consolidating multiple records into a single golden record. The certification process for ID matched records is slower than for basic entity records.

  • A Matcher can be defined for the entity, for detecting potential duplicates when manually creating records in the hub.

  • When authoring an ID matched entity record in a stepper, you may create a new record that only exists in the hub, or override the consolidated values. The Survivorship Rule defines how data creation or override takes place for attributes.

Use ID Matching when you need to match and merge records from various sources and have a truly unique and shared identifier for all these sources.

Fuzzy-matched entity

  • Fuzzy Matching means that applications in the enterprise have different IDs, and Semarchy xDM needs to generate a unique identifier (Primary Key - PK) for the golden records. This PK can be either a sequence or a Unique ID (UUID).

  • Similar records may exist various systems, representing the same master data, with different IDs. These similar records must be matched using fuzzy matching methods that compare their content.

  • A Matcher defines how similar master records are matched. A Survivorship Rule defines how they are consolidated into a single golden records, and how users can override the consolidated values.

  • Duplicate Managers define the user interfaces into which users review, merge or split groups of matching records.

  • Due to the complex processing involved with fuzzy-matching and then merging records, the certification process for fuzzy-matched records is slower than for ID-matched or basic entity records.

  • When authoring a fuzzy-matched entity record in a stepper, you may create a new record that only exists in the hub, or override the consolidated values. The Survivorship Rule defines how data creation or override takes place for attributes.

Use Fuzzy Matching when you need to match and merge records from various sources and do not have a shared identifier for all these sources.

ID generation

The entity type impacts the method used for generating the record IDs:

  • Basic Entities: The Golden Record Primary Key is the ID sent or authored when creating a record. When creating records, this ID may be:

    • Manually provided by users

    • Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.

  • ID Matched Entities: The Golden Record Primary Key is also the ID that exists in the source systems. When creating source/master records, this ID may be:

    • Manually provided by users.

    • Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.

  • Fuzzy Matched Entities: The Golden Record Primary Key is managed and always generated by the system, using a Sequence or UUID.
    When creating source/master records, this source/master ID may be:

    • Manually entered by users

    • Automatically generated using a Sequence, Universally Unique Identifier generator (UUID) or a SemQL expression.

When using a Sequence for ID generation, you must take into account external source records, that is records integrated from publishers or records that you will import using applications.
These records may arrive with ID values above the current sequence value. Such IDs will collide with future records created in the hub, causing effects such as unexpected matching (ID-matched entities) or record updates (basic entities).

To separate the records created into the hub with a sequence ID from external records, it is recommended to set the sequence Start With value above the range of IDs used by possible records.

Note that the interactive import available in applications will prevent importing records when their ID is above the Start With value.

An ID generated with a SemQL expression is immutable. This ID will not change after the initial record creation even if the value of the attributes used in the expression change. Such an ID is created when a record form is saved for the first time, for example when a record is imported from a file containing no IDs.

References

Entities are related using Reference Relationships. A reference relationship defines a relationship between two entities. For example, an Employee is related to a CostCenter by the EmployeeHasCostCenter relationship.

Data quality rules

Data quality rules are created in the design of an entity. These constraints include:

  • Mandatory columns

  • List of Values range check

  • Unique Key

  • Record level Validations.

  • Reference Relationships

These constraints are checked on the source records and the consolidated records as part of the certification process. They can also be checked to enforce data quality in data authoring.