Logical Modeling |
Previous
|
|
Next
|
Introduction to the Semarchy Workbench |
|
Integration Process Design |
Logical Modeling
Logical modeling allows defining the business entities that make up the model.
Introduction to Logical Modeling
In Semarchy Convergence for MDM, modeling is performed at the logical level. You do not design physical objects such as tables in a model, but logical business entities independently from their implementation.
Logical Model Objects
Logical modeling includes creating the following objects:
- Customized types such as
User-Defined Types,
Complex Types and
List of Values reused across the model.
-
Entities representing the business objects with their
Attributes.
-
References Relationships linking the entities of the model, for example to build hierarchies.
- Constraints such as
Validations,
Unique Keys and
Mandatory attributes.
-
Display Names,
Table Views,
Table Views and
Business Objects defining how the entities appear to the data stewards and end-users.
In the modeling phase, integration processes objects such as
Enrichers,
Matchers and
Consolidators are also created. Creating these objects is described in the
Integration Process Design chapter. After completing the modeling phase, you can create
Applications to access the data stored in your model.
Objects Naming Convention
When designing a logical model, it is necessary to enforce a naming convention to guarantee a good readability of the model and a clean implementation.
There are several names and labels used in the objects in Semarchy Convergence for MDM:
-
Internal Names (Also called
Names) are used mainly by the Model designers and are unique in the model. They can only contain alphanumeric characters, underscores and must start with a letter.
-
Physical Names and
Prefixes are used to create the objects in the database corresponding to the logical object. These can only contain uppercase characters and underscores.
-
Labels and
Descriptions are visible to the users (end-users and data stewards) consuming data through the UI. These user-friendly labels and descriptions can be fixed at later stages in the design. They are externalized and can be localized (translated) in the platform.
The following tips should be used for naming objects:
- Use meaningful Internal Names. For example, reference relationships should all be named after the pattern
<entity name><relation verb><entity name> , like
CustomerHasAccountManager .
- Do not try to shorten internal names excessively. They may become meaningless. For example, using
CustAccMgr instead of
CustomerHasAccountManager is not advised .
- Use the
CamelCase for internal names as it enables the use of the Auto fill feature. For example,
ContactBelongsToCustomer,
GeocodedAddressType.
- Define team naming conventions that accelerate object type identification. For example, types and list of values can be post-fixed with their type such as
GeocodedAddressType,
GenderLOV.
- Define user-friendly Labels and Descriptions. Internal Names are for the model designers, but labels and descriptions are for end users.
Model Validation
A model may be valid or invalid. An invalid model is a model that contains a number of design
errors. When a model is invalid, you cannot deploy it, and you cannot close this model edition. Model validation detects errors or missing elements in the model. For example, entities with no consolidation strategies are detected.
To validate the model:
- In the
Model Edition view of the
Model Design Perspective, select the model node at the root of the tree. You can alternately select on entity to validate only this entity.
- Right-click and select
Validate.
- The validation process starts. At the end of the process, the list of issues (errors and warnings) is displayed in the
Validation Report view. You can click an error or waning to open the object causing this issue.
Tip: It is recommended to regularly run the validation on the model or on specific entities. Validation may guide you in the process of designing a model. You can perform regular validations to assess how complete the model really is, and you need to pass model validation before deploying or closing a model edition.
Generating the Model Documentation
When a model is complete or under development, it is possible to generate a documentation set for this model.
This documentation set may contain the following documents:
- The
Logical Model Documentation, which includes a description of the logical model and the rules involved in the integration processes.
- The
Applications Documentation, which includes a description of the applications of the model, their related components (views, business objects, etc.) as well as their workflows.
- The
Physical Model Documentation, which includes a description of the physical tables generated in the data location when the model is deployed. This document is a physical model reference document for the integration developers.
The documentation is generated in HTML format and supports hyperlink navigation within and between documents.
To generate the model documentation:
- In the
Model Edition view of the
Model Design Perspective, select the model node at the root of the tree.
- Right-click and select
Export Model Documentation.
- In the
Model Documentation Export dialog, select the documents to generate.
- Select the appropriate
Encoding and
Locale for the exported documentation. The local defines the language of the generated documentation.
- Click the
Download link to download the documentation.
The documentation is exported in a zip file containing the selected documents.
Note: It is possible to export the model documentation only for a valid model.
Types
Several types can be used in the Semarchy Convergence for MDM models:
-
Built-in Types are part of the platform. For example: string, integer, etc.
-
List of Values (LOVs) are a user-defined list of string codes and labels. For example:
Gender (M:Male, F:Female),
VendorStatus (OK:Active, KO:Inactive, HO:Hold).
-
User Defined Types are user restriction on a built-in type. For example the
GenericNameType type can be defined as a
String(80) and the
ZipCodeType can be used as an alias for
Decimal (5,0).
-
Complex Types are a customized composite type made of several
Definition Attributes using Built-in Type, User-Defined Type or a List of Values. For example, an
Address complex type has the following definition attributes:
Street Number,
Street Name,
Zip Code,
City Name and
Country.
All these type as the user-defined are reused across the entire model.
Tip: A list of values, user-defined or complex type is designed in the model and can be used across the entire model. Changes performed to such a type impact the entities and attributes using this type. To list the attributes using a type and analyze the impact of changing a type, open the editor for this type and then select the
Used in item in the left sidebar.
Built-in Types
Built-in types are provided out of the box in the platform.
Built-in types include:
-
Numeric Types:
- ByteInteger: 8 bytes signed. Range [-128 to 127]
- ShortInteger: 16 bytes signed. Range [-32,768 to -32,767]
- Integer: 32 bytes signed. Range [-2^32 to (2^32-1)]
- LongInteger: 64 bytes signed. Range [-2^64 to (2^64-1)]
- Decimal: Number(Precision,Scale), where Precision in [1-38] and Scale in [-84-127]
-
Text Types:
- String: Length smaller than 4000 characters.
- LongText: No size limit – Translated to a CLOB in the database
-
Date Types:
- Datetime: Date and Time (to the Second)
- Timestamp: Date and Time (to fractional digits of a Second)
-
Binary Types:
- Binary: Store any type of binary content (image, document, movie, etc.) with no size limit – Translated to a BLOB in the database
-
Misc. Types:
- UUID: 16 bytes Global Unique ID
- Boolean: 1 character containing either ‘1’ (true) or ‘0’ (false).
List of Values
List of Values (LOVs) are a user-defined list of code and label pairs.
They are limited to 1,000 entries and can be imported from a Microsoft Excel Spreadsheet.
Examples:
-
Gender (M:Male, F:Female)
-
VendorStatus (OK:Active, KO:Inactive, HO:Hold)
Info: Lists of values are limited to 1,000 entries. If a list of value needs to contain more than 1,000 entries, you should consider implementing in the form of an entity instead.
To create a list of values:
- Right-click the
List of Values node and select
Add List of Values.... The
Create New List of Values wizard opens.
- In the
Create New List of Values wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the list of values.
-
Label: User-friendly label in this field. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Length: Length of the code for the LOV.
- Click
Finish to close the wizard. The
List of Values editor opens.
- In the
Description field, optionally enter a description for the user-defined type.
- Add values to the list using the following process:
- In the
Values section, click the
Add Value button. The
Create New LOV Value dialog appears.
- In this dialog, enter the following values:
-
Code: Code of the LOV value. This code is the value stored in an entity attribute.
-
Label: User-friendly label displayed for a field having this value.
-
Description: Long description of this value.
- Click
Finish to close the dialog.
- Repeat the previous operations to add the values. You can select a line in the list of value and click the
Delete button to delete this line. Multiple line selection is also possible.
- Press
CTRL+S to save the editor.
- Close the editor.
List of values can be entered manually as described above and can be translated as described in the Localization section of the
Working with Model Display chapter.
In addition, you can also import values or translated values from a Microsoft Excel Spreadsheet.
This spreadsheet must contain only one sheet with three columns containing the Code, Label and Description values. Note that the first line of the spreadsheet will be ignored in the import process.
To import a list of values from an excel spreadsheet:
- Open the editor for the list of value.
- Expand the
Values section.
- In the
Values section, click the
Import Values button. The
Import LOV Values wizard appears.
- Use the
Open button to select a Microsoft Excel spreadsheet.
- Choose the type of import:
- Select
Import codes, default labels and descriptions to simply import a list of codes, default labels and descriptions.
- Select
Import translated labels and description for the following locale then select a locale from the list to import translated labels and descriptions in a given language. Click the
Merge option to only update existing and insert new translations. If you uncheck this box, the entire translation is replaced with the content of the Excel file. Entries not existing in the spreadsheet are removed.
- Click
Next . The changes to perform are computed and a report of object changes is displayed.
- Click Finish to perform the import. The
Import LOV wizard closes.
- Press
CTRL+S to save the editor.
- Close the editor.
User-Defined Types
User-Defined Types (UDTs) are user restriction on a built-in type. They can be used as an alias to a built-in type, restricted to a given length/precision.
Examples:
-
GenericNameType type can be defined as a
String(80)
-
ZipCodeType can be used as an alias for
Decimal(5,0).
To create a user-defined type:
- Right-click the
User-defined Types node and select
Add User-defined Type.... The
Create New User-defined Type wizard opens.
- In the
Create New User-defined Type wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the user-defined type
-
Label: User-friendly label in this field. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Built-in Type: Select a type from the list.
-
Length,
Precision,
Scale. Size for this user defined type. The fields available depend on the built-in type selected. For example a String built-in type will only allow entering a Length.
- Click
Finish to close the wizard. The
User-defined Type editor opens.
- In the
Description field, optionally enter a description for the user define type.
- Press
CTRL+S to save the editor.
- Close the editor.
Complex Types
Complex Types are a customized composite type made of several Definition Attributes using Built-in Type, User-Defined Type or a List of Values.
For example, an
Address complex type has the following definition attributes:
Street Number,
Street Name,
Zip Code,
City Name and
Country.
To create a complex type:
- Right-click the
Complex Types node and select
Add Complex Type.... The
Create New Complex Type wizard opens.
- In the
Create New Complex Type wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
- Click
Finish to close the wizard. The
Complex Type editor opens.
- In the
Description field, optionally enter a description for the complex type.
- Select the
Definition Attributes item in the editor sidebar.
- Repeat the following steps to add definition attributes to this complex type:
- Select the
Add Definition Attribute... button. The
Create New Definition Attribute wizard opens.
- In the
Create New Definition Attribute wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Physical Column Name: Name of the physical column containing the values for this attribute. This column name is prefixed with the value of the
Physical Prefix specified on the entity complex attribute of this complex type.
-
Type: List of values, built-in or user-defined type of this complex attribute.
-
Length,
Precision,
Scale. Size for this definition attribute. The fields available depend on the built-in type selected. For example a
String built-in type will only allow entering a Length. If a list of values or a user-defined type was selected, these values cannot be changed.
-
Mandatory: Check this box to make this definition attribute mandatory when the complex type is checked for mandatory values.
- Click
Finish to close the wizard. The new definition attribute appears in the list. You can double-click the attribute in the list to edit it further and edit its advanced properties (see below).
- Press
CTRL+S to save the
Complex Type editor.
- Close the editor.
A complex type has the following advanced properties that impact its behavior:
-
Mandatory: When an entity attribute is checked for mandatory values, and this attribute uses a complex type, each of the definition attributes of this complex type with the
mandatory option selected are checked.
-
Searchable: This option defines whether this attribute is used for searching.
-
Translated: Options reserved for a future use.
-
Multi-Valued: This option applies to definition attributes having the type list of values. Checking this box allows the definition attribute to receive several codes in the list of values, separated by the
Value Separator provided. For example, a multi-valued
Diplomas field can receive the
DM, DP, DPM codes meaning that that contact is
Doctor of Medicine,
Pharmacy and
Preventive Medicine.
Entities
Entities are the key components of the logical modeling. They are not database tables, but they represent
Business Entities of the domain being implemented in the MDM Project. Example of entities:
Customers,
Contacts,
Parties, etc.
Entity Characteristics
Entities have several key characteristics. They are made of
Attributes, have a
Matching Behavior and
References. They also support
Inheritance.
Attributes
Entities have a set of properties, called
Attributes. These attributes can be either:
-
Simple Attributes using a built-in types, user-defined types or a list of values created in the model.
-
Complex Attributes using complex types created in the model.
For example, the
Contact entity may have the following attributes:
-
FirstName and
LastName: Simple attributes using the user-defined type called
GenericNameType
-
Comments: Simple attribute using the built-in type
LongText.
-
Gender: Simple attributes based on the
GenderLov list of values.
-
Address: Complex Attribute using the
GeocodedAddress complex type.
Matching Behavior
Each entity has a given a matching behavior. This matching behavior expresses how similar instances (duplicates) of this entity are detected:
-
ID Matching (formerly known as UDPK): Records in entities using ID Matching are matched if they have the same ID. This matching behavior is well suited when there is a true unique identifier for all the applications communicating with the MDM hub and for simple Data Entry use cases.
-
Fuzzy Matching (formerly known as SDPK): Records in entities using Fuzzy Matching are matched using a SemQL condition defined in a matcher.
Important: The choice of a Matching Behavior is important. Please take into account the following differentiators when creating an entity.
ID Matching
- ID Matching means that all applications in the enterprise “share” a common ID. It may be a Customer ID, an SSID, etc.. This ID can be used as the unique identifier for the golden records.
- This ID is stored into a single attribute which will be the golden data Primary Key. If the ID in the information system is composed of several columns, you must concatenate these values into the PK column.
- As this ID is common to all systems,
matching is always be made using this ID.
- A Matcher can be defined for the entity, for detecting potential duplicates when manually creating records in the hub via a data entry workflow.
Use ID Matching only when there is a true unique identifier for all the applications communicating with the MDM Hub, or for simple data entry use cases.
Fuzzy Matchings
- Fuzzy Matching means that applications in the enterprise have different IDs, and Semarchy Convergence for MDM needs to generate a unique identifier (Primary Key – PK) for the golden records. This PK can be either a sequence or a Unique ID (UUID).
- Similar records may exist in the various systems, representing the same master data. These similar records must be matched using fuzzy matching methods that compare their content.
- A Matcher must be defined in such entity to describe how source records are matched as similar records to be consolidated into golden records.
Use Fuzzy Matching only when you do not have a “shared” identifier for all systems, or when you want to perform fuzzy matching and consolidation on the source data.
ID Generation
The matching behavior impacts the method used for generating the values for the Golden Record Primary Key:
-
ID Matching Entities: The Golden Record Primary Key is also the ID that exists in the source systems. It may be generated in the MDM hub only when creating new records in data entry workflows. In this case, the ID may be generated either manually (the user enters it in the form), or automatically using a Sequence or a Universally Unique Identifier generator.
-
Fuzzy Matching Entities: The Golden Record Primary Key is managed and always generated by the system, using a Sequence or a Universally Unique Identifier generator. When creating records in data entry workflows, a Source ID is automatically generated and can be manually modified by the user.
Note: When generating IDs automatically using a Sequence in data entry forms for an ID Matching entity, you must take into account records pushed by other publishers (using for example a data integration tool). These publishers may use the same IDs for the same entity, and in this case the records will match by ID. If you want to separate records entered manually from other publishers' records and avoid unexpected matching, configure your sequence using the
Start With option to start beyond the range of IDs used by the other publishers.
References
Entities are related using Reference Relationships. A reference relationship defines a relation between two entities. For example, an
Employee is related to a
CostCenter by a
EmployeeHasCostCenter relation.
Constraints
Data quality rules are created in the design of an entity. These constraints include:
-
Mandatory columns
-
List of Values range check
-
Unique Key
- Record level
Validations.
-
Reference Relationships
These constraints are checked on the source records and the consolidated records as part of the integration process. They can also be checked to enforce data quality in data entry workflows .
Inheritance
Entities can extend other entities (Inheritance). An entity (child) can be based on another entity (parent).
For example, the
PersonParty and
OrganizationParty entities inherit from the
Party entity.
They share all the attributes of their parent but have specificities.
When inheritance is used:
- The child entity inherits the following elements: Attributes, Unique Keys, Validations, Enrichers, References and Display Name.
- Matchers and Consolidators are not inherited
- It is not possible to modify the matching behavior. The child inherits from the parent’s behavior.
- It is possible to add elements to the child entity: Attributes, Unique Keys, Validations, Enrichers and References.
- The display name defined for the parent entity can be amended in a child entity by appending additional attributes from the child. The separator can be changed.
When using inheritance, the underlying physical tables generated for the child entities and parent entity are the same. They contain a superset of all the attributes in the cluster of entities.
Display Options
Entities also have display options, including:
- A
Display Name defining how the entity is display in compact format.
-
Translations to display the entities information in a client’s locale.
These display characteristics are detailed in the
Working with Model Display chapter.
Integration Rules
In addition to the display characteristics, an entity is designed with integration rules describing how master data is created and certified from the source data published by the source applications.
These characteristics are detailed in the
Integration Process Design chapter.
Creating an Entity
Creating a New Entity
To create an entity:
- Right-click the
Entities node and select
Add Entity.... The
Create New Entity wizard opens.
- In the
Create New Entity wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Plural Label: User-friendly label for this entity when referring to several instances. The value for the plural label is automatically generated from the Label value and can be optionally modified.
-
Extends Entity: Select an entity that you want to extend in the context of
Inheritance. Leave this field to an empty value if this entity does not extend an existing one.
- If the entity extends an existing one, the remaining options cannot be changed as they are inherited. Click
Finish to close the wizard and then press
CTRL+S to save the editor.
- If you do not use inheritance, proceed to the next step.
-
Physical Table Name: This name is used to name the physical table that will be created to store information about this entity. For example, if the physical table is
CUSTOMER, then the golden data is stored in a
GD_CUSTOMER table.
-
Matching Behavior: Select the matching behavior for this entity.
ID Matching or
Fuzzy Matching. See
Matching Behavior for more information.
- Click the
Next button.
- In the
Primary Key Attribute screen, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the primary key attribute.
-
Label: User-friendly label for this primary key attribute.
-
Physical Column Name: This name is used to name the physical column that will be created to store values for this attribute.
-
ID Generation: Select the method for generating the primary key :
-
Sequence: Use this option to generate the ID as a sequential number. You can specify a startup value for the sequence in the
Starts With field.
-
UUID: Use this option to generate the ID as a Universally Unique Identifier.
-
Manual: Use this option to type in the ID manually. This option is only possible for
ID Matching entities. When this option is selected, you can choose the
Type and
Length for the primary key attribute.
- Click
Finish to close the wizard. The
Entity editor opens.
- In the
Description field, optionally enter a description for this entity.
- Press
CTRL+S to save the editor.
When an entity is created, it contains no attributes. Simple Attributes and Complex Attributes can be added to the entity now.
Note: You cannot modify the entity matching behavior, primary key or inheritance from the
Entity editor. To change such key properties of the entity after creating it, you must use the
Alter Entity option. See
Altering an Entity for more information.
Adding a Simple Attribute
To add a simple attribute:
- Expand the entity node, right-click the
Attributes node and select
Add Simple Attribute.... The
Create New Simple Attribute wizard opens.
- In the
Create New Simple Attribute wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Physical Column Name: This name is used to name the physical column that will be created to store values for this attribute.
-
Type: Select the type of the attribute. This type can be a built-in type, a user-defined type or a list of values.
-
Length,
Precision,
Scale. Size for this definition attribute. The fields available depend on the built-in type selected. For example a
String built-in type will only allow entering a Length. If a list of values or a user-defined type was selected, these values cannot be changed.
-
Mandatory: Check this box to make this attribute mandatory.
-
Mandatory Validation Scope: This option is available if the
Mandatory option was selected. Select whether this attribute should be checked for null values pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
-
LOV Validation Scope: This option is available if the selected type is a list of values. It defines whether the attribute’s value should be checked against the codes listed in the LOV. For more information, refer to the
Integration Process Design chapter.
- Click
Finish to close the wizard. The
Simple Attribute editor opens.
- In the
Description field, optionally enter a description for the simple attribute.
- Press
CTRL+S to save the editor.
Adding a Complex Attribute
To add a complex attribute:
- Expand the entity node, right-click the
Attributes node and select
Add Complex Attribute.... The
Create New Complex Attribute wizard opens.
- In the
Create New Complex Attribute wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Physical Prefix: This name is used to prefix the physical column that will be created to store values for this complex attribute. The column name is
<Physical Prefix>__<Definition Attribute Physical Column Name>
-
Complex Type: Select the complex type of the attribute.
-
Mandatory Validation Scope: This option is available if the
Mandatory option was selected for at least one of the definition attributes of the selected complex type. Select whether the mandatory definition attributes of the complex type should be checked for null values pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
-
LOV Validation Scope: This option is available if at least one of the definition attributes of the complex type is a list of values. It defines whether the definition attribute’s value should be checked against the codes listed in the LOV. For more information, refer to the
Integration Process Design chapter.
- Click
Finish to close the wizard. The
Complex Attribute editor opens.
- In the
Description field, optionally enter a description for the complex attribute.
- Press
CTRL+S to save the editor.
Working with Attributes
It is possible to edit, order or delete attributes in an entity from the
Attributes list in the entity editor.
To order the attributes in an entity:
- Open the editor for the entity.
- Select the
Attributes item in the sidebar.
- Select an attribute in the Attributes list and use the
Move Up and
Move Down buttons to order this attribute in the list.
Note: With entities inheriting attributes from a parent in the context of
Inheritance and have additional (not inherited) attributes, you can perform this ordering from the
Inherited Attributes (All) section. Note that it is not possible to order additional attributes before the inherited attributes.
To delete attributes from an entity:
- Open the editor for the entity.
- Select the
Attributes item in the sidebar.
- Use the
Delete buttons to remove the attribute from the list.
- Click
OK in the confirmation dialog.
Note: With entities inheriting attributes from a parent in the context of
Inheritance and have additional (not inherited) attributes, you can delete attributes from the
Inherited Attributes (All) section.
Important: Deleting an inherited attribute on a child entity removes it from the parent entity, and by extension from all the child entities inheriting this attribute from the parent.
Altering an Entity
Altering an entity allows modifying the key properties of an entity, including its matching behavior, inheritance and primary key attribute.
To alter an entity:
- Select the entity node, right-click and select
Alter Entity. The
Modify Entity wizard opens.
- In the wizard, modify the properties of the entity using the same process used for
creating an entity
Reference Relationships
Reference Relationships functionally relate two existing entities. One of them is the
referenced, and one is
referencing.
For example:
- In an
EmployeeHasCostCenter reference, the referenced entity is
CostCenter, the referencing entity is
Employee.
- In a
CostCenterHasParentCostCenter reference, the referenced entity is
CostCenter, the referencing entity is also
CostCenter. This is a
self-reference.
They are used for:
-
Displaying Hierarchies: For example, a hierarchy of cost centers is built as
CostCenter has a self-reference called
CostCenterHasParentCostCenter.
-
Displaying Lists of Child Entities to navigate the list of entities referring a master entity. For example,
a CostCenter can appear with a list of
Employee belonging to this cost center according to the
EmployeeHasCostCenter relation.
-
Display Links to navigate to the parent entity in a similar relation. The
Employee entity will list a link to the
CostCenter he belongs to.
-
Referential Integrity. It is enforced as part of the golden data certification process. References are also constraints on entities.
A reference is expressed in the model in the form of a foreign attribute that is added to the referencing attribute.
To create a reference relationship:
- Right-click the
Reference Relationships node in the model and select
Add Reference.... The
Create New Reference Relationship wizard opens.
- In the
Create New Reference Relationship wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Physical Name:Name of the database indexes created for optimizing access via this reference.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Validation Scope: Select whether the referential integrity should be checked pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
- In the
Referencing [0..*] group, enter the following values:
-
Referencing Entity: Select the entity which references the referenced (parent) entity. For example, in an
EmployeeHasCostCenter relation, it is the
Employee entity.
-
Referencing Role Name: Name used to refer to this entity from the referenced entity. For example, in an
EmployeeHasCostCenter relation, it is the
Employees.
-
Referencing Role Label: User-friendly label to refer to one record of this entity from the referenced entity. For example, in an
EmployeeHasCostCenter relation, it is the
Reporting Employee .
-
Referencing Role Plural Label: User-friendly label to refer to a list of records of this entity from the referenced entity. For example, in an
EmployeeHasCostCenter relation, it is the
Reporting Employees (plural) .
-
Referencing Navigable: Check this box to display the navigation items to this entity from the parent entity. For example, in an
EmployeeHasCostCenter relation, selecting this option will make visible the
Reporting Employees node under each CostCenter in the outline, and a list of
Reporting Employees in the CostCenter master/detail page.
- In the
Referenced [0..1] group, enter the following values:
-
Referenced Entity: Select the entity which is referenced. For example, in an
EmployeeHasCostCenter relation, it is the
CostCenter entity.
-
Referenced Role Name: Name used to refer to this entity from the referencing entity. For example, in an
EmployeeHasCostCenter relation, it is the
CostCenter . This name is also the name given to the foreign attribute added to the referencing entity to express this relation.
-
Referenced Role Label: User-friendly label to refer to this entity from the referencing entity. For example, in an
EmployeeHasCostCenter relation, it is the
Cost Center.
-
Referenced Navigable: Check this box to display the navigation items to this entity from a child entity. For example, in an
EmployeeHasCostCenter relation, selecting this option will make visible the link to open a parent
CostCenter from an
Employee page. Note that this link will show the Display Name defined for the CostCenter entity. For more information on display names, refer to the
Working with Model Display chapter.
-
Mandatory: Define whether the reference to this entity is mandatory for the child. For example, in an
EmployeeHasCostCenter relation, this option must be checked as an employee
always belong to a cost center. For a
CostCenterHasParentCostCenter reference, the option should not be checked as some cost centers may be at the root of my organization chart.
-
Physical Name: Name of the physical column created for the foreign attribute in the tables representing the referencing entity.
- Click
Finish to close the wizard. The
Reference Relationship editor opens.
- In the
Description field, optionally enter a description for the Reference Relationship.
- Press
CTRL+S to save the editor.
- Close the editor.
Data Quality Constraints
Data Quality Constraints include all the rules in the model that enforce a certain level of quality on the entities. These rules include:
-
Mandatory Attributes: An attribute must not have a null value. For example, the
Phone attribute in the
Customer entity must not be null.
-
References (
Mandatory References): An entity with a non-mandatory reference must have a valid referenced entity or a null reference (no referenced entity). For mandatory references, the entity must have a valid reference and does not allow null references.
-
LOV Validation: An attribute with an LOV type must have all its values defined in the LOV. For example, the
Gender attribute of the
Customer entity is a LOV of type
GenderLOV. It must have its values in the following range: [M:Male, F:Female].
-
Unique Keys: A group of column that has a unique value. For example, for a
Product entity, the pair
ProductFamilyName,
ProductName must be unique.
-
Validations: A formula that must be valid on a given record. For example, a
Customer entity must have either a valid
Email or a valid
Address.
The Mandatory Attributes and LOV Validations are designed when creating the
Entities. The references are defined when creating
Reference Relationships.
In this section, the
Unique Keys and
Validations are described. Refer to the previous sections of the chapter for the other constraints.
More information:
Unique Keys
A Unique Key defines a group of attributes should be unique for an entity.
- Expand the entity node, right-click the
Unique Keys node and select
Add Unique Key.... The
Create New Unique Key wizard opens.
- In the
Create New Unique Key wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Validation Scope: Select whether the unique key should be checked pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
- Click
Next.
- In the
Key Attributes page, select the
Available Attributes that you want to add and click the
Add >> button to add them to the
Key Attributes.
- Use the
Move Up and
Move Down buttons to order the selected attributes.
- Click
Finish to close the wizard. The
Unique Key editor opens.
- In the
Description field, optionally enter a description for the Unique Key.
- Press
CTRL+S to save the editor.
- Close the editor.
Note: In the data certification processes unique keys are checked after the match and consolidation process, on the consolidated (merged) records. Possible unique key violations are not checked on the incoming (source records).
Note: Unique keys can be checked in data entry workflows to verify whether records with the same key have been entered previously by a user. This option is available only for entities using ID Matching. Besides, source records submitted to the hub via external loads (not via workflows) are not taken into account in such check. A unicity check enforced in a workflow will behave similarly to the check performed in the data certification process only if the consolidation rule gives precedence to the data submitted by users via workflows over data submitted by applications via external loads, of if data is only submitted via workflows.
Validations
A record-level validation validates the values of a given entity record against a rule. Several validations may exist on a single entity.
There are two types of validation:
-
SemQL Validations express the validation rule in the SemQL language. These validations are executed in the hub’s database.
-
Plug-in Validations use a Plug-in developed in Java. These validations are executed by Semarchy Convergence for MDM. Controls that cannot be done in within the database (for example that involve calling an external API) can be created using plug-in validations.
SemQL Validation
To create a SemQL validation:
- Expand the entity node, right-click the
Validations node and select
Add SemQL Validation.... The
Create New SemQL Validation wizard opens.
- In the
Create New SemQL Validation wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Description: optionally enter a description for the SemQL Validation.
-
Condition: Enter the SemQL condition that must be true for a valid record. You can use the
Edit Expression button to open the
SemQL Editor .
-
Validation Scope: Select whether the SemQL Validation should be checked pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
- Click
Finish to close the wizard. The
SemQL Validation editor opens.
- Press
CTRL+S to save the editor.
- Close the editor.
Plug-in Validation
Note: Before using a plug-in validation, make sure the plug-in was added to the platform by the administrator. For more information, refer to the
"Semarchy Convergence for MDM Administration Guide".
To create a plug-in validation:
- Expand the entity node, right-click the
Validations node and select
Add Plug-in Validation.... The
Create New Plug-in Validation wizard opens.
- In the
Create New Plug-in Validation wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
-
Plug-in ID: Select the plug-in from the list of plug-ins installed in the platform.
-
Validation Scope: Select whether the Validation should be checked pre and/or post consolidation. For more information, refer to the
Integration Process Design chapter.
- Click
Finish to close the wizard. The
Plug-in Validation editor opens. The
Plug-in Params and
Plug-in Inputs tables show the parameters and inputs for this plug-in.
- You can optionally add parameters to the
Plug-in Params list:
- In the
Plug-in Params table, click the
Define Plug-in Parameters button.
- In the
Parameters dialog, select the
Available Parameters that you want to add and click the
Add >> button to add them to the
Used Parameters.
- Click
Finish to close the dialog.
- Set the values for the parameters:
- Click the
Value column in the
Plug-in Params table in front a parameter. The cell becomes editable.
- Enter the value of the parameter in the cell, and then press
Enter .
- Repeat the previous steps to set the value for the parameters.
- You can optionally add inputs to the
Plug-in Inputs list:
- In the
Plug-in Inputs table, click the
Define Plug-in Inputs button.
- In the
Input Bindings dialog, select the
Available Inputs that you want to add and click the
Add >> button to add them to the
Used Inputs.
- Click
Finish to close the dialog.
- Set the values for the inputs:
- Double-Click the
Expression column in the
Plug-in Inputs table in front an input. The SemQL editor opens.
- Edit the SemQL expression using the attributes to feed the plug-in input and then click
OK to close the SemQL Editor.
- Repeat the previous steps to set an expression for the inputs.
- Press
CTRL+S to save the editor.
- Close the editor.
Diagrams
A Diagram is a graphical representation of a portion of the model or the entire model.
Using the Diagram, not only you can make a model more readable, but you can also create entities and references in a graphical manner, and organize them as graphical shapes.
It is important to understand that a diagram only displays
shapes which are
graphical representations of the entities and references. These shapes are not the real entities and reference, but graphical artifacts in the diagram:
- When you double click on a shape from the diagram, you access the actual entity or reference via the shape representing it.
- It is possible to remove a shape from the diagram without deleting the entity or reference.
- You can have multiple shapes representing the same entity in a diagram. This is typically done for readability reasons. All these shapes point to the same entity.
- If you delete an entity or reference, the shapes representing it automatically disappear from the diagrams.
Creating Diagrams
To create a diagram:
- Right-click the
Diagrams node and select
Add Diagram.... The
Create New Diagram wizard opens.
- In the
Create New Diagram wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Label: User-friendly label for this object. Note that as the
Auto Fill box is checked, the
Label is automatically filled in. Modifying this label is optional.
- In the
Description field, optionally enter a description for the Diagram.
- Click
Finish to close the wizard. The
Diagram editor opens.
Working with Entities and References
In this section, the creation/deletion of entities and references from the diagram is explained.
To create an entity using the diagram:
- In the Palette, select
Add Entity.
- Click the diagram. The
Create New Entity wizard opens.
Follow the entity creation process described in the
Creating an Entity section.
The entity is created and a shape corresponding to this entity is added to the diagram.
Note that you can also create, edit and delete attributes from the diagram. Select an attribute or entity and use the context menu options.
To create a reference using the diagram:
- In the Palette, select
Add Reference.
- Select the referencing entity in the diagram. Keep the mouse button pressed, and move the cursor to the referenced entity.
- Release the mouse button. The
Create New Reference wizard opens. It is pre-filled based on the two entities.
Follow the reference relationship process described in the
Reference Relationships section.
The reference is created and a shape corresponding to this reference is added to the diagram.
To delete a reference or an entity from the diagram:
- In the diagram, select the entity of reference that you want to delete.
- Right-click and select
Delete.
- Click
OK in the
Confirm Delete dialog.
The reference or entity, as well as the shape in the diagram disappear.
Important: Deleting an entity or reference cannot be undone.
Working with Shapes
In this section, the creation/deletion of shapes in the diagram without changing the real entity or reference is explained.
To add existing entities to the diagram:
- In the Palette, select
Add Existing Entity.
- Click the diagram. The
Selection Needed dialog opens showing the list of entities in the diagram.
- Select the entities to add to the diagram.
- Click
OK. The shapes for the selected entities are added to the diagram.
You can repeat this operation if you want to add multiple shapes for an entity in the diagram.
To add existing references to the diagram:
It is not possible to manually create a shape for reference in a diagram.
When an entity is added to the diagram, shapes for the references relating this entity to entities already in the diagram are automatically added.
To remove a shape from the diagram:
- In the diagram, select the shape representing the entity of reference that you want to delete.
- Right-click and select
Remove Shape.
The shape disappears from the diagram. The entity or reference is not deleted.
Database Reverse-Engineering
Reverse-engineering can be used to quickly populate a model from table structures stored in a schema.
During this process, entities are created with attributes named after the column names.
Note: This process connects to a database schema using a JDBC datasource that must be previously configured on the application server for the Semarchy Convergence for MDM application. Please contact the application server administrator for declaring this datasource.
Note: Reverse-engineering is provided as a one-shot method to quickly populate the initial model from a database schema. The entities created using a reverse-engineering are by default fuzzy matching entities. Foreign keys existing between the tables in the schema are not reverse-engineered as references in the model. Besides, the reverse-engineering process is not incremental.
To perform a database reverse-engineering:
- Select the
Entities node in the model.
- Right-click and select
Database Reverse-Engineering. The
Reverse-Engineer Tables to Entities appears.
- Select the
Datasource to Reverse-Engineer from the list.
- Click
Next . The list of tables and columns is reverse-engineered. At issue, an
Entity Preview page shows the list of tables and columns and the suggested logical names for these.
- The
Logical Name column correspond to the internal names of the entities and attributes generated by the reverse-engineering process.
- The
Physical Name columns is filled in with the physical name of the table and column in the database
- Select only those of the entities and attributes that you want to reverse-engineer.
- Modify the logical and physical names in this page.
- Press
Finish . The entities and attributes are created according to your definition.
Note: The entities created via a reverse-engineering process are provided as is. It is recommended to review their definition, particularly the
Matching Behavior .
Model Variables
Model variables store values retrieved from remote servers (declared as
Variable Value Providers). Variable values are local to each user session, and are refreshed when the user accesses Semarchy Convergence for MDM. Model variables can be used in SemQL filters and expressions created at design and run-time to adapt the experience to the connected user.
For more information about Variable Value Providers, see the
"Configuring Variable Value Providers" section in the
"Semarchy Convergence for MDM Administration Guide".
Creating Model Variables
Note: Before creating model variables, make sure that the variable value providers from which you want to retrieve information are declared for your Convergence for MDM instance.
To create a model variable:
- Right-click the
Model Variables node and select
Add Model Variable.... The
Create New Model Variable wizard opens.
- In the
Create New Model Variable wizard, check the
Auto Fill option and then enter the following values:
-
Name: Internal name of the object.
-
Variable Type: select the data type of the variable:
boolean,
number or
string.
-
Variable Value Provider: select the variable value provider that will be queried to retrieve the values for this variable.
- Click
Next.
- Select the
Edit Expression button.
- In the
Variable Lookup Query dialog, enter the query to retrieve the variable value:
- For a Datasource Variable Value Provider, enter a
SQL Query. You can use the
:V_USERNAME
(connected user name) as a bind variable in this query.
- For an LDAP Variable Value Provider, enter the parameters of the LDAP search:
-
Base DN: The name of the base object entry (or possibly the root) relative to which the search is to be performed. For example:
dc=myCompany
.
-
Filter: Criteria to use for selecting elements within the scope. For more information about LDAP filters, see LDAP Search Filter Syntax
-
Attribute: The attribute from the returned result to set in the variable value.
-
Search Scope: The depth of the search. Possible search scope values are
Base Object (search just the object specified in the Base DN),
Single Level (search entries immediately below the base DN), or
Whole Subtree (the entire sub-tree starting at the base DN).
- Click
Finish to close the wizard. The
Model Variable editor opens.
- Press
CTRL+S to save the editor.
- Close the editor.
Variable Lookup Queries
The variable lookup query defined in a model variable retrieves information from a variable value provider. This information can be made specific to the connected user using the
V_USERNAME
variable.
The built-in variable called
V_USERNAME
stores the connected user name, and can be referred to in the variable lookup queries using the following syntax:
-
:V_USERNAME
in
SQL queries. For example,
SELECT COUNTRY FROM USERINFO WHERE USER=:V_USERNAME
retrieves the COUNTRY for the connected user name (variable binding using the
:V_USERNAME
syntax) from a USERINFO table.
{V_USERNAME}
in
LDAP Filters. For example, the filter (&(objectClass=person)(cn=*{V_USERNAME}*))
will select “persons” (elements of the objectClass
person) for which the common name (
cn) contains the currently connected user name (variable binding using the {V_USERNAME}
syntax).
Warning: Lookup queries should return a single value (column) and a single result (record). If the query returns multiple results or multiple values, only the first value of the first result is taken into account and set in the variable value.
Note: It is not possible to use a variable in the lookup query of another variable.
Testing Model Variables
After creating a new model variable, it is recommended to test it.
To test a model variable:
- In the
Model Edition view, double-click the model variable. The Model Variable editor for this variable opens.
- In the editor toolbar, click the
Retrieve Current Value button.
- The variable value refreshed for the current session appears in the
Current Value field of the editor. If the variable cannot be refreshed, an error is issued.
Using Model Variables
Model Variable can be used in the following SemQL expressions:
In these SemQL expressions, you can bind the model variable using the
:<variable name>
syntax. You can also use in these expressions the built-in
:V_USERNAME
bind variable.
For example, to create a privilege grant allowing the connected user to see only his own record in the
Employee master data:
- In the model, we create a variable called
CurrentUserEmail
, refreshed from the LDAP directory attribute
email filtered with (&(objectClass=person)(cn=*{V_USERNAME}*))
.
- We create the privilege grant on the
Employee entity, filtered with the following SemQL Expression:
EmailAddress=:CurrentUserEmail
The connected user will be granted these privileges only for the master data record matching this expression.
Note: Variable values are not persisted. They are retrieved when the user connects, and disposed at the end of the user session. If the content of the Variable Value Provider (Remote LDAP Directory or Database) is modified, the changes are taken into account only when the user re-connects to Semarchy Convergence for MDM.
Display Names
Certain display properties are defined as part of the logical modeling effort.
The
Labels and
Descriptions provided when creating and editing the entities, attributes, types and references are are used when displaying the entities of the model in the context of applications, and as default labels and descriptions when using attributes from these entities. Other artifacts specific to displaying a model can be defined in the model. They include the
Entity Display Names and
Complex Types Display Names, described in the following sections.
Entity Display Names
An entity is a structure containing several attributes (simple and complex). When an entity value needs to be displayed in a compact form (for example, in a table, or in a single field), the
Display Name is used.
The display name defines how an entity is displayed in compact form. It is a concatenation of several attributes, separated by a
Separator.
For example, a
Contact entity is shown as
<first name>˽<last name>.
To create or modify an entity display name:
- Right-click the entity node and select
Define Display Name.... The
Modify Display Name wizard opens.
- In the
Modify Display Name wizard, enter the following values:
-
Separator: String that will separate the selected attributes in the display name.
- Click
Next.
- In the
Display Name Attributes page, select the
Available Attributes that you want to add and click the
Add >> button to add them to the
Selected Attributes.
- Use the
Move Up and
Move Down buttons to order the selected attributes.
- Click
Finish to close the wizard.
- Press
CTRL+S to save the
Display Name editor.
- Close the editor.
Note: Only one display name can be created for a given entity.
Note: The default display name of the entity can be customized when the entity is used:
* In a business object view transition referring to this entity, you can customize the string displayed for this entity in the hierarchy.
* In a form view that refers to this entity, you can customize the string displayed for the entity in the form.
Both these customizations are expressed as SemQL expressions that use the attributes of the entity.
Complex Type Display Name
A complex type is a structure containing several attributes. When a complex attribute value needs to be displayed in a compact form (for example, in a table, or in a single field), the
Display Name is used.
The display name defines how a complex attribute is displayed in compact form. It is a concatenation of several definition attributes, separated by a
Separator.
For example, the
GeocodedAddress complex type contains a large number of attributes, from the simple
StreetNumber down to the
longitude and
latitude. A display name would be for example:
StreetNumber
StreetName
City
Country .
To create or modify a complex attribute display name:
- Right-click the complex attribute node and select
Define Display Name.... The
Modify Display Name wizard opens.
- In the
Modify Display Name wizard, enter the following values:
-
Separator: String that will separate the selected definition attributes in the display name.
- Click
Next.
- In the
Display Name Attributes page, select the
Available Attributes that you want to add and click the
Add >> button to add them to the
Selected Attributes.
- Use the
Move Up and
Move Down buttons to order the selected attributes.
- Click
Finish to close the wizard.
- Press
CTRL+S to save the
Display Name editor.
- Close the editor.
Note: Only one display name can be created for a given complex type.
Previous
|
Top
|
Next
|
Introduction to the Semarchy Workbench |
|
Integration Process Design |