Enable Data Quality
Data Quality in Semarchy xDI is enabled through constraints that are defined on datastores and applied at run-time.
Define Constraints on Metadata
Constraints in Semarchy xDI are defined on datastores and include:
-
primary, alternate, and foreign keys.
-
Format constraints: size, number of digits.
-
Nullable/mandatory constraints.
-
Custom constraints, configured with conditions.
Constraints are automatically added by Reverse Engineering and can be added or modified manually.
To view and edit constraints on a datastore:
-
Navigate to the concerned datastore from the Project Explorer or the metadata’s editor.
-
When a column is selected, the Standard finger tab of the Properties view contains the data quality constraints that apply to this column:
Property |
Description |
Type |
Datatype of the column. |
Size |
Maximum size. Unit depends on the value of Type (bits for numeric data types, digits for |
Decimal Digits |
Maximum number of decimal digits. Automatically set to |
Is Nullable |
Defines the default behavior if the column contains a null value. If this option is selected and if Reject Null Value is set to |
Nullable Security Level |
Defines if a null value should raise a Fatal error, a Reject or a Warning. |
Primary, alternate, and foreign keys, as well as conditions, defined for the datastore are displayed at the same level as columns. For more details, please refer to the sections below.
Primary keys
Primary Keys uniquely identify each record in a datastore.
To add a Primary Key constraint to a datastore, proceed as follows:
-
Right-click the datastore in the metadata’s editor and then select New > Primary Key.
-
Enter a Name for the constraint.
-
Optionally, set the following properties:
-
User Message: Defines the specific code to return if this constraint is violated.
-
Reject Code: Defines the specific code to return if this constraint is violated.
-
Severity Level: Defines whether a violation of this constraint should raise a Fatal error, a Reject or a Warning.
-
The columns that compose a primary key are defined using Column Reference child objects. To add a Column Reference to a new or existing Primary Key constraint:
-
Right-click the new node added under the datastore for the new constraint and select New > Column Reference.
-
In the Properties view, select the Primary Column you want to add to the primary key.
-
Save the editor.
Foreign Keys
Foreign Keys define a referential integrity constraint on a datastore column towards another datastore.
To add a Foreign Key constraint to a datastore:
-
Right-click the datastore in the metadata’s editor and then select New > Foreign Key.
-
Enter a Name for the constraint.
-
In the Referenced Datastore section, set the Schema and the Datastore targeted by this foreign key.
-
Optionally, set the following properties:
-
User Message
-
Reject Code
-
Severity Level
-
The columns referenced by a Foreign Key are defined using Column Relation child objects. To add a Column Relation to a new or existing Foreign Key:
-
Right-click the new node added under the datastore for the new constraint and select New > Column Relation.
-
In the Properties view, select the Column from the current Datastore.
-
Select the Referenced Column from the referenced datastore.
-
Save the editor.
Alternate Keys
Alternate Keys define secondary keys that are composed of one or more columns. By definition, Alternate Keys imply a unicity constraint.
To add an Alternate Key constraint to a datastore:
-
Right-click the datastore in the metadata’s editor and then select New > Alternate Key.
-
Enter a Name for the constraint.
-
Optionally, set the following properties:
-
User Message
-
Reject Code
-
Severity Level
-
The columns that compose an alternate key are defined using Column Reference child objects. To add a Column Reference to a new or existing Alternate Key constraint:
-
Right-click the new node added under the datastore for the constraint and select New > Column Reference.
-
In the Properties view, select the Primary Column to add to the alternate key.
-
Save the editor.
Conditions
Conditions describe a custom constraint defined by a condition expression that will be evaluated at run-time.
To add a Condition constraint to a datastore:
-
Right-click the datastore in the metadata’s editor and then select New > Condition.
-
Enter a Name for the constraint.
-
Optionally, set the following properties:
-
User Message
-
Reject Code
-
Severity Level
-
-
Enter a Condition Expression.
Configure And Execute Rejects
By default, constraints do not generate rejects. This must be configured on mappings or Processes.
Configure Rejects in Mappings
Rejects are configured on mappings to send to a Rejects table the records that do not meet the constraints defined on a target datastore. For more details, refer to Rejects Management.
Execute Rejects in processes (Standalone Mode)
Rejects that are activated on a mapping apply every time this mapping is executed in a data flow. Additionally, it is possible to check constraints and generate rejects in processes without loading data into the target datastore. This procedure is designated as standalone mode and is enabled by adding a Reject step to processes.
For more details, rejer to Work with Processes