Manage xDM Discovery datasources

xDM Discovery Datasources are data stores containing the data to profile. An xDM Discovery datasource may be directly connected to the application data to profile, or a staging location into which application data is loaded using an integration layer.

xDM Discovery can only use data sources with the SQL Server, Oracle, or Postgres technologies. Other database technologies are not supported.

When you connect to xDM Discovery, the datasources list appears:

  • When this list is empty, a Create Datasource button displays allowing you to create your first datasource.

  • When this list already contains datasources, you can click any datasource in the list to browse its profile, or click the edit Edit action to configure this datasource.

  • You can switch the list to a table view by selecting the table view Table view item in the more menu More menu. In the table view:

    • Click the column header to sort the datasources or use the sort Sort menu item.

    • Customize the table contents with the select columns Select columns menu item.

Datasources in the datasource list support the following operations:

  • view dashboard View Dashboard to view the datasource profile.

  • edit Edit to configure the datasource.

  • copy Copy a selected datasource. When copying a datasource, you are prompted for the name of the new datasource.

  • rename Rename a selected datasource. This operation changes the internal name of the datasource and not its label. When renaming a datasource, the incoming references to this datasource are not automatically updated.

  • delete Delete selected datasources. This operation cannot be undone.

  • seed profiling app Seed a profiling app from a selection of datasources. You will be able to customize this dashboard application to browse the profiles.

Renaming or deleting a datasource changes or invalidates the URL used to browse this datasource. These two operations must be performed with care if you have shared this URL with other users.
Some of these operations are available from the navigation drawer, in the more menu More menu that is available on each datasource shortcut.

Create a datasource

To create a datasource:

  1. Select Datasources in the navigation drawer.
    The Datasources list opens.

  2. Click the Add datasource floating action button in the lower-right corner of the screen.

  3. In the New Datasource dialog:

    1. Select the datasource name corresponding to a datasource configured in the Semarchy xDM platform.
      The Database type is automatically detected, and the Name and Label are automatically generated based on the datasource name.

    2. Change the Name and Label for the datasource.

    3. Click Create.

The datasource editor opens with the list of tables found in the database.

Configure a datasource

To configure a datasource:

  1. In the datasource editor, select the edit Open datasource property item in the editor toolbar.
    The datasource properties sidesheet opens.

  2. Change the datasource label:

    • Click the rename Rename button in the sidesheet header and enter a new label for the datasource.

  3. Configure the datasource properties:

    • In the Profiling Options:

      • Max processes define the maximum number of tables that can be simultaneously profiled.

    • In the Display Properties:

      • Select Show in shortcuts to have the datasource appear in the xDM Discovery navigation drawer.

      • Select an Icon and Icon color. This optional icon appears in the shortcuts next to the datasource label.

      • Select a Color for the datasource label.

    • In the Documentation section, edit the rich text describing the datasource.

  4. Click save Save to save your changes.

Manage tables

The datasource contains the list of tables retrieved from the database at a certain point in time. If you have just created the datasource or added new tables in the database, you can refresh the list of tables.

To refresh the list of tables:

  1. In the datasource editor, select the more menu More menu in the upper-right corner of the list of tables.

  2. Select refresh Refresh table list.

The list of tables refreshes:

  • Existing tables remain untouched and keep their profile.

  • New tables are added to the list with no profile. You may start profiling these tables.

  • Tables that were removed from the database now appear with a not found indicator. If these tables had profiles, they are preserved until you choose to remove these tables.

The list of tables supports the following capabilities:

  • Sorting by clicking the column header.

  • Multiple selections and select all.

  • Filtering by table name using the filter Filter button.

To remove tables from the list:

  1. In the datasource editor, select the tables that you want to remove.

  2. In the more menu More menu, select remove table Remove table.

  3. When you remove the existing profiles of tables, you must confirm the deletion of these profiles.

If you have inadvertently removed a table, refresh the table list to restore it in the list.
Dropping a table profile deletes all profiling information for this table, which cannot be undone.

To configure table properties:

  1. In the datasource editor, click the table that you want to configure.
    The table properties sidesheet opens.

  2. Configure the table properties:

    • Select Scheduled to profile this table regularly on a schedule, and then define Cron expression defining the profiling schedule.

    • In the Profiling Options:

      • Maximum distinct values stored defines the maximum number of values stored for distinct value analysis.

      • Maximum patterns stored defines the maximum number of patterns stored for pattern analysis.

  3. Click save Save to save your changes.

To edit the properties for multiple tables, select these tables and then edit Edit menu in the more menu More menu.
You can enable/disable the Scheduled flag on the tables directly from the list of tables.

Profile tables

To profile tables:

  1. In the datasource editor, select the tables that you want to profile.

  2. Select profile selection Profile selection in the table toolbar.
    Profiling starts for the selected tables.

When a table is in the Profiling or Queued | status, you can stop its profiling processes by selecting it from the list and then clicking cancel profiling Cancel profiling from the more menu More menu.

Profiling execution

When profiling runs for a table, its Profile Status is set to Profiling, followed by a progress percentage. When complete, the table state changes to Ready.

The Last profiled column indicates when the last successful profiling process ran for this table, and the View profile link opens the profile for this specific table.

Only a limited number of tables are profiled simultaneously. This limit is set by the Max processes property in the datasource configuration. If you start profiling more tables, some of them appear as Queued. They are moved to the Profiling state when the previous tables have been profiled.

If the profiling process fails for a table, for example, if the table was dropped from the database, then the status is Failed. A tooltip on the failed status icon shows the error message.

The Profile Status reflects the current state of profiling for each table in the datasource. An empty value reflects a table for which there is no profiling data. The following table describes the Profile Status values:

Table 1. Profile Status values

Profile status

Description

Ready

The latest profiling process ran successfully. Fresh profiling data is available for this table.

Profiling

A profiling process is running for the table.

Queued

A profiling process is queued for the table.

Failed

The latest profiling process failed. Previous profiling data may be available.

Canceled

The latest profiling process was stopped by the user.