This tutorial teaches you how to retrieve data in Semarchy xDM using the REST API.

Knowing how to query data using the REST API is essential to building a successful MDM program. While you ordinarily would rely on a middleware tool or a user-designed program to query and load data from xDM, there are business cases where an integration developer needs to connect Semarchy xDM to an external system using the REST API. This allows near real-time queries versus batch loading, which SQL is better suited for.

In this tutorial, you will learn to use simple queries and named queries.

This tutorial is based on the Customer B2C demo application, so you will get experience dealing with retrieving fuzzy-matching entities and learning how the REST API works.

What you'll learn

Before you start

This tutorial is part of the Integration track, which is composed of SQL-based and REST-based tutorials.

Before following this tutorial, you must have set up Semarchy xDM and completed the Customer B2C Demo tutorial. Additionally, you must have configured a REST client, which is described in the Set up a REST client for Semarchy xDM.

If you have not followed these prerequisites, go back to the Tutorials menu.

GO TO TUTORIALS

The estimated duration of this unit is about 40 minutes

Enjoy this tutorial!

The REST API is available at the following base URL:

http://[host]:[port]/semarchy/api/rest

In the rest of the tutorial, this URL may be referred to as the [base_url].

Each operation in the REST API is available under this base URL. The minimum structure for a request is:

[base_url]/query/[data_location_name]/[entity_name]/[view_type]

Parameters definition:

The URL may also take additional parameters.

Before we begin constructing the REST call, let's look at the REST API documentation that Semarchy xDM generates for you automatically. This is a fantastic resource that will help you construct your REST URLs to reduce the risks of mistakes.

You can find the REST API documentation at the URL:

http://[public_dns_or_ip_address]:[port]/semarchy/api/rest/api-ui/index.html

On a typical on-premises instance of xDM, the REST documentation URL is: http://localhost:8088/semarchy/api/rest/api-ui/index.html

Navigate the documentation by clicking on the drop-down menu in the upper right corner to find the menu. Click on CustomerB2CDemo.

All entities from the data location are listed in the left menu. Click on the Person entity to see all available operations.

Scroll down to see the Query GD records section. There you can see a sample request which we will build from scratch in the next section to retrieve all golden records for an entity.

In this step, you will learn how to retrieve all records from a table using the REST API. You are going to query the source data (SD), master integration data (MI), and golden data (GD) tables for one entity.

What you'll learn

Retrieve Golden Data (GD)

You will retrieve all the customer records in the Person entity from the CustomerB2CDemo data location. The template request you need is:

[base_url]/query/[data_location_name]/[entity_name]/[view type]/[record_ID]

For Golden Data records, the value for the view_type parameter is GD.

[base_url]/semarchy/api/rest/query/CustomerB2CDemo/Person/GD

You need to modify the request to use your host and port:

http://localhost:8088/semarchy/api/rest/query/CustomerB2CDemo/Person/GD

To test this first request:

  1. Enter the customized request URL in the request field.
  2. Check the request is using the GET method.
  3. Check in the Authorization tab that you have provided your semadmin credentials. If you have not, please refer to the tutorial Set up a REST client for Semarchy xDM
  4. Click on Send.
  5. If you have correctly set up the authorization with your credentials, then you should see results returned. These are all the Source Data customer records returned in JSON format.

Retrieve Master Integration data (MI)

Replace the url used in the previous section with the following one (don't forget to change host and port) and click on Send:

http://<host>:<port>/semarchy/api/rest/query/CustomerB2CDemo/Person/MI

Retrieve all SD records

Replace the url used in previous step with the following one (don't forget to change host and port) and click on Send:

http://<host>:<port>/semarchy/api/rest/query/CustomerB2CDemo/Person/SD

Note the difference of results for GD, MI and SD tables.

Congratulations

Great job! You successfully queried xDM via REST API:

This is just a preview of what you can do with xDM REST API. In the next step, you will learn how to get a record with its identifier.

In this step, you will learn how to retrieve a record using its golden identifier. This is useful for example, if your organization has a call center application that needs to retrieve from Semarchy xDM up-to-date data for a customer known by its ID.

What you'll learn

Retrieve the golden ID

For the needs of this tutorial, you must retrieve the ID of the golden customer record Mary-Ann Markes. You will use this ID within your REST request.

The value for this ID is specific to your environment as it depends on the order in which records were processed by Semarchy xDM.

To retrieve this golden ID, proceed as follows:

  1. Connect to the demo application Customer B2C.
  2. Navigate to the list of Customers.
  3. Search for Mary-Ann Markes.
  4. Open the record and retrieve the golden ID on the Basic Information tab:

In the example below, the golden record has the ID 10024.

Construct the request

As indicated by the REST API documentation, use the GET method to retrieve a single customer record. This is the skeleton of the REST call you will make searching on customer ID:

[base_url]/query/[data_location_name]/[entity_name]/[view type]/[record_ID]

In practice, you will use the view GD to query golden records. You request will look like:

http://[host]:[port]/semarchy/api/rest/query/CustomerB2CDemo/Person/GD/[record_id]

Launch the request in Postman

To send this request in Postman:

  1. Enter your request URL into the request field. Remember to substitute your public IP address or public DNS address and port.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

This is the result you can expect:

{
    "ID": 11305,
    "FirstName": "Mary-Ann",
    "LastName": "Markes",
    "PhoneticFirstName": "MRN",
    "PhoneticLastName": "MRKS",
    "NormalizedFirstName": "MARY-ANN",
    "NormalizedLastName": "MARKES",
    "Nickname": "mary-ann",
    "DateOfBirth": "1899-12-31",
    "SourceEmail": "maryann@markes.com",
    "CleansedEmail": "maryann@markes.com",
    "ValidEmailDomain": "1",
    "SourcePhone": "203-967-5612",
    "StandardizedPhone": "(203) 967-5612",
    "PhoneGeocodingData": "Stamford, CT",
    "ValueStatus": "HIGH",
    "PersonType": "CUSTOMER",
    "Address": "1200 Summer St #-103 Stamford CT 06905 US",
    "Address.Street": "1200 Summer St #-103",
    "Address.City": "Stamford",
    "Address.State": "CT",
    "Address.PostalCode": "06905",
    "Address.Country": "US"
}

Note how the information differs from the data you see in the application UI and the SQL query results observed in the Load data via the SQL API tutorial.

Congratulations

You learned how to construct a request URL using a record's ID.

In the next step, you will learn a more advanced request URL to look up a customer record based on first names and no IDs. This involves using a SemQL expression to search through all customer records.

In the previous step, you learned how to retrieve a customer record using the customer's account ID number. It's also common for customers to not have their account ID numbers handy. Customer service representatives will frequently ask the customer for other identifying information, such as the customer's first name and last name or address.

Following this business case, let's build on the previous request URL using the customer's name instead of account number.

What you'll learn

Retrieve a record by last name

The REST API allows external systems in your organization to retrieve records in xDM using filters. In this section, you will use the GET method to retrieve a single customer record by its name, using a SemQL filter.

In our environment, we selected the golden customer record of John Edelman to retrieve. Let's retrieve using the last name, Edelman.

Construct the request

This is the skeleton of the REST call you will make using a filter:

[base_url]/query/[data_location_name]/[entity_name]/[view type]?$f=[SemQL_filter]

In practice, this is what request URL looks like on a typical on-premises quick-start instance:

http://localhost:8088/semarchy/api/rest/query/CustomerB2CDemo/Person/GD?$f=LastName%20LIKE%20%27Edelman%27

The SemQL filter originally started as LastName LIKE 'Edelman'.

Remember to substitute the spaces in the URL with %20 and the single quotes with %27. Learn more about percent-encoding (also called URL encoding).

Launch the request in Postman

To run this request in Postman:

  1. Enter your request URL into the request field.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

Note that the $f parameter was correctly interpreted by Postman as a query parameter (in the Params tab).

This is the result you can expect:

{
    "records": [
        {
            "ID": 11303,
            "FirstName": "John",
            "LastName": "Edelman",
            "PhoneticFirstName": "JN",
            "PhoneticLastName": "ETLM",
            "NormalizedFirstName": "JOHN",
            "NormalizedLastName": "EDELMAN",
            "Nickname": "jack",
            "DateOfBirth": "1976-09-03",
            "SourceEmail": "john@jedelman.com",
            "CleansedEmail": "john@jedelman.com",
            "ValidEmailDomain": "1",
            "SourcePhone": "415.381.9176",
            "StandardizedPhone": "(415) 381-9176",
            "PhoneGeocodingData": "Mill Valley, CA",
            "ValueStatus": "NORMAL",
            "PersonType": "CUSTOMER",
            "Address": "591 Redwood Hwy #-1200 Mill Valley CA 94941 US",
            "Address.Street": "591 Redwood Hwy #-1200",
            "Address.City": "Mill Valley",
            "Address.State": "CA",
            "Address.PostalCode": "94941",
            "Address.Country": "US"
        }
    ]
}

Note how the information differs from the data you see in the application UI and the SQL query results observed in the Load data via the SQL API tutorial.

Congratulations

In this step, you built an advanced request URL using a filter that allowed you to filter on the customer's last name with a SemQL filter instead of account number.

Let's summarize:

In the next step, you will learn about named queries and how to build a custom data structure to create a customized REST endpoint.

In the previous step, you learned how to retrieve a customer record using a SemQL filter that queries the customer's last name.

So far in this tutorial, you have used predefined REST endpoints, such as retrieving a single record and using filters. There will be situations where you need a custom REST endpoint because your systems require a special data structure.

In this step, you will learn how to create a Named Query in the model design. Then you will use the Named Query in Postman to retrieve the custom data structure.

What you'll learn

Create the Named Query

Let's create the Named Query in the model designer:

  1. Go to the Application Builder. (You will need to log in as a user with Application Design privileges. It is most likely semadmin.)
  2. Open the CustomerB2CDemo model.
  3. Right-click the Named Queries node and select Add Named Query.

  1. Enter the following values and then press next:

  1. Give the object the name Customer and then click Next.

  1. Select the following Object Properties and then click Finish:

  1. Select the PersonProducts object in the Object Properties table view. In the Properties views, rename this object to Products.

  1. Expand the Products object properties.
  2. Select the PersonProduct node. Add a new property.

  1. In the object property, set the following values and then click Finish:

  1. Select the PersonProduct node again. Add a new property again.
  2. In the object property, set the following:

  1. Select the PersonProduct node again. Add a new property again.
  2. In the object property, set the following:

  1. Save this Named Query:

  1. For the new Named Query to take effect, you need to refresh the application: go back to the Customer B2C application, click the user profile button and click Refresh application.

  1. You are now ready to use this Named Query in Postman and see the results of a custom REST endpoint.

Call the Named Query

Let's call the Named Query to see the results.

This is the skeleton of the REST call you will make using a Named Query:

[base_url]/named-query/[data_location_name]/[named_query]/[view type]

In practice, this is what our request looks like on an on-premises quick-start xDM instance:

http://localhost:8088/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD

To run this request in Postman:

  1. Enter your request URL into the request field.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

The result you should see are all the golden customer records with their corresponding products:

{
    "records": [
        {
            "ID": 10001,
            "CleansedEmail": "angelica@aldaba.com",
            "NormalizedFirstName": "angelica",
            "NormalizedLastName": "miller",
            "PersonType": "CUSTOMER",
            "StandardizedPhone": "(503) 228-3027",
            "ValueStatus": "NORMAL",
            "Products": [
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-12-23",
                    "Registrationdate": "2017-03-13"
                },
                {
                    "Product": "Streamliner Saddle",
                    "PurchaseDate": "2015-11-04",
                    "Registrationdate": "2015-12-14"
                }
            ]
        },
        {
            "ID": 10002,
            "CleansedEmail": "saraj@yahoo.com",
            "NormalizedFirstName": "sara",
            "NormalizedLastName": "johnson",
            "PersonType": "CUSTOMER",
            "StandardizedPhone": "(415) 472-3289",
            "ValueStatus": "NORMAL",
            "Products": [
                {
                    "Product": "Foxy Elite Cycling Shoes",
                    "PurchaseDate": "2016-10-12",
                    "Registrationdate": "2016-10-16"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-05-09",
                    "Registrationdate": "2016-05-11"
                },
                {
                    "Product": "Streamliner Saddle",
                    "PurchaseDate": "2016-04-12",
                    "Registrationdate": "2016-05-10"
                }
            ]
        },
...(continued)
    ]
}

Note that the list of Products linked to each customer is returned by the Named Query with the Product, PurchaseDate and RegistrationDate properties.

Congratulations

Great job creating a Named Query. Named queries are useful for creating custom data structures to support your organization's specific requirements when integrating Semarchy xDM with other systems.

In this step:

In the next step, you will build on the basic Named Query you created by adding query parameters that allow you to filter results.

In the previous step, you learned how to create a basic Named Query. This created a custom REST endpoint that allows you to retrieve a customer record with that customers' corresponding products.

In this step, you will build upon the Named Query by adding in advanced features. You will add query parameters that allow you to filter the results with a SemQL expression, similar to the filter you used during the step Get a customer record by last name.

What you'll learn

Add a Query Parameter

In our business case of a customer support call center, the customer service representative wants to pull up information about the customer in the call center system. Usually, the representative will ask for the customer's information, such as an email address or account ID number, which both can serve as identifiers.

The REST endpoint must have a way to submit this identifier information to xDM. The way we achieve this is by using query parameters.

Let's create a query parameter in the Named Query:

  1. Go to the Application Builder.
  2. Open the PersonByIDWithProducts Named Query you created in the previous step.
  3. Add a new query parameter.

  1. In the query parameter, set the following:
  1. Remember to save your Named Query. You need to refresh the application (alternatively, you can also deploy the model if you don't want to move to the front-end application due to convenience) for the query parameter to take effect.

Non-regression test

Start by testing that your NamedQuery is still working: use the same request that you used in the previous section in Postman to call the named query.

http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD

You should get the same results as before adding the parameter.

Create a filter

Adding query parameters is the first operation required to filter the records xDM retrieves. The second operation consists of setting up a filter that uses these query parameters. The filter is a SemQL expression that compares the incoming information with the existing data in xDM.

Let's create the filter and see how it works with query parameters.

  1. Scroll to the top of the Named Query configuration in the Application Builder. Find the Filter field and open the SemQL editor:

  1. Copy and past the SemQL expression for your database:

Oracle

SEM_NUMBER_TO_CHAR( ID ) = :QUERY_PARAM_IDENTIFIER
OR CleansedEmail = :QUERY_PARAM_IDENTIFIER
OR REGEXP_REPLACE( StandardizedPhone, '[^[:digit:]]', '' ) = REGEXP_REPLACE( :QUERY_PARAM_IDENTIFIER, '[^[:digit:]]', '' )

PostgreSQL

SEM_NUMBER_TO_CHAR( ID ) = :QUERY_PARAM_IDENTIFIER
OR CleansedEmail = :QUERY_PARAM_IDENTIFIER
OR REGEXP_REPLACE( StandardizedPhone, '[^[:digit:]]', '', 'g' ) = REGEXP_REPLACE( :QUERY_PARAM_IDENTIFIER, '[^[:digit:]]', '', 'g' )

  1. Save the NamedQuery and refresh the application.

Call the updated Named Query

To use the Named Query, include the parameters in the request URL. In this part, you will learn how to update your request URL to include the query parameters and corresponding values (the arguments).

These are the 3 identifiers that this Named Query can accept:

The query parameter IDENTIFIER is set up to only accept strings. However, the filter is able to check for account IDs, email addresses, and phone numbers. It handles the account ID by casting a number into a string on the fly to compare with the xDM ID. It takes a phone number and strips out non-digit characters to compare to the phone number in xDM.

This is the skeleton of the REST call you will make using a Named Query:

[base_url]/named-query/[data_location_name]/[named_query]/[view type]?IDENTIFIER=[customer_ID_or_cleansed_email_or_phone_number]

After the IDENTIFIER query parameter, you can provide the customer's account ID number, email address, or phone number.

Retrieve a record with the customer's account ID

First, start with the customer's account number (golden record ID).

Imagine the customer Toni Mattos calls into the customer support number with a warranty issue. The customer support representative asks Toni for her account number to pull up her information. She supplies her account number (in other words, golden record ID), 11299. This is an example of the request that the call center system can make to retrieve Toni's record from xDM.

http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=11299

To run this request in Postman:

  1. Enter your request URL into the request field. Remember to substitute your public IP address or public DNS address.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered:

{
    "records": [
        {
            "ID": 11299,
            "CleansedEmail": "toni@mattos.com",
            "ValueStatus": "HIGH",
            "StandardizedPhone": "(321) 727-6512",
            "NormalizedFirstName": "TONI",
            "PersonType": "CUSTOMER",
            "NormalizedLastName": "MATTOS",
            "PersonProducts": [
                {
                    "Product": "Carbonite Century ",
                    "PurchaseDate": "2016-07-15",
                    "RegistrationDate": "2016-08-04"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2014-09-17",
                    "RegistrationDate": "2014-11-26"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-08-20",
                    "RegistrationDate": "2016-10-08"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-10-09",
                    "RegistrationDate": "2016-10-29"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2015-06-16",
                    "RegistrationDate": "2015-07-23"
                },
                {
                    "Product": "Streamliner Saddle",
                    "PurchaseDate": "2015-03-24",
                    "RegistrationDate": "2015-05-10"
                }
            ]
        }
    ]
}

Notice that the attributes related to the person are not in a user-friendly order. In the next step, you will reorder the attributes so that the first name and last name attributes come after the ID.

Reorder the attributes in the Named Query

Let's reorder the attributes so that they are in a logical order:

  1. Go to the Named Query PersonByIDWithProducts in the Application Builder.
  2. Use the Move Up and Move Down arrows to rearrange the object properties to be in the following order:

  1. Refresh the application so that the new properties order takes effect.

Retrieve a record with the customer's email

In case the customer doesn't have her account number, let's see what it would look like to retrieve her customer account information based on her email:

http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=toni%40mattos.com

To run this request in Postman:

  1. Enter your request URL into the request field. Remember to substitute your public IP address or public DNS address.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered:

{
    "records": [
        {
            "ID": 11299,
            "NormalizedFirstName": "TONI",
            "NormalizedLastName": "MATTOS",
            "StandardizedPhone": "(321) 727-6512",
            "CleansedEmail": "toni@mattos.com",
            "PersonType": "CUSTOMER",
            "ValueStatus": "HIGH",
            "Products": [
                {
                    "Product": "Carbonite Century ",
                    "PurchaseDate": "2016-07-15",
                    "RegistrationDate": "2016-08-04"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2014-09-17",
                    "RegistrationDate": "2014-11-26"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-08-20",
                    "RegistrationDate": "2016-10-08"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-10-09",
                    "RegistrationDate": "2016-10-29"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2015-06-16",
                    "RegistrationDate": "2015-07-23"
                },
                {
                    "Product": "Streamliner Saddle",
                    "PurchaseDate": "2015-03-24",
                    "RegistrationDate": "2015-05-10"
                }
            ]
        }
    ]
}

Retrieve a record with the customer's phone number

In case the customer doesn't have her account number or email, let's see what it would look like to retrieve her customer account information based on her phone number (321) 727-6512:

http://[host]:[port]/semarchy/api/rest/named-query/CustomerB2CDemo/PersonByIDWithProducts/GD?IDENTIFIER=3217276512

To run this request in Postman:

  1. Enter your request URL into the request field. Remember to substitute your public IP address or public DNS address.
  2. Double check the method is GET.
  3. Click Send.
  4. See the results returned in JSON format.

The result you should see is Toni Mattos' information and the corresponding products she has purchased and registered. Here is an excerpt of what you can expect:

{
    "records": [
        {
            "ID": 11299,
            "NormalizedFirstName": "TONI",
            "NormalizedLastName": "MATTOS",
            "StandardizedPhone": "(321) 727-6512",
            "CleansedEmail": "toni@mattos.com",
            "PersonType": "CUSTOMER",
            "ValueStatus": "HIGH",
            "Products": [
                {
                    "Product": "Carbonite Century ",
                    "PurchaseDate": "2016-07-15",
                    "RegistrationDate": "2016-08-04"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2014-09-17",
                    "RegistrationDate": "2014-11-26"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-08-20",
                    "RegistrationDate": "2016-10-08"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2016-10-09",
                    "RegistrationDate": "2016-10-29"
                },
                {
                    "Product": "Renaud Martin Skin Suit",
                    "PurchaseDate": "2015-06-16",
                    "RegistrationDate": "2015-07-23"
                },
                {
                    "Product": "Streamliner Saddle",
                    "PurchaseDate": "2015-03-24",
                    "RegistrationDate": "2015-05-10"
                }
            ]
        }
    ]
}

Congratulations

You learned how to retrieve data in xDM using a more advanced Named Query:

Well done! You have run your first REST GET requests to retrieve data from Semarchy xDM.

In this tutorial, you learned:

What's next?

In the next unit, you will learn how to load data using the REST API.