Manage REST clients

REST clients extend the capabilities of Semarchy xDM to process data by calling external services via their REST API.

You can define a REST client for the entire Semarchy xDM platform in the Configuration module, and then use this client in all the models in the repository.

REST client overview

When xDM calls a REST client to enrich a record, the client sends a request to the external service.
This request is defined by:

An HTTP URL, which is accessed using an HTTP method (GET, POST, etc.).
(Optional) HTTP headers (e.g., to pass authentication information).
(Optional) A body, typically in JSON format.

The elements of the request can be parameterized by request inputs, passed to the REST client. These inputs are typically data from the record being enriched.

The external service returns a response that contains:

An HTTP status code (e.g., 200: success, 404: not found).
A response body, typically in JSON format from which response outputs will be extracted. These outputs are mapped to attributes in the records to enrich.

Declare a REST client

To create a REST client:

In the navigation drawer of the Configuration module, select REST Clients.
In the REST Clients editor toolbar, select Add New > New REST Client.
The Create New REST Client dialog opens.
Enter the following information:
- Name: the name of the REST client. When using REST services, models refer to this client by the specified name.
- Label: a user-friendly label for the REST client. This label appears in the Workbench. If you selected the Auto-Fill checkbox, this field is automatically populated based on the value of the Name field.
- (Optional) Description: a meaningful description for the model designers who will use this client.
Click Next.
HTTP Method: select the HTTP method used in the REST API call.
URL: enter the URL of the REST endpoint.
You can make parts of this URL variable by replacing constant values with request inputs, using the ${request_input_name} syntax. For example, if the service URL incorporates a variable segment to pass an email address, you can use the syntax below to designate this address as an input of the REST client: https://information.acme.com/information?email_adr=${email}
HTTP Headers: specify any necessary HTTP headers to include with the API call to the endpoint. You can use variable inputs in headers, similar to how they are used in URLs.
Body Format: select the format of the request body. Use NONE if the endpoint does not require a body, JSON for a JSON payload, and BINARY for a binary payload.
Body: the payload sent as the request body. Compose the JSON or binary payload required by the endpoint. You can use variable inputs in the body, similar to how they are used in URLs.
Click Finish.
The REST Clients editor opens.
Review and update the Name, Label, and Description fields for the client, as well as the HTTP Request parameters.
If you have used inputs using the ${request_input_name} syntax in the wizard, they are automatically added to the Request Inputs list.
You can also manage these inputs manually:
- To detect new inputs added to the URL, headers, or body, click Detect Inputs in the Request Inputs table toolbar.
- To create a new input, click Add Input in the Request Inputs table toolbar.
- To delete unused request inputs, select them and then click Delete in the Request Inputs table toolbar.
In the Request Inputs table, fill the following fields for each input:
- Label: a user-friendly label for the model designer who will use the REST client.
- Description: a meaningful description for the model designer who will use the REST client.
- Data Type: select the expected data type for the input values. For more information, see Request input datatype and formatting below.

In the Response Output table, click Add Output.
The new output appears in the list.

Fill the Name, Label, and Description fields for this output. The label and description appear in the Workbench for model designers. The name is used by an enricher to refer to the output.
Value Selector: a selector is a JSONPath expression, starting from the root of the response payload and pointing to a single value in the payload.
Data Type: select the data type of the output values.

Behavior on Absent Value: defines whether the enricher should fail or return the default value if the JSONPath expression returns no value for the payload.

The behavior defined for absent values does not apply to null values. Null values received in the response payload are returned by the REST client as they are, without any processing or alteration.

Behavior on Data Type Mismatch: defines whether the enricher should fail or return the default value if the JSONPath expression returns a value that is not compatible with the output datatype.
Default Value: specify a default output value or leave it empty to have a null default value.

Repeat the previous step for each output returned by the REST client. For more information about outputs, see Response processing.
In the Advanced Configuration section of the editor, set the following parameters:
- Timeout: duration for the HTTP request to wait for a response. If this timeout period elapses without receiving a response, the call is deemed failed.
- Max Retries: maximum number of attempts for the request. Retries occur only if the service reports a code indicative of a temporary failure, such as HTTP 404, 410, 429, and 503.
- Success Status Codes: a comma-separated list of HTTP response status (e.g., 200, 201, 202) and response status patterns (e.g., 2xx) considered successful. The default is 2xx.
- Error Selector: JSONPath expression returning a possible error to search in the response payload. Note that this path is evaluated even for successful responses. If a value is returned in that case, the error is raised.
Press Control+S (or Command+S on macOS) to save the REST client.

Before proceeding, make sure to test the REST client.

Request input datatype and formatting

When using a request input in the REST client definition, the datatype defines how values for this input are processed and inserted into the request.

Automatic formatting:

Binary inputs are automatically encoded in Base64 when used in the URL, headers or a JSON body. In a binary body, a binary input remains unformatted.
Date and timestamp inputs are formatted as strings in the ISO_8601 format (YYYY-MM-DD for dates, YYYY-MM-DDTHH:mm:ss.SSSZ for timestamps).
Number inputs are formatted as strings with a dot as the decimal separator and no thousand separators.
Boolean inputs retain their boolean values (true or false) without any formatting.

The inputs are then substituted by their formatted values in the request.

During this substitution:

Encoding and escaping processes are automatically handled in the URLs and JSON.
Quotation marks are added around strings, binary, dates, and timestamps in the JSON body.
Inputs with null values are replaced with null in the JSON payload.

Formatting in a JSON body

Consider the following example:

{
    "Name"      : ${fullName},
    "Comment"   : ${comment},
    "Height"    : ${height},
    "Photo"     : ${picture}
}

fullName and comment are strings. They are automatically escaped and surrounded by quotation marks.
height in a numeric input. It is formatted with no quotation marks.
picture is a binary. It is encoded in Base64 and surrounded by quotation marks.

A possible result can be:

{
    "Name"      : "John \"Joe\" Doe",
    "Comment"   : null,
    "Height"    : 180.2,
    "Photo"     : "data:image/png;base64,iVBOR...."
}

Example of URL encoding

Consider that you pass the value Mountain View California as the address input in the following URL: https://information.acme.com/address?addr=${address}.

The address input is automatically encoded in the URL: https://information.acme.com/address?addr=Mountain%20View%20California.

Response processing

The REST client extracts from the response payload:

The output values, using value selectors.
A possible error message, using the error selector.

Both these selectors use JSONPath, a JSON query language. For the detailed JSONPath syntax supported by Semarchy xDM, see https://github.com/json-path/JsonPath.

JSONPath supports retrieving arrays of data. For the value selectors, in cases where the JSONPath expression returns an array, the first element of the array is extracted in the output.

Basic JSONPath examples

Consider the following payload:

{
    "customers":  [
        {
        "first name": "Joe",
        "last name": "Celko",
        "age": 58,
        },
        {
        "first name": "Roger",
        "last name": "Moore",
        "age": 69,
        }
    ],
    "customers_count": 2
}

The following JSONPath expressions return the following values:

$.customers_count returns 2.
$.customers[0].age returns 58.
$['customers'][0]['age'] returns 58 (different syntax).
$['customers'][1]['last name'] returns "Moore".

Test a REST client

To test a REST client:

In the REST Clients editor toolbar, select Manage > Test.
The REST client test dialog opens.
Enter values for the request inputs, and then click Execute.

The REST client calls the endpoints, processes the outputs, and shows the resulting values.

Import and export REST clients

REST clients are stored in the repository. When moving a model to a different repository (e.g., from a development to a test environment), you must also move the REST clients used by this model.

In the source environment, click Export in the REST Clients editor to export the client definition to a file.
In the target environment, in the REST Clients editor toolbar, select Add New > Import to import the exported client definition.

You can also use the platform’s REST API to automate the export/import process of REST clients across environments.