Getting started with HTTP REST

Overview

The HTTP REST component ships with a wizard to help users reverse-engineer and test HTTP REST APIs.

This wizard allows defining the input and output parameters, the endpoint URL and HTTP method. It allows testing the API for a response. It also offers the possibility to define the reverse-engineered metadata.

Create a metadata

To create a metadata:

  1. Right-click a folder in your project and then select New > Metadata.

  2. In the Create Metadata dialog, select HTTP REST and then click Next.

  3. Name the metadata and click Next.

  4. Select the HTTP REST module and click Finish.
    A wizard opens to define the HTTP security, proxy settings, and to reverse-engineer the metadata structure.

  5. In Security Definition, set the following properties:

    • HTTP Security: Select an HTTP Security configuration previously created in an HTTP Security metadata. APIs may be accessed using one of the following methods:

      • None (default): No credentials are required to access the API.

      • User credentials: User credentials are required to access the API. The credentials specified in Semarchy xDI Designer are used by default.

      • OAuth2 security: OAuth2 is required to access the API.

    • In HTTP Proxy: Select an HTTP Proxy configuration, previously configured in a Proxy Security metadata.

      The above fields display and use Proxy Security and HTTP Security metadata available in the workspace.
  6. Click Next.

  7. In the Reverse step, set the following fields:

    • Reverse URL or File: The component can reverse-engineer the API from an OpenAPI 3 specification. Enter the URL or use the browse button to specify the path to a file containing that specification.

    • HTTP Keystore: Specify the keystore required to access the API. This information must be defined in a Certificate and Keys metadata in the workspace.

    • HTTP Truststore: Specify the truststore required to access the API. This information must be defined in a Certificate and Keys metadata in the workspace.

  8. Click Next.

  9. In the Selective Reverse step, select the nodes that you want to reverse-engineer.

  10. Click Finish.

The HTTP REST metadata node is created and the data is reverse-engineered.

The main node contains path nodes that contain operation nodes. Each operation node contains parameter and response nodes.

Test and reverse-engineer the API

You can test the HTTP REST API and reverse-engineer the data structure after checking the response.

To test the API response:

  1. Right-click an operation node. For example, the GET node.

  2. Select Actions > Reverse REST.

  3. In the REST Reverse dialog, define the API call: the parameters, headers, body, and authentication.

  4. Click getting started run test to call the API.
    The response appears in the Response section.

  5. Review the response, and click Next.

  6. In the Selective Reverse page, select the nodes that you want to reverse-engineer and click Finish.

Request parameters

In the Parameters section, you can define new parameters by clicking the getting started add button.

Value

The value is a test value that is used when performing a test request. It is not saved in the metadata.

Type

The following table lists the types of parameters:

Type Description

HEADER

Parameter added as an HTTP header. Headers are inputs defined in the metadata and can be set dynamically in the mappings that use the API.

QUERY

Parameter appended to the request URL.

PATH

Parameter replacing a user-defined mask in the request URL. When using such a parameter, set the mask to be replaced in the request URL.

FORM

With this option, the Allow Empty Value checkbox is automatically selected as its values are derived from the fields in the Input section.

In the Input section:

  • The Media Type field is automatically set to FORM and cannot be modified.

  • Set the Content-Type Header field to application/x-www-form-urlencoded.

COOKIE

With this option, the specified cookies are sent with each request; They are not persisted.

Request body

If the API call requires an body, select the associated type and fill in the body expression:

An option to link the selected Media Type with the Content-Type Header is available:

  • When the link is disabled, users can select Media Type and manually select the Content-Type Header.

  • When the link is enabled, after each Media Type selection, the Content-Type Header is automatically updated according to the selected Media Type. However, users can manually change the content type.

Request authentication

The Authentication section allows you to define the mechanism to authenticate to the API.

Field Description

Authentication Type

Authentication to use. Possible values are None, Basic, OAuth, OAuth2 and NTLM.

Use None or Basic if the API does not require any authentication.

Security Reference

Node in the HTTP security metadata containing the API authentication information.

Security Proxy Reference

Node in the Proxy metadata to use if the API call is redirected by a proxy.

This property does not define the proxy used for the API call, but the authentication that should be used if the request is redirected by a proxy.

Keystore Reference

Certificate and Keys metadata node to use as keystore reference.

Truststore Reference

Certificate and Keys metadata node to use as truststore reference.

Create mappings

Once the HTTP REST metadata is ready, you can use it in mappings.

In the following mapping, the columns of the source table are mapped to nodes in the Request Body of the POST method of the API. Nodes from the Responses are mapped to the columns in the target table. This mapping calls the API’s POST method for each source record, building the body with source columns, and loads the response’s content to the target table.

getting started mapping

Error management

Summary

The HTTP REST component provides parameters to manage both the functional and technical errors that can be encountered when making API calls.

The parameters described in this section are used to define the retry behaviors, the number of accepted errors, and more.

The parameters are available directly within the mapping, in the INTEGRATION Rdbms to HTTP Template.

Functional error parameters

Functional error acceptance number

A functional error happens when the target API is reachable but returns an HTTP code different than 200. (E.g. 404, 500, …​).

This parameter defines the number of functional errors tolerated before considering the data flow as a failure.

When a functional error is encountered, the data flow continues to perform the subsequent API calls until this number is reached. If this number is reached, the data flow stops and displays an error.

When this parameter is not set, the default value is -1 (no error limit).

The HTTP Error responses can be designed in the metadata and mapped into a target datastore.
An API call is considered a functional error if, after all the retries, it is still failing. See the functional retry number parameter.

Functional retry number

This parameter specifies the number of retry attempts performed when there is a functional error, before giving up and continuing with the next API call.

When the amount of retry is reached without succeeding, the API call is considered a functional error, and the runtime proceeds with the next API call.

The number of retries is independent of the functional error acceptance number parameter.
An API call is considered as a functional error if all the retries fail.

Functional retry delay

This parameter specifies the delay in milliseconds to wait between each retry attempt when there is a functional error.

Technical errors parameters

Technical error acceptance number

A technical error happens when the target API is unreachable when there is a technical error that avoids contacting the API.

This parameter defines the number of technical errors tolerated before considering the data flow as a failure.

When a technical error is encountered, the data flow continues to perform the subsequent API calls until this number is reached. If this number is reached, the data flow stops and displays an error.

When this parameter is not set, the default value is 0 (no error limit).

An API call is considered a technical error if, after all the retries, it is still failing. See technical retry number parameter.

Technical retry number

This parameter specifies the number of retry attempts performed when there is a technical error, before giving up and continuing with the next API call.

When the amount of retry is reached without succeeding, the API call is considered a technical error, and the runtime proceeds with the next API call.

The number of retries is independent of the technical error acceptance number parameter.
An API call is considered as a technical error if all the retries fail.

Technical retry delay

This parameter specifies the delay in milliseconds to wait between each retry attempt when there is a technical error.