Getting Started with Couchbase Database

Overview

This article explains how to work with Couchbase in Semarchy xDI.

Connect to your Data

Metadata creation

The first step is to create the Couchbase Metadata.

  1. Launch the Metadata creation wizard, then select Couchbase.

  2. Select Couchbase Module.

  3. Click Finish.

  4. Then, set the properties of the root node.

Property Description

Name

A label/alias for this cluster.

Connection String

The Couchbase connection string is a comma-delimited list of IP addresses and/or hostnames.

Examples:

127.0.0.1
nodeA.example.com,nodeB.example.com
nodeA.example.com:11210,nodeB.example.com:11210

The optional connection string parameters cannot be used.

This is because the Couchbase Java SDK ignores them when connecting with a configured cluster environment, which we are currently doing.

Authentication Method

The method used to authenticate with Couchbase.

User

The Couchbase user to connect with.

Password

The Couchbase password to connect with.

Only Enable Plain Sasl Mechanism

Enables only the PLAIN authentication mechanism, which is used with LDAP.

When the Couchbase cluster is configured to use LDAP on the backend, depending on the configuration, you may need to enable this parameter for the authentication to work.

Certificate Path

Path to the certificate file to be used when Couchbase is secured with TLS.

Reverse-engineering Sample Size

The number of sample documents to parse to reverse-engineer the structure of documents.

Reverse-engineering Automatic Index Creation Method

This defines if indexes should be created before performing reverse-engineering.

Having indexes is mandatory to perform reverse-engineering.

You can use this parameter to automatically create the indexes if they do not exist yet on the items you want to reverse-engineer.

Reverse-engineering Automatic Index Cleanup

This defines how to manage the cleanup of the indexes that have been automatically created during reverse-engineering.

This applies only when using the "Reverse-engineering Automatic Index Creation Method".

Below is an example:

getting started couchbase metadata overview

Metadata Reverse-engineering

Each node of the Couchbase metadata offers the possibility to reverse-engineer its structure.

Right-click the desired node, then select > Actions > Reverse.

  • On the cluster node: All the buckets, scopes, collections, and underlying documents are reverse-engineered.

  • On a bucket node: All the scopes, collections, and underlying documents are reverse-engineered.

  • On a scope node: All the collections and underlying documents are reverse-engineered.

  • On a collection node: All the underlying documents are reverse-engineered.

Define a bucket node

To add a bucket node manually, right-click the server node, then select New > Bucket.

The following properties are available on this node:

Property Description

Name

A label/ alias for the bucket.

Database Physical Name

Bucket’s physical name, as defined on the cluster.
You can reverse all the underlying scopes, collections, and documents. Right-click the bucket node, then select Actions > Reverse.

Define a scope node

To add a scope node manually, right-click a bucket node, then select New > Scope.

Property Description

Name

A label/ alias for the scope.

Scope Physical Name

Scope’s physical name, as defined on the cluster.
You can reverse all the underlying collections and documents. Right-click the scope node, choose Actions, then Reverse.

Define a collection node

To add a collection node manually, right-click a scope node, then select New > Scope.

Property Description

Name

A label/ alias for the collection.

Scope Physical Name

Collection’s physical name, as defined on the cluster.
You can reverse all the underlying documents. Right-click the collection node, then select Actions > Reverse.

Define an FTS Query node

A Full-Text Search (FTS) Query node is a placeholder for a Couchbase FTS query.

To define an FTS Query:

  1. Right-click the root node.

  2. Select New > FTS Query.

  3. Specify the query properties.

  4. Optionally, add the limit and skip fields to define the pagination. The limit field allows you to specify the number of results to retrieve. The skip field allows you to specfiy the number of results to skip before the retrieving the results.

  5. Right-click the FTS Query node, select Actions > Reverse. This runs the query and parses the response to get the metadata structure. This is optional. The results can also be designed manually.

Example:

getting started couchbase metadata fts query common

The following properties are available:

Property Description

Name

Label for the FTS query.

Index

Name of the index to use for performing the FTS.

Query Type

Defines the FTS query type.

Currently, only the queryString type is supported.

Fields

Comma-delimited list of fields to retrieve in the result of the FTS.

Use a wildcard "*" to retrieve all the fields if it is supported by the defined index.

The fields returned depend on how the index is created and configured. Make sure that your index is correctly configured if the reverse-engineering of the FTS query does not return the fields you specified in the metadata.

Query String

The FTS Query String to be used.
It must match the Couchbase syntax.

Define a N1QL Query node

A N1QL Query node is a placeholder for a Couchbase N1QL query.

To define a N1QL query:

  1. Right-click the root node.

  2. Select New > N1QL Query.

  3. Specify a name for the query.

  4. Click the N1QL Query field to enable edition.

  5. Write the query in the multiline text field.

  6. Right-click the N1QL Query node, select Actions > Reverse N1QL Query. This runs the query and parses the response to get the metadata structure.

Create your first mappings

Write data into Couchbase

Couchbase collections can be loaded from any database.

  1. Create a Mapping.

  2. Drag and drop your source database table.

  3. Drag and drop your target Couchbase document.

  4. Map the fields between the source and the target.

  5. Define the 'Couchbase Operation' of the Couchbase Template to the expected value.

Read data from Couchbase

You can read data from Couchbase and load it into a target table.

There are two ways to do this:

  • Read documents based on their ID.

  • Read the result of an FTS query.

Read documents

To read documents based on their ID:

  1. Create a Mapping

  2. Drag and drop:

    1. The source table that will be used to iterate the Couchbase FTS Query.

    2. The document to read.

    3. The target table to load data into.

  3. Map the root node of the Couchbase document/ FTS Query from the source.

  4. Map the ID field with the ID of the documents to retrieve.

  5. Map the fields between Couchbase and the target table as usual.

  6. Define the 'Couchbase Operation' of the Couchbase Template to get.

Example:

getting started couchbase read document example

Refer to the Template and parameters descriptions for further details.
The source table calling Couchbase is used for the repetition key. Couchbase will be called once for each source row.

Read an FTS query

To read the result of an FTS query:

  1. Create a Mapping.

  2. Drag and drop:

    1. The source table that will be used to iterate the Couchbase documents to retrieve.

    2. The FTS Query to read.

    3. The target table to load data into.

  3. Map the root node of the Couchbase FTS Query from the source.

  4. Optionally map the query parameters.

  5. Map the fields between Couchbase and the target table as usual

Example:

getting started couchbase read fts query example

Refer to the Template and parameters descriptions for further details
The source table calling Couchbase is used for the repetition key. Couchbase will be called once for each source row.

Read a N1QL query

To read the result of a N1QL query:

  1. Create a Mapping.

  2. Drag and drop:

    1. The source table that will be used to iterate the Couchbase N1QL query.

    2. The N1QL Query to read.

    3. The target table to load data into.

  3. Map the root node of the Couchbase N1QL query from the source.

  4. Optionally map the query parameters.

  5. Map the fields between Couchbase and the target table as usual.

Refer to the Template and parameters descriptions for further details
The source table calling Couchbase is used for the repetition key. Couchbase will be called once for each source row.