Install and Run the Harvesting Client

The harvesting client is a docker image that runs recipes to collect metadata from sources and publish assets to Semarchy xDG.

System requirements

You need Docker to run the harvesting client. For installation instructions, see the Docker documentation.

To check if you have Docker Compose installed, run:

docker version

Install the harvesting client

Since the harvesting client a docker image, no additional installation is needed. Download the script to start it:

You may still use the former harvesting clients scripts data-intelligence-harvest.sh & data-intelligence-harvest.cmd.

Run a harvester

To run a harvester, first, configure a recipe and then run the following command:

Example 1. Run a harvester on Linux, Unix or macOS
./xdg-harvest.sh -c <recipe>.yaml
Example 2. Run a harvester on Windows
./xdg-harvest.cmd -c <recipe>.yaml
Prior to running a harvester, configure a recipe containing your source as well as a sink, with a personal access token to connect your Semarchy xDG site.
The harvester returns its execution results on the command line. You can also monitor your harvesting from the Semarchy xDG user interface.