Getting started with Google Cloud Storage

Overview

This getting started gives some clues to start working with Google Cloud Storage

Prerequisites

You must have created before a Google Cloud Project Metadata, which contains the account and credentials to connect to Google Cloud Platform.

This is mandatory, Google Cloud Storage Metadata will use it to gather the account and credentials.

Metadata Configuration

Creation

To create a Google Cloud Storage Metadata, launch the Metadata creation wizard, select the Google Cloud Storage Metadata in the list and follow the wizard.

The wizard will ask you to choose the credentials to use, with a list of all credentials defined in your workspace. If the list is empty, make sure that you have read carefully the prerequisites.

getting started google cloud storage metadata credentials

Buckets definition

Credentials being configured, you can now start defining Buckets nodes in your Metadata.

This is just a Metadata definition, the Buckets does not necessarily need to exist physically on Google Cloud Storage, as the Component allows to create or delete them.

To add a new Bucket node, right click on the root node and choose New  Bucket

getting started google cloud storage metadata bucket 1

You can then define a logical name for this Bucket, and the real Bucket Name that will be used on Google Cloud Storage.

There is several additional storage information that can be defined in the standard and advanced tab, which allows to define more precisely the Bucket’s properties, such as its location, its storage class, its retention period, …​

Note that those additional information will only be used when trying to create the corresponding Bucket with the Process Tool allowing the create the Bucket, and that when they are left empty, the default value from Google API is used.

getting started google cloud storage metadata bucket props

You can also define Bucket labels and Bucket Life Cycle rules which should be used to configure the Bucket at Bucket creation.

getting started google cloud storage metadata bucket rules

Finally, you can define 'Directories' which will be used to store and retrieve Blobs from Buckets.

This is optional and corresponds to the prefix that can be added on Blobs to organize them in virtual 'Directories'.

This will be used to filter the list of Blobs to retrieve and to create Blobs within a specific path.

getting started google cloud storage metadata bucket folders

you can also perform get or put operations directly on Bucket’s root. The folders defined in Metadata can help you to get organised.

Perform bucket operations

Several Process Tools are available to perform bucket operations.

You can find them in the Process Palette and use them as any other Process Tool.

  1. Create a Process

  2. From the Process Palette, add the Google Cloud Storage Process Tool you want to use

  3. Drag and Drop on it the required Metadata Link

  4. Define the parameters accordingly to your requirements.

Below, a quick example to create a bucket:

getting started google cloud storage example animation

Below, some other examples.

Create a Bucket:

getting started google cloud storage example 1

Send a bunch of files into a Bucket

getting started google cloud storage example 2

Retrieve a bunch of files from a Bucket

getting started google cloud storage example 3

Retrieve a filtered list of files from a Bucket, filtered using Blob Metadata Properties

getting started google cloud storage example 4

Copy Blobs from one folder to another folder

getting started google cloud storage example 5

Sample project

The Google Cloud Platform component is distributed with sample projects that contain various examples and files. Use these projects to better understand how the component works, and to get a head start on implementing it in your projects.

Refer to Install components in Semarchy xDI Designer to learn about importing sample projects.