Spark Component


Semarchy xDI allows to work with Spark to produce fully customized Data Flows.

Install the Spark Component

The Spark Component is installed from Semarchy xDI Designer using the component installation feature.

Supported Features

Spark 2

Feature Description


Data can be loaded to Spark: HBase, HDFS, Hive, RBDMS, Vertica, Parquet, Elasticsearch

Data can also be loaded from Spark: Hive, RDBMS, Vertica, Parquet, Elasticsearch


Data can in integrated from Spark: HDFS , Hive, RDBMS


Spark Metadata can be used as a stage (between loading and integration) to boost Hadoop Mappings.

Spark Stage can be: SQL, Java