Spark Component
Overview
Semarchy xDM Data Integration allows to work with Spark to produce fully customized Data Flows.
Install the Spark Component
The Spark Component is installed from Semarchy xDI Designer using the component installation feature.
Supported Features
Spark 2
Feature | Description |
---|---|
LOAD |
Data can be loaded to Spark: HBase, HDFS, Hive, RBDMS, Vertica, Parquet, Elasticsearch |
INTEGRATE |
Data can in integrated from Spark: HDFS , Hive, RDBMS |
STAGE |
Spark Metadata can be used as a stage (between loading and integration) to boost Hadoop Mappings. |