Hadoop component release notes
This page lists the main features added to the Hadoop Component.
New Features
Version 2023.1.0
Ability to choose Hive table type
The Hive metadata now supports defining the Hive Table Type, which can be EXTERNAL or MANAGED.
The default type is EXTERNAL, which requires a specific configuration in the metadata. Update your hive metadata to define the external storage location, or change the type of table to MANAGED depending on your configuration.
Changelog
Version 2023.1.0
Breaking changes
-
DI-5390: The metadata now supports defining the Hive Table Type, which can be EXTERNAL or MANAGED.
The default type is EXTERNAL, which requires a specific configuration in the metadata Update your hive metadata to define the external storage location, or change the type of table to MANAGED depending on your configuration.
See External and managed tables.
Features
-
DI-5312: The metadata now supports hexadecimal properties.
-
DI-5389: The metadata reverse-engineering has been improved.
-
DI-5653: Log4j version 1 has been removed from the dependencies.
-
DI-5817: Multiple third-party libraries upgarde.
-
DI-6225: The Post Processing Operation option has been added to the Load Xml To Hive template.
Version 3.0.0 (component pack)
Features
-
DI-4053: Query Editor menu renamed to "Launch Query Editor"
-
DI-4508: Update Components and Designer to take into account dedicated license permissions
-
DI-4727: Rebranding: Templates and sample projects
-
DI-4731: Rebranding: Template messages
-
DI-4813: Rebranding: Drivers classes and URLs
-
DI-4962: Improved component dependencies and requirements management
Version 2.0.3 (Hadoop component)
Features
-
DI-1775: New template Load Salesforce to Hive
-
DI-1777: TOOL Hdfs File Connect - an "HDFS Error" session variable is now published when a connection fails, providing more detailed information about the error
-
DI-1910: Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet
-
DI-1909: Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking
Bug fixes
-
DI-1729: HDFS File Get Process Tool - Module specification was missing when using WebHDFS mode, which was causing errors as the required dependencies could not be found
-
DI-1908: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates on Lock / Unlock CDC steps
-
DI-1907: Templates updated - The 'Cdc Subscriber' parameter was ignored in some Templates when querying the source data