Google BigQuery Component Release Notes
This page lists the main features added to the Google BigQuery Component.
Change Log
Version 2023.3.0
Feature improvements
-
DI-5987: New parameters connectionTimeout and readTimeout can be defined in the JDBC URL.
-
DI-6915: New template DESERIALIZER BigQuery String as Json.
-
DI-7270: Target filters have been implemented in the INTEGRATION BigQuery template.
Bug fixes
-
DI-6228: LOAD GCS File to BigQuery: The error
NullPointerException
is thrown on the "select on source data" step when loading a file to bigquery with direct mode. -
DI-7893: The error
Access Denied
is displayed when accessing an external table in BigQuery that points to a Google Drive Spreadsheet.
Version 2023.1.0
Feature improvements
-
DI-4919: The location used to run queries can now be changed using the
jobsLocation
parameter in the JDBC URL. -
DI-5839: A retry mode has bee added to re-run the query when an
internalerror
orjobBackendError
error message is returned. -
DI-6041: Detailed information about the error is displayed when a Google Big Query job fails.
-
DI-6140: Multiple third-party libraries upgrade.
-
DI-6244: The Support DEFAULT constraint option is set to false by default on all datatypes on BigQuery.
Bug fixes
-
DI-3067: The Create table script action fails on a Bigquery Table.
-
DI-4955: In the Reject BigQuery template, when checking foreign keys, the
EXIST
clause is used instead of theJOIN
clause. -
DI-4956: In the Reject BigQuery template, null values are not taken into account when checking the uniqueness of alternative keys.
-
DI-5367: In the BigQuery Integration Template, Reject Recycling is ignored when working with partitioned tables.
-
DI-5475: The date and timestamp formats that can be specified on the SQL File Export action are ignored when exporting data from Google BigQuery.
-
DI-5717: The INTEGRATION BigQuery template fails when the source contains the
<
or>
characters in filters or joins and when the enableTargetPartitionPruning parameter is set totrue
. -
DI-5827: When mapping an SQL Server date column having a
NULL
value to a Google BigQuery column, the NullPointerException error is thrown. -
DI-6277: Multiple third-party libraries upgrade.
Version 5.3.12 (Component Pack)
Bug fixes
-
DI-7893: An
Access Denied
error is displayed when accessing an external table in BigQuery that points to a Google Drive spreadsheet. -
DI-7981: Conversion issues with INT64, FLOAT64, BIGNUMERIC, DATETIME, and TIME data types cause
SQLFeatureNotSupportedException
errors when using Google BigQuery as a source. -
DI-8207: Google BigQuery mappings that use Google storage as a temporary location fail unexpectedly because of permission issues.
-
DI-8799: Error when parsing datetime fields.
Version 3.0.1 (Component Pack)
Version 3.0.0 (Component Pack)
Feature improvements
-
DI-2530: Added automatic reverse of partitioned tables when reversing BigQuery tables
-
DI-2795: Google BigQuery: Failing queries return information beyond the failed Job ID if available.
-
DI-3848: Google BigQuery partitions: Add "Year" and "Month" to the list of "Timestamp truncation" values.
-
DI-4508: Update Components and Designer to take into account dedicated license permissions
-
DI-4731: Rebranding: Template messages
-
DI-4813: Rebranding: Drivers classes and URLs
-
DI-4962: Improved component dependencies and requirements management
Version 2.2.6 (Google BigQuery Component)
Version 2.2.1 (Google BigQuery Component)
Feature improvements
-
DI-4453: Template - INTEGRATION BigQuery - Expressions executed on target where not supported with merge incremental method
-
DI-4455: Template - INTEGRATION BigQuery - New parameter "Deletion Strategy"
-
DI-4456: Template - INTEGRATION BigQuery - New parameter "Truncate Target Table"
-
DI-4457: Template - INTEGRATION BigQuery - Create as select is now used to create integration table
-
DI-4458: Template - INTEGRATION BigQuery - New parameters "Enable Target Partition Pruning" and "Target Partition Filter"
-
DI-4460: Template - INTEGRATION BigQuery - Change partition boundaries calculation method so it executes once in a Mapping
-
DI-4461: Template - INTEGRATION BigQuery - When using a merge, a comparison is made on 'Non target' fields to minimize and optimize the number of updated records
-
DI-4462: Template - INTEGRATION BigQuery - remove option 'Useless Update Detection Method' which is not used anymore
Version 2.2.0 (Google BigQuery Component)
Bug fixes
-
DI-1209: Template - INTEGRATION BigQuery - handle nullity in update key when performing updates or comparison with target table
-
DI-3331: Google Cloud Storage and Google BigQuery wizards - When launching the wizard on an existing Metadata, the first step of the wizard did not automatically select the account defined in Metadata
Version 2.1.2 (Google BigQuery Component)
Feature improvements
-
DI-4297: Template - Reject BigQuery - when using "Drop Reject Table" parameter, the drop and re-creation of the reject table is now performed in one query using REPLACE or IF NOT EXISTS clauses for performances and optimizations purposes
Bug fixes
-
DI-4285: Template - Integration BigQuery - sometimes the preintegration table is unexpectedly created and populated even if it is not used later in the Template
-
DI-4298: Template - Reject BigQuery - when both "Drop Reject Tables" and "Delete Previous Rejects" parameters are enabled at the same time, the delete of previous rejects should not be performed because it is not necessary, as the table is recreated empty
Version 2.1.0 (Google BigQuery Component)
Feature improvements
-
DI-3510: EMF compare utility - Component has been updated to support EMF Compare comparison utility
Bug fixes
-
DI-1245: Google BigQuery Templates - when loading data into BigQuery, some Templates were not using the correct working schema
-
DI-2378: Template - Load Rdbms to Bigquery - an error such as "No such field: L1" was thrown when there was multiple time this Load Template for the same target datastore (parallelism was not handled properly)
-
DI-1165: Template - "INTEGRATION Bigquery" and "Load Rdbms to BigQuery" - SQL_STAT_INSERT and SQL_STAT_UPDATE statistics were not computed and not displayed
-
DI-1277: Google BigQuery - When loading DATE (or TIME) type from Bigquery into some databases (such as PostgreSQL) an error such as the following could unexpectedly be thrown "l1_c_time is of type time without time zone but expression is of type character varying"
-
DI-3833: Google BigQuery - Randomly, the "EngineExceptionI: A class was not found in the engine" error was displayed even when the Module was complete, because there was an unexpected jar in the Module which caused conflicts, and which has now been removed
Version 2.0.9 (Google BigQuery Component)
Version 2.0.8 (Google BigQuery Component)
Feature improvements
-
DI-1009: Template - INTEGRATION BigQuery (Hierarchical) BETA - New Template to write into repeated / nested fields
-
DI-1052: The isRepeated attribute was not reversed in the metadata, to use repeated fields.
Bug fixes
-
DI-3407: Templates - "LOAD GCS File to BigQuery" and "LOAD GCS JSON to BigQuery" Templates were throwing an error when using credentials string - WrappedException: Wrapped java.lang.NumberFormatException: For input string: "{"
-
DI-3409: Templates - "Load GCS File to BigQuery" and "Load GCS Json to BigQuery" Templates - an exception was thrown when using credentials "string" authentication method (Null Pointer in Method Invocation BSF info)
-
DI-3480: BigQuery Metadata - Java Type used for NUMERIC columns was incorrect
-
DI-3491: When reversing a BigQuery table with multiple records some of the records (not all of them) was reversed on the wrong level.
-
DI-3565: Template - INTEGRATION BigQuery - SQL_STAT_INSERT and SQL_STAT_UPDATE statistics were not displayed
Version 2.0.6 (Google BigQuery Component)
Feature improvements
-
DI-1781: BigQuery - Support using constant values in GROUP BY clauses by automatically using aliases
-
DI-1944: Support a new authentication mode named "applicationDefault" which searches for authentication credentials in the current environment automatically through default Google’s mechanism
-
DI-1912: Templates updated - support having CDC sources on Templates which were not supporting it (such as staging templates)
-
DI-1910: Templates updated - New parameter 'Cdc Subscriber' on Templates on which it was not handled yet
-
DI-1909: Templates updated - New Parameters 'Unlock Cdc Table' and 'Lock Cdc Table' to configure the behaviour of CDC tables locking