Release Notes 2.7.0

Date: 01 APR 2019

New Features and Enhancements

Component: Data Ingestion and Synchronization

IPD-6335 - Redshift Ingestion: Infoworks now supports ingestion from Redshift database into the data lake.
IPD-6385 - Teradata Wallet Integration for TPT Ingestion: Teradata wallet integration allows user to retrieve the password from the wallet by providing the Teradata wallet key. Password-less authentication can now be implemented for all TPT crawl jobs.
IPD-6194 - Fixed Width Enhancements: Start Position and width of columns can be defined for fixed width file layout. Extraction of multiple columns with overlapping position is now supported. Datatypes for the columns are also recommended now.
IPD-5204 - Datatype Mapping in RDBMS Ingestion: In RDBMS ingestion, the target datatypes of columns can now be modified by the user during the ingestion itself, saving unnecessary datatype mapping pipelines. The precision and scale of Decimal datatype can also be modified.
IPD-7013 - Case-Sensitive Schema Names: All RDBMS schema names are now case-sensitive.
IPD-7015 - Oracle with RAC Setup: Log based incremental ingestion now supports Oracle with RAC setup.

Component: Data Transformation

IPD-6383 - Custom Target: Pipeline extensions can now be added to customize targets. This ensures customization of code to support various types of targets other than Hive.
IPD-6386 - Lookup Node: A data transformation node, Lookup, has been added which can be used to fetch column information (derived column) from the lookup source based on the rule that applies to each input record.
IPD-6207 - Undo and Redo Pipeline Changes: Support has been added to undo and redo changes made in the active pipeline session.
IPD-6178 - Multiple Pipeline Import: Support has been added to import multiple sequential pipelines with export configuration.
IPD-6419 - Split Advanced Mode: The Advanced Mode in Split node allows user to add expression without wrapper functions. The users can now add UDTF functions in split node.

Component: Orchestration

IPD-6230 - Support for Sub-workflow: Users can now trigger a workflow within a workflow using the Run Workflow node. This ensures reusability of workflows, and
IPD-6552 - Export Node in Workflow: A workflow node, Export, has been added which allows users to perform export job as part of the workflow orchestration.
IPD-5233 - EMR Scale in Workflow: A workflow node, EMR Scale, has been added which allows users to scale up or scale down the number of instances existing in node groups of EMR clusters.

Component: Data Export

IPD-6554 - BigQuery Datatype Mapping: BigQuery export supports limited datatype mapping. The column types can now be remapped by using the bq_col_overrides configuration at table-level or pipeline-level.

Component: Data Catalog

IPD-6422 - Data Catalog: A new component data catalog has been added which enables you to search data in a single section regardless of access controls.

Component: Cloud

IPD-6517 - RDBMS Support for ECB: Teradata and Oracle databases are now supported in ECB.

Component: Platform

IPD-6091 - Start/Stop Streaming API: The following APIs have been added to start and stop streaming job respectively on a source: Start streaming, Stop streaming.
IPD-5500 - Enhanced HA Support: New platform services have been added. Following platform services support Active-Active configuration: Security Service, MetaDB Service, Notification Service, Messaging Service, RabbitMQ. Following platform services support Active-Passive configuration: Postgres.

NOTE: During failover, user must manually modify the slave postgres configuration to act as master.

IPD-5252 - Notification Service: Users will now be notified via email and slack for Ingestion, Data Transformation and Cube jobs. Clicking the bell icon in the menu bar displays the job notifications.

Java 7 is not supported from this version (Infoworks ADE 2.7.0).

NOTES:

If LDAP is enabled with SSL, the server certificate must be added in the JAVA Keystore. Following is the command to add certificate to the keystore: keytool -import -keystore {JAVA_HOME}/jre/lib/security/cacerts -alias <ldap-server> -file /path/to/crt/file
For any existing schedule for a table group, the schedule set under Different Schedules > Delta Merge Schedule with value Run after CDC will be deleted from the metadata. This option is no longer available, CDC and merge must be scheduled separately.

Limitations

IPD-7060 - Mapping for Interval Datatype Column as Timestamp: To support interval datatype column as timestamp, after upgrading to the Infoworks ADE 2.7.0 version, user must add mapping in the $IW_HOME/conf/mappings/netezzaMappings file manually as INTERVAL=Long.

Known Issues

IPD-7128 - Incorrect API Response: When an internal server error occurs, the start/stop streaming API might not return correct error details in the response.

Installation

Refer Installation to install Infoworks ADE 2.7.0/2.7.0.1.

Upgrading to This Release

To upgrade your current Infoworks ADE version, execute the following commands on the edge node:

NOTE: Before starting the upgrade, ensure that all Infoworks services are running and no Infoworks jobs are running.

Navigate to the bin directory using the following command: cd $IW_HOME/bin; where, $IW_HOME is the directory where the Infoworks ADE is installed.
Download the update script using the following command: wget <link-to-download>
Reach out to your Infoworks representative to get the link to download.
Upgrade the Infoworks ADE version using the following command: bash update.sh <version>; ensure you specify the target version. For example, bash update.sh 2.7.0.

Release Notes 2.7.0.1

Date: 17 APR 2019

New Features and Enhancements

Component: Data Ingestion and Synchronization

IPD-7289 - Data Export to Azure SQL Data Warehouse: Support to export data to Azure SQL Data Warehouse has been added.

NOTE: When using Azure SQL Data Warehouse with Active Directory - Password authentication for export, ensure to add the $IW_HOME/lib/sqlserver_az_ad/ path to iw_export_classpath in the $IW_HOME/conf/conf.properties file.

Release Notes 2.7.0.2

Date: 30 APR 2019

New Features and Enhancements

Component: User Interface

IPD-7392 - Edit Schema Page Issue: Edit schema page for some tables was not visible when the source had 500+ tables, this has now been fixed

Last updated on