Introduction

Infoworks Autonomous Data Engine

Eliminates Big Data Complexity Through Automation

The Challenge

Complexity limits the ability of the enterprises to capture the strategic value of their data

There is a consistent "buzz" today about how leading companies are harnessing big data for competitive advantage. Your organization is striving to become one of those market-leading companies. However, the reality is that over 80% of big data projects fail to deploy to production because project implementation is a complex, resource-intensive effort that takes months or even years. The technology is complicated, and the people who have the necessary skills are either extremely expensive or impossible to find.

Alternative approaches such as stitching together multiple point solutions or custom development are expensive, inflexible, time-consuming and require specialized skills to assemble and maintain.

The Solution

Automation is the key to eliminating complexity

The Infoworks Autonomous Data Engine (ADE) applies an unprecedented level of automation to data workflows and data engineering to eliminate big data complexity. The Infoworks ADE,

• Automates the complete data workflow from source to consumption
• Automates delivery of data to BI and advanced analytics applications
• Automates migration of data and workloads from legacy Data Warehouse systems to big data platforms
• Automates orchestration and management of complex data pipelines in production.

The Infoworks ADE is the most efficient and agile solution to implement enterprise big data use cases.

Automation delivers tremendous value to your business across a wide variety of factors.

Agility

• Deploy new use cases in days....not months
• Add new data sources with a single click

Flexibility

• Choose any analytics tools, methods, or algorithms
• Choose any big data platform, on-premise or cloud

Operating Efficiency

• Focus resources on generating business value
• Decrease demand on IT through self-service
• Improve reliability in production operation of analytics use cases

Cost Efficiency

• Reduce cost and time of building and maintaining analytics use cases
• Reduce cost associated with legacy DW's

Infoworks in Action

The Infoworks ADE has been deployed in production by large enterprises to run business-critical applications. Infoworks customers have successfully implemented complex, large-scale use-cases in days instead of months with minimal resources. Some examples of these successes are:

Fortune 10 Retailer: Advanced Data Application

Implemented near-real-time, machine learning business process in 19 days:

• Synchronized business process data from Teradata every 10 mins
• Achieved a data availability SLA of 15 minutes
• Implemented by 2 engineers in 19 days from requirements to production

Leading CPG Company: Self-Service BI and Analytics

Reduced development cycle from 6 months to 1 week:

• 7 data sources, 3 years of production data
• 8 pipelines with all transformation logic
• 8 optimized data models and 3 cubes
• 13 reports and dashboards

Infoworks Platform

Infoworks provides a complete solution that automates end-to-end data workflows from source to consumption as well as automating the ongoing operational management of those workflows.

Infoworks delivers automated capabilities.

Automated Workload Migration

Automates migration of workloads (ETL logic, BTEQ in Teradata, SQL workloads, and other such programs) from legacy data warehouses to a big data environment. With automated data, schema and workload migration, the Infoworks ADE provides a comprehensive solution for data warehouse offload and migration.

Automated Data Ingestion and Synchronization

Data Source Crawling and Ingestion

Automatically crawls data sources, ranging from flat files, XML, JSON to relational databases such as Teradata, Oracle, and SQL Server.

Google crawls the web to get web data; likewise, the Infoworks ADE crawls data sources and ingests source data in a high-performance parallel process, while automatically preserving data precision.

Metadata Synchronization

Learns the metadata and infers data relationships for the data ingested from external data sources as well as data sets created using Infoworks. It also tracks end-to-end data lineage so that users can trace data elements back to the original source systems and perform downstream impact analysis.

Data Synchronization

Continuously synchronizes source data from enterprise databases, data warehouses, and file sources. Changing data is captured from the source systems using log-based and query-based methods. The changed data is merged with the base data in a high-performance continuous merge process.

• Automatically handles slow-changing-data and schema changes and creates current and historical tables.
• Supports export functionality to other enterprise's operational and data warehouse systems.
• Supports streaming, batch and incremental mode of data synchronization and export.

Data Transformation and Pipeline Design

Provides self-service data preparation using an interactive, drag-and-drop data transformation capability with support for SQL-based and other transformations. Users work with data in a collaborative, suggestion-based interface that reduces or eliminates dependence on IT skills.

Data Models, Cubes and Memory Models

Builds target models including in-memory models that are automatically optimized for fast access. Visually design star schemas, and automatically build high-performance OLAP cubes accessible from industry standard tools such as Tableau, Microsoft Power BI, MicroStrategy, etc.

Advanced Analytics Integration

Integrates data pipelines with advanced analytics algorithms from libraries such as SparkML & R, with no need for coding. Builds trained models or import pre-trained models into data pipelines.

Orchestration and Production Operations Management

Designs end-to-end work-flows and orchestrate in production with fault-tolerant, distributed execution. Migrates from development environments to production across big data or cloud platforms with single-click operations.

Portability

Infoworks automation also makes it easy to move from an on-premise Hadoop platform to the cloud, or from one cloud environment to another. One Infoworks customer moved an entire set of production workflows from Microsoft Azure to Google Cloud Platform in less than a day.

Enterprise-Grade Security Integration

The Infoworks ADE provides security integration for user authentication and data security policies. It supports Single-sign-on/LDAP integration, Kerberos based authorization. It supports encryption for data in motion and at rest.

Demo

See the Demo for a brief understanding of the Infoworks ADE functionalities.