Data Catalog

Data Catalog enables you to search data in a single section regardless of access controls. It displays a catalog of the following:

all data sources that have been crawled, ingested and synchronized
all data pipeline targets that have been created using transformation pipelines
all cubes and accelerated data models created using Infoworks

Global Search

Click the Data Catalog menu. The data catalog page is displayed. This page allows you to search data by Name, Description, Tags, Categories (source, table, cubes) and Favorites. A global search will be performed on all data sources available in the sources collection.

Enter the data to be searched and click the search icon. A list of records matching the search keyword is displayed.

In the search result, click the Name of the required data. You will be redirected to the details page.

Following are the searches supported:

Search by Name

A regex search is performed on the name property of the sources collection and table property on the tables collection.

Search by Description

A regex search is performed on the description provided for the sources, tables and pipelines.

Search by Tag

A regex search is performed on the tags label property of the tags collection, and the sources and tables associated with this tag label are displayed.

Search by Category

During global search, you can filter the data by the following categories:

Data Sources: Includes all sources in the sources collection regardless of access control.
Data Models: Includes all tables in the tables collection regardless of access control.
Accelerated Data Models: Includes all cubes associated with a pipeline.
Favorites: See Search by Favorites

Search by Favorites

During global search, you can filter data by favorites, which includes sources and tables.

Source Detail View

Clicking the source displays a detailed view of the source.

It includes the the following details:

Configure Source Link

Data Catalog allows you to only view data. To edit or configure the data source, click the edit icon next to the name of the Source. You will be redirected to the sources settings page.

Global Search in Data Source

You can perform a global search from the Data Sources page. A regex search is performed on the name, table, description, tags.label properties. This search does not include search by categories and favorites.

Description Field

The Description section displays the following details:

Description: Description for the Source.
Owner: User who created the source.
Status: Parsed from the source state property.
Record Count: Number of tables associated with the source.
Last Refreshed: Last modified time of the source.
New Records Added: Number of tables added in the previous week.

Favorite

Click the star icon to add the source as favorite. A black colored star represents the source has been added as favorite.

Tags

This option allows you to add, remove and view tags associated with the source.
Add Tag: Click the + icon, enter the new tag and click Enter.
Remove Tag: Click the x icon to remove the tag from the source.

Tables

List of all tables associated with the source.
Click the name of a table to navigate to the table detail view.

Table Detail View

Clicking the table displays a detailed view of the table.

Configure Table Link

Clicking the edit icon navigates to the source configuration page if the table is associated with a source or navigates to the pipeline configuration page if the table is associated with a pipeline.

Navigation Tabs

You can view the schema, sample data, data and lineage of the table by clicking the respective tabs.

Description Field

The Description section displays the following details:

Status: state property of the table.
Columns: length of the columns property of the table
Records: rowCount property of the table.
Last Refreshed: last_job_execution_time property of the table.
New Records Added (Week): number of rows that have been added in the previous week.

Last updated on