Job Management

Canceling Job

You can manually cancel the process of Ingestion, Recrawl Table Data, or Recrawl Table Metadata.

Following are the steps to cancel a job:

• Click the Ingestion Logs icon on the left menu. The Ingestion Logs page opens listing out the jobs that are in process and the jobs that have been completed.

• Click the job that is in progress (the record with blue circle in the Result column).
• Click cancel job on the right of the Ingestion Logs

Audit Logs

The Infoworks DataFoundry tracks the source changes in the Entity Audit Collection. Logs are created for all updates on the source and advance configurations of the source. It contains the information about the action, user, and timestamp.

To view the audit logs, click the Audit Logs collapsable menu in the Source Settings page. The changes performed on the source are automatically updated in this table.

NOTE: This screen displays only the changed properties. If a property was left unchanged in this iteration, it will not be displayed as an update, but the property will be stored.

Scheduling Ingestion Jobs

Prerequisite

Ensure you specify the scheduler username and password in the Source Scheduler settings (Source > Scheduler).

Procedure

Following is the procedure to schedule a table group job:

• Navigate to the required source.
• Click the Table Groups tab.
• Click the Schedule icon for the required table group.
• In the Schedules page, select the Ingestion Schedule Type. The options include the following: No Schedule, Combined Schedule, Different Schedules.

Combined Schedule: The mechanism by which the ingest job can be scheduled for a particular table group and frequency. This can be used when the data must be refreshed periodically, incrementally or fully, based on the table.

Different Schedule: The mechanism by which the different processes that are part of the ingestion job (cdc, merge, switch) can be scheduled independent of each other. This provides more flexibility during tight SLAs. For example, if a source system has a specific time window to fetch data, the CDC jobs can be scheduled at that particular time. After multiple CDC jobs, a merge job can be scheduled to merge the data fetched from multiple CDCs to the table in Hive.

• In theData Expiry Schedulesection, clickEditto schedule the data expiry.
• In theExport Table Schedulesection, clickEditto schedule the table export.