Consistency Guarantees

Batch Replication

Once the batch replication is completed successfully, it guarantees that if a table exists in the metastore, it is consistent with the state of the source table at some point after batch replication was kicked off. In general, user must wait for batch replication to finish running before running any queries on the destination warehouse.

Incremental Replication

Incremental replication provides eventual consistency. Incremental replication also guarantees that data directories never contain partial data. When incremental replication is running and there are updates to a table on the source warehouse, tables on the destination warehouse will either contain old data, new data, but never a partial result. These are the same semantics that Hive provides when overwriting a table with a query. In addition, incremental replication guarantees that data for a table will be copied before the metadata, so if a table is present in the metastore, the table can be queried.