I need to understand the difference between Full and Complex Delta Load in MDM.
As per this document on Complex Delta Load, Delta Dependency Mode needs to be setup for each entity. I see there are two options for it -Autonomous and Central. Can someone please explain the difference between these and in what scenario these should be used?
I also see that The engine expects an additional column in the input interface denoting the type of record change (change_type: I
- insert, U
- update, D
- delete). Where shall we define these and what how can we identify which records needs which change_type?
TIA
Difference between Full Load and Complex Delta Load
Hi
This document describes autonomous, central, and dependent entities.
The difference between full loads and delta loads is as follows:
Full Loads
- Expect the entire dataset as input for a given source.
- Missing records are considered deleted.
- Remaining records are classified as new, updated, or unchanged.
- Easier to configure.
- Requires transferring the entire dataset, which can be problematic for larger implementations or micro-batching.
Delta Loads
- Expect only a subset of the data as input, with a change type (insert/update/delete) for each record.
- Suitable if your source provides change data capture (CDC) or you have ETL infrastructure (e.g., data lake or ODS) with CDC capabilities.
Reply
Login to the Ataccama Community
No account yet? Create an account
For Ataccama Customers and Partners
or
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.