Hi @RandomStranger37,
When it comes to survivorship in MDM, it revolves around the Source_ID column. So, it's basically an UPSERT. Once MDM detects that the Source_ID does not exist in the table, it will create a record with that Source_ID, if it detects that the Source_ID is there, then it will do an update on that record.
So for example, lets say you have 2 entries coming into MDM
Source_id | name | phone_val |
---|
1 | John | 555 |
2 | Steve | 333 |
You can see that each person has a source_id, name, and phone value. Lets say now we send in an updated sheet
Source_id | name | phone_val |
---|
1 | John | |
Here now John with Source_id of 1 comes in without a phone number. This new entry upserts into the MDM table, and updates the record on the source_id of 1, removing the phone val. The MDM table looks like this now:
Source_id | name | phone_val |
---|
1 | John | |
2 | Steve | 333 |
Lets say now 2 more entries come in now:
Source_id | name | phone_val |
---|
1 | John | 954 |
1 | John | 754 |
The first entry will come in and update the table on the record with source_id of 1, adding the phone val to be 954. After that, the next entry will get processed, and now the final record for source_id 1, will have phone val of 754, with the final table looking like this:
Source_id | name | phone_val |
---|
1 | John | 754 |
2 | Steve | 333 |
In summary, we received these entries and processed them into our load plans:
Source_id | name | phone_val |
---|
1 | John | 555 |
2 | Steve | 333 |
1 | John | |
1 | John | 954 |
1 | John | 754 |
And our final MDM table after processing all these will look like this:
Source_id | name | phone_val |
---|
1 | John | 754 |
2 | Steve | 333 |
When it comes to deleting in MDM, you will handle this scenario in the delta load component, where it has the mapping columns step. You need to go into the step and change the change_type from ‘U’ to ‘D’. The issue with this, is that any source ID you run through there now will be deleted, so to solve this, you can attach an ‘active_flag’ value to each of your records, which then can be mapped in that change_type column expression. For example:
source_id | name | phone_val | active_flag |
---|
1 | John | 754 | Y |
2 | Steve | 333 | Y |
1 | John | 754 | N |
Inside the change_type expression, you will need a condition that is something like this iif(active_flag is ‘Y’, ‘U’, ‘D’)
This will make sure that you are mapping your records in using that active flag, so that MDM can translate the active_flag from YES and NO to DELETE and UPDATE (UPDATE also inserts)
Hope this helps!
Best regards
Hi @RandomStranger37, I’m closing this thread for now, if you have any follow-up questions please don’t hesitate to share them in the comments or create a new post ♀️
but how to implement this in ataccama? Thats my question.
Hi @RandomStranger37, the merge step is the last process we do in the plan. May I ask if you have already completed setting up connected systems and the instance layer? If you haven’t yet, you could create a CDI example project and go top down seeing how things were created there as an example flow, I’m sharing an article from our documentation that can be helpful https://docs.ataccama.com/mdm/latest/logical-models/creating-a-logical-model.html