I am trying to create a transformation plan in DQ One Web that merges existing data in a One Data table to the most recent run from a Monitoring Project.
The situation is difficult and I will try to be as concise as possible:
- We have a One Data table that is created from a Monitoring Project Data Remediation plan.
- We input notes on the One Data table that explains what needs to be done to that invalid record
- Every time the monitoring project is ran, the notes in that One Data table are deleted.
The fix in my mind is that we could union what is currently in the One Data table (with notes) with the new remediation export in order to retain the notes on the “older” version of the One Data table, while allowing any new invalid records to be written into the One Data table.
Essentially, I need to somehow create (within the transformation plans on One Web) a left outer join where everything with the current run coming in, but joining with those on the right (the older run) that has notes.
Any help on this would be amazing as its cogging up our efforts with our business stewards.