We are working to migrate a long running set of data quality rules into Ataccama. Most have been easily implemented and are running smoothly.
However, the last set of about 10 deals with overlapping date ranges related to effective dates. We want to validate that one Processor (for instance) only has one “active” record for a date (as defined by the start effective date and end effective dates on those records).
For instance, let’s say Processor ABC has the following records, there is a problem with records 2 and 3.
- ABC, Start: 1/1/2024, End: 12/31/2024
- ABC, Start: 1/1/2025, End: 12/31/2025
- ABC, Start: 10/1/2025, End: 12/31/2026
Ideally, we would like to pass an identifier (in this case ABC), and then the start effective date, and then the end effective date, as we have to do this on about ten + tables. So, we’d like a universal rule that we can use across the board.
I’m reaching out to see if anyone else has implemented a similar rule and can share with us their logic and process.