DQ Rules - extend the logic to apply a fix to the input
Hello,
I am new to Ataccama One and had a question around using the DQ rules in other ETL tools and integrations. It’s my understanding that a DQ rule returns whether the inputs are valid\invalid and it can be called as a web service using DQ firewalls. Is there a way to extend this logic to apply a fix to the input and returning this fix for invalid inputs ? Can this be exposed and reflected in the DQ firewall ? Do we need to build a component rule for this or can this be achieved using rule variables \ parameters ? It is not clear to me when parameters and variables are typically used in rule configuration.
Regards,
Ihab
Page 1 / 1
Hi Ihab, Ataccama rules are meant to perform DQ evaluation and return one of the results\explanations based on defined conditions and rule logic. Rules used, for example, in monitoring project or exposed through DQF always return pre-defined set of attributes which includes explanation, score, dimension results. So while you can return a info about what’s actually wrong through explanations field you can’t return a fixed version of the input. To achieve that you might have to build a custom set of components/online services (outside of DQF context) using ONE Desktop application that would try to suggest a fix based on the input and results of condtion evaluation. I hope this answers your question. Ivan
Thanks for the response. Please take a look at the attached one page pdf that shows how a tool , Microsoft DQS, offers corrections and suggestions out of the box at no cost. DQS allows users to invoke this logic on any of their data assets in situ.
Can our user call Ataccama in a similar fashion and get corrections and suggestions in their ETLs ?
Does Ataccama product allow remediation of data in situ without having to load them in Catalog\ Ataccama One product ?
Once I develop the custom components you suggest above, will the corrected or suggested values be returned to users via DQ Firewall or API calls ?
Users wish to be able to invoke the Ataccama DQ engine , just like they did with DQS, to cleanse their data in the source. Are you able to support this very common use case ?
