I am making a DQ evaluation rule in ataccama one ui where I have to subtract each long value of catalogue item from its corresponding long value in lookup and if the difference is greater than 100, I want to raise that particular value as invalid sample, so far I am not able to achieve this, it would be great if anyone can help me with this.
Solved
Arithmetic operations on Lookup using DQ evaluation rule in Ataccama One ui
- Data Enthusiast
- 6 replies
Best answer by Lisa Kovalskaia
Refreshing lookups used in component rules requires some additional setup, since there's currently no way to "link” a lookup from the Web app to a component.
Typically an orchestration workflow is configured to run a lookup building plan on schedule and to place fresh lookup file(s) into the expected location(s). The validation component, and more specifically the Lookup step in it, should be look for the resource in that location (e.g. MinIO object storage). The file will change but the resource path should remain the same so that the component continues working automatically.
If you haven't worked with orchestration workflows before, here's a couple of links to get you started:
https://docs.ataccama.com/runtime-server/latest/workflow-and-scheduler/workflow-and-scheduler.html
Once you have the workflow it can be easily used to establish a refresh cadence for many lookups. Please let me know if you have additional questions!
+3- Ataccamer
- 39 replies
In the Lookup step configuration, use the lookup.key expression to store the value into some working column:
From there you could calculate the difference between the src_value and the working column, and decide the VALID/INVALID result based on the outcome.
Does that help?
- Data Enthusiast
- 6 replies
Thanks for the solution
+3- Ataccamer
- 39 replies
With that integration, you can access and work with many key assets of the Web app, here's more on that:
For your specific use case, you’ll want to build a DQ rule that relies on a validation component:
Note that you will need to download the lookup file you have in your ONE Web app and upload it to the component in ONE Desktop - once you publish the component the lookup file will be uploaded and stored in Ataccama's object storage. Please refer to ONE Desktop Tutorials available within the app for additional info and examples.
When the component rule is ready, you can go back to your ONE Web app and apply the rule to a catalog item - just like you would apply any other rule built in the Web app.
I would also encourage you to check out the Ataccama Academy, there's a wealth of resources for self-paced learning, could serve as a great reference for all things Ataccama.
If you have additional questions, let me know!
- Data Enthusiast
- 6 replies
+3- Ataccamer
- 39 replies
- Answer
Refreshing lookups used in component rules requires some additional setup, since there's currently no way to "link” a lookup from the Web app to a component.
Typically an orchestration workflow is configured to run a lookup building plan on schedule and to place fresh lookup file(s) into the expected location(s). The validation component, and more specifically the Lookup step in it, should be look for the resource in that location (e.g. MinIO object storage). The file will change but the resource path should remain the same so that the component continues working automatically.
If you haven't worked with orchestration workflows before, here's a couple of links to get you started:
https://docs.ataccama.com/runtime-server/latest/workflow-and-scheduler/workflow-and-scheduler.html
Once you have the workflow it can be easily used to establish a refresh cadence for many lookups. Please let me know if you have additional questions!
- Data Enthusiast
- 6 replies
Thanks
1 person likes this
Reply
Most Liked this week
Login to the Ataccama Community
No account yet? Create an account
For Ataccama Customers and Partners
or
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.