Skip to main content

Export Data Quality results

  • January 31, 2023
  • 6 replies
  • 641 views

Maxim Kim
Ataccamer
Forum|alt.badge.img+3

Eventually, you might need all your data quality results evaluated in One monitoring projects or within regular catalog items (via business terms and related DQ rules) exported to some reporting system or into excel. And it would be nice if it looked like this:

 

 

TLDR; There are 2 components attached you can use to export DQ results, one for monitoring projects, another one for regular catalog items.

 

Monitoring Projects

The idea is to read all monitoring projects metadata, join it with their check results as well as rules names, catalog item names and voilΓ , we can save it to excel/csv or to some database table.

It sounds like something very simple, and indeed there is no black magic, but nonetheless component looks a bit more elaborated:

 

Catalog Items DQ (based on Term evaluations)

The idea is to find all catalog items that have aggregated overall data quality and join them with the DQ rules results:

This component has a bit simpler implementation:

 

 

Of course, the output of the components could be and probably should be changed per your own need, which might additionally include term name, for example, or a connection/data source names or something else :).

You can come with a plan that calls said components, saving results into a DB of PowerBI/Tableau/Whatever and put it into a scheduled workflow...

Forum|alt.badge.img+2

Hi,

 

Very nice! Thank you so much.

 

Marnix


Forum|alt.badge.img+1
  • Data Voyager
  • September 4, 2023

Hi

Can we include following columns in the existing Component as well - Monitoring Project Name, Validity Percent, Business Term, Term description?

Thanks!


Forum|alt.badge.img+2

You could use a GraphQL to replace the metadata reader. That gives you more flexibility.


Forum|alt.badge.img+1
  • Data Voyager
  • September 5, 2023

Thanks Marnix for the response.

Please confirm if this GraphQL feature/functionality can be implemented in Ataccama 13.9.1 version?

Could you also share sample plans/comp etc for me to refer.

Thanks!


Forum|alt.badge.img+1
  • Data Voyager
  • September 11, 2023

Any update on above query please?


Forum|alt.badge.img+2

Hi Bhumi,

I think I was a bit hasty with GraphQL. (It works in 13.9.1, that is not the issue.) In you case, it might be faster to go to the top MonitoringProject. Change this MM-reader to get the name and terms-reference as an embedded object (termInstances). These can be joined with MM-reader with terms via β€˜target’.

The second plan gets you DQ Results. Overall validity can be calculated there right?

In 13.9.4 things become a lot easier. So if this a key feature you need, consider the upgrade. For us getting data out became a lot easier with much better performance by using One Data Quality steps. Huge improvement. This also has DQMonitoring Project Check Results.


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings