Extracting Rules and other Metadata Information from a Monitoring Project ?

Hi ​@Apurva Kapoor, these you can extract quite easily via the ONE Metadata reader step:

1. DQ Rule UID
2. Name of DQ Rule
3. Definition of DQ Rule
4. Threshold
5. DQ Dimension
6. DQR Owner

Most of that information is stored in the properties of the rule entity, so you can start there. To get the threshold of each rule instance applied in a given project, you will need to dig into the embedded streams of monitoringProjectConfiguration → projectCatalogItemInstance → dqCheck → checkAlertInstance.dqThreshold

The timestamps you're looking for are not directly available in entity properties out of the box. Have you added them into your metadata model, or do you want to export them based on rule or monitoring project version history?

Can you please share the plan for the mentioned rule configuration setup to extract the metadata so that we can try to use and test it ?

Is there any reference that you can share to extract the Rules Metadata from Monitoring projects ​@Lisa Kovalskaia ?

​@Apurva Kapoor hi! Apologies for the radio silence! Please see an example plan attached (built and tested on v.15.4) - you may enrich it further to collect details like attributes to which rules are applied, rule alert thresholds, etc.

As I mentioned, by default versions and dates are stored differently in the Ataccama DB, so rule start date/end date/modified date either could be added as custom fields in your rules, or that info could be exported with GraphQL queries. Alternatively, if you run this metadata export regularly on a fixed schedule, you may use each individual export as a snapshot of the current state of rules, and calculate which rules are added/decommissioned/modified based on the differences between the snapshots.

Hope this helps! I'm curious to hear more about how you'll use this metadata - is that for reporting? For synchronisation with another system? Something else?