Skip to main content
Solved

Adding/Removing Attributes of Catalog item in Monitoring Project?

  • 25 July 2024
  • 5 replies
  • 35 views

Hi all! 

I am working on a monitoring project. I have a catalog item consisting 130 columns. I want to create a custom catalog item selecting only few attributes that I need to have them in my monitoring project.

However, based on my previous experience: Once I have created a catalog item with selected attributes. Then I had to add new attributes according to my project needs. I have run the monitoring project again with new attributes of catalog item, then I lost my monitoring project. It showed errors for all DQ rules because as I understood from error message, DQ rules need to have attribute key ids, and with adding new attributes I have changed attribute key ids. 

Now, again I have a huge catalog item and I want to reduce the volume with selecting few attributes but I don’t want to face with the same problem again. If you have any suggestion it would be so great for me. 

Thanks !

5 replies

ivan.kozlov
Rank
Userlevel 3
Badge +3

Hi @sumeyra,

If you need to create new catalog items with subset or different set of fields you’d need to either create new VCI’s or SQL CI’s with desired structure.
Changing the structure of CI’s that are a part of the existing monitoring project can indeed cause some issues as it can potentially break rule mappings and project in general.

Monitoring Project import feature can be used to migrate the DQ configuration from one project to another as long as they have more or less identical structure/list of attributes. In v15.2 there quite a few improvements available for Monitoring Project Import functionality which should give you granular control over the parts of configuration that need to be migrated.
https://docs.ataccama.com/one/latest/monitoring-projects/monitoring-project-import-configuration.html

And if you’re interested in results of specific DQ checks on subsets of attributes you can just use attribute filter under MP Configuration & Results tab and only look at the attributes you’re interested in.

I hope this helps,
Ivan

Cansu
Rank
Userlevel 6
Badge +3

Hi @sumeyra, I’m closing this thread for now, if you have any follow-up questions please don’t hesitate to share them in the comments or create a new post 🙋‍♀️

Check out Our Community Recommended Journey to get started on the community 🙋‍♀️
S
Rank
Userlevel 1
Badge +2

Hi @ivan.kozlov , thanks for sharing your comment, with this solution we might lose our history of monitoring project, right ? How can we create a solution without losing any historical results ?

ivan.kozlov
Rank
Userlevel 3
Badge +3

Hi @sumeyra , if you would migrate configuration from existing project to a newly created project with the same set of CI’s the history would not be imported, however the original (source) project will preserver the history.
So if you need to see the changes within the same project I'd probably suggest extending the project with additional CI’s which would have the data you need and then you should be able to compare.
If you change the SQL Query for existing CI in the project and some attributes used in DQ check mappings will disappear - you’ll have to resolve those conflicts in MP configuration. However if attributes used in dq checks will remain a part of the CI and only unused ones would be removed and new ones will be added then the existing mappings should still be maintained. The history for this CI in MP should be preserved. All of it basically depends on the changes in internal id’s for different entities (CI’s, MP’s, CI Instances, Rule Instances, Attributes etc.). Usually update operation don’t change the ID’s, but if you delete something or create something new that could have impact on connection between different metadata entities.

I would recommend you to test this scenario in your dev environment on some smaller project:
1) Create a copy of existing CI’s in the project
2) Create new project and add these new CI copies to that project
3) Use import configuration from original project to new project and map the import on the CI copies
4) Run this new project 1-2 times to create some history.
5) Try to update the structure of these new copied CI’s (either update the SQL query or VCI configuration).
6) Run the new project one more time
7) If project run failed due to some structural inconsistencies - resolve the DQ check mapping conflicts (remap the checks or remove dq checks)
8) Run the new project one more time and once it’s finished check the results and compare the historical results between the latest run and the previous runs

Before doing this on “live” CI’s and MP’s please try to test it on copies of existing items.
I hope this info will be useful

S
Rank
Userlevel 1
Badge +2

Thanks for sharing in detail. Yes you are right, i was planning to test this way in dev environment. @ivan.kozlov 

Reply


Community Terms of Use & ConditionsCookie settings