How to find the completeness of the descriptions for all the tables and their attributes
Good day!
I would like to find the completeness of the descriptions for all the tables and their attributes in Ataccama Web. Is it possible to do this in AtaccamaWeb using Rules? There are approximately 700 tables in the system. I will be very grateful for your help
Thank you!
Page 1 / 1
Hi! I don’t think you can apply rules to this in the web application, if you mean the metadata description in the catalog:
The DQ rules are used to control the DQ of the data of the tables, not metadata.
But you can use this AQL filter to filter tables that have either the description null or some attributes with empty description → for the second case it won’t tell you which attribute has an empty description, only that there is some in the table.
description is null or attributes.any(description is null)
Different approach would be using the ONE Desktop, there you have more options how to check this. I can provide more information if you are interested.
Kind regards,
Anna
Hi, @anna.spakova !
Yes, I am interested, I will be grateful for the information. What ways are there in ONE Desktop?
Kind regards,
Assel
Hi @FCB developer , sorry for the delay, I was on vacation.
So in the desktop tool, you can download the CIs/attributes using ONE Metadata Reader step and either
download the descriptions and do some action on those,
or you can download only the CIs/attributes with empty descriptions using the Filter tab.
That will give you list of all CIs/attributes with empty descriptions, and you can then work with the list. You can for example:
create a CSV export,
push the data into some database,
create some Excel file and share it with usera via email,
compute some completeness statistics that you can also share in various ways
The options are quite wide. Also, if you want to check this regularly, you can wrap it into a workflow and schedule it using the orchestration server component.
Thanks for the detailed information, I will try and give you feedback
Kind regards,
Assel
Hi, @anna.spakova My apologies for the late question. I have a new question regarding attribute descriptions. I need to find the count of attributes with not empty descriptions and the count of attributes with empty descriptions, along with the schema and database names. I was only able to obtain the count of attributes with empty descriptions in the following format:
But I need it in something like this format:
I will be very grateful for your help
Thank you!
Hello @FCB developer ,
attached is a plan that should do the above. To explain a little bit the logic there: The database and schema has to be exported through catalog items. Both schema and database are in Catalog represented as locations. There is this hierarchy: Parent of a catalog item is a location (=schema). Parent of a location can be also a location (=database). So you need to do some joins between catalog items and locations to get to those.
Please keep in mind that this won’t work for some special types of catalog items (VCIs, SQL CIs, sources like Salesforce,...) - the hierarchies hier are usually more simple (catalog item is in a workspace folder or directly in database etc.)
Another thing to keep in mind, if you have thousands of CIs in catalog, the join with the attributes can take quite some time and resources.
Let me know if anything is unclear.
Kind regards,
Anna
@anna.spakova Thank you so much! It was very helpful . I'm currently studying the algorithm, and if I have any questions, I'll let you know.
Нello, @anna.spakova I apologize for the late question. I have a new question regarding attribute descriptions. I need to find attributes that are not linked business terms specifically with the name of tables and attributes. Using the "Filter" tab, I wrote such a query schema is 'cli_dwh' and attributes.any(termInstances.count() == 0) in ONE Metadata Reader, but it seems it takes only the schema but skips the business term, gives a list of attributes that are already linked to the business term. Can you give advice on this problem?
I will be very grateful for your help
Thank you!
Hello @FCB developer ,
I would change the “any” to “all”:
schema is 'BUSINESS_OPERATIONS' and attributes.all(termInstances.count() == 0)
Becuase otherwise you are saying that you want tables where at least some attribute doesn’t have any term, not all of them. Also just be careful using the “schema” property, some technologies don’t have it populated. Then it’s better to use $parent.name