Grabbing associated terms from an attribute in one desktop

Hi Andrew,

The second metadata reader is for the catalog item attributes right? Does it also contain a filter on the parent ID, so of the incoming catalog items of the first reader? So I gues something like $parent.$id = "#catalogItemID#", where catalogItemID would be the value in the Id Column Name in the first reader.

Kind regards,

Albert

Hi ​@apejko ,

I’m not sure what kind of level of detail regarding Catalog Items you need but I think just reading the “attribute” entity and it’s “termInstances” embedded entity might be enough to identify all the terms. You can INNER JOIN these streams on attributeId and you’ll have list of attributes with terms applied to them.
If you need to narrow down the list of CI’s (which are parents of the attributes) - you can use the parent id filter that Albert suggested and provide the list of id’s for catalog items you’re interested in.
 

Hope this is useful,
Ivan

Hi Albert,

Yes the second catalog item is for attributes, and is filtered accordingly. The first catalog item reader will read the catalog items and output the attribute ids and the second reads these ids to check for term instances on the attributes.

Hi ​@apejko ,

I’m not sure what kind of level of detail regarding Catalog Items you need but I think just reading the “attribute” entity and it’s “termInstances” embedded entity might be enough to identify all the terms. You can INNER JOIN these streams on attributeId and you’ll have list of attributes with terms applied to them.
If you need to narrow down the list of CI’s (which are parents of the attributes) - you can use the parent id filter that Albert suggested and provide the list of id’s for catalog items you’re interested in.
 

Hope this is useful,
Ivan

Thank you Ivan,

I dont believe this will work for my use case as I do not want to read all attributes, but just the specific ones that belong to the jurisdiction of the catalog item. Currently my plan is using the filter but it still runs extremely slowly.

Hi Andrew,

Do you know if a specific step is taking much time to complete? You will probably know already, but you can use below button to monitor run-progress.
 

Hopefully Ataccama ( ​@ivan.kozlov ) then can provide more detailed guidance on how to proceed. Roughly how many catalog items and catalog item attributes are there?

Kind regards,

Albert

Hi Andrew,

Do you know if a specific step is taking much time to complete? You will probably know already, but you can use below button to monitor run-progress.
 

Hopefully Ataccama ( ​@ivan.kozlov ) then can provide more detailed guidance on how to proceed. Roughly how many catalog items and catalog item attributes are there?

Kind regards,

Albert

It is the second Metadata reader in the chain labelled as “One Metadata Reader” which is the step accepting the attribute id and returning the term instances. I have attached a copy of my plan as reference.

​@apejko ​@Albert de Ruiter, i can’t really provide any performance benchmarks for the metadata readers but obviously the larger the catalog is - the longer it’s going to take.
Though reading attributes can indeed take quite some time if you have a very large number of tables imported to catalog, the filters that you have configured should limit the scope of the export and make things get executed faster.
I would be curious to see if the plan is only slow when all the operations/steps (both metadata readers and lookup builder) are executed together in chain or it’s going to look the same way if the steps will be executed separately. You can try separating the steps and write the output form the steps to a text file and see if thing would run faster. 

​@apejko, i just noticed that you’ve shared the component as well.
I had a quick look and noticed that you’re basically reading the attribute entity twice - first time as embedded entity under catalog item, and the second time - as the main entity and then read term instances as well. I’d say that this alone can cause some slowness due to overlapping operations.I would recommend limiting the number of times you read such a large metadata entity as attributes.

The 2nd metadata reader alone should give you give you attributeId, catalogItemId, termInstanceId, businessTermId (termInstance.target), i believe that should be enough to find the information you need. If you need more details, you can always also read metadata for “catalogItem” and “businessTerm” and join in with results from this attribute/termInstance metadata reader.