DQD Business entity


(Ra Li) #1

Hi there, we have a hierarchy for data where it starts with level 1 then 2,3 and so on.

We’re trying to build a dashboard that rolls the data quality results up based on the hierarchy. For example:

I have one company, with multiple departments, and each department has many employees. We would like to build DQ rules for the company’s data, then the departments and finally it’s employees. but we would like this data to be rolled up. Meaning, if employees data has 80% score and departments DQ score 90, we would combine those when we view the data on the department level, and so on for the company.

What is the best way to execute the above?

Thanks in advance,


(Katrin Popova) #2

Hello Ramzi,

Yes, what you have described could be achieved with DQD. You would need to define the rules and then apply them accordingly and then build the hierarchy. Based on your description the hierarchy in DQD are Business Entities, they are defined per model and aggregate the outcomes of evaluating rules. Everything you have as subcategories of a category will be aggregated and shown as overall result for the given category. For Business entities please refer to this documentation.

Please note that giving step by step instructions according to technical or business requirements is out of scope of the support. In this case, I could propose you our consultancy services. Please let me know if you are interested.

Regards,

Katrin


(Ra Li) #3

Hi Katrin- Do we need to use the multi entity expression to perform that or can we use regular expression rules (or other type of rules) to aggregate data?
The business entities are already done, but it doesn’t seem to be aggregating the data accordingly.

Thanks,


(Katrin Popova) #4

Hello Ramzi,

I am not sure I understand, the problem with the aggregation of data. If you would like to have something like this:
image
For example A, B and C data quality results (valid-invalid) will be aggregated in the demographics, same with every set of children and their parent node in the hierarchy in business entities.

I would advise you to go through Business entities configuration documentation, because there is a detailed description that would help you build your logic.

In addition here you can find description of the use case for the given rule types and how the assigning of rules to attributes is done.

Please keep in mind that implementation of the solution is out of the scope of the support. For this I could propose you to use our consultancy services. Please let me know if you are interested.

Regards,
Katrin


(Ra Li) #5

Hi Katrin- Thanks for your response. I’m looking to understand if it’s possible on the diagram you presented to have the data aggregated instead of having “Retail Customer” to fail due to only one rule failing. The aggregated data will then be aggregated across different rules and rolled up to “Retail Customer”, if the thresholds limit was exceeded, then it would fail otherwise it would pass.

In the model provided by the documentation, it seems as if one rule failed, the entire business entity is considered a “fail”. Instead, I would think that Contact would fail (assuming we set a threshold for failure was met by “E”) but “Retail Customer” may not fail with the aggregation (again dependent on the threshold that is defined)

Does DQD support the ability to aggregate the data on a record level as it gets rolled up and passes/fails each entity level on it’s own?

Thanks,


to add some clarity, in your picture provided, it was determined that the failed record met or exceeded the fail threshold thus it shows as red. However, in my example, I’m looking to see if when “E” fails, the “Contact” entity also incorporate the counts from D and F on top of E and measures whether Contact meets the fail threshold or not based on these sub entities, and if not, it would Pass.

I hope this helps a little.


(Katrin Popova) #6

Hello Ramzi,

It could be achieved in DQD. It all depends on how you will structure your hierarchy in your project. You might consider even having different systems. If you would like help with the implementation of your solution I could propose you our professional services. Please let me know if you are interested.

Regards,

Katrin