Hi everyone,

We are continuing with our Data Quality best practice series! In the next several posts we will cover everything from Rules to Monitoring Projects in Data Quality. If you have missed the introduction series to DQ you can start here:

In this post, we'll explore the concepts of Detection rules and Data Quality (DQ) rules in Ataccama ONE's web application. Understanding these rules is crucial for effective data management. Let's dive in!

Detection Rules
 

Detection rules in Ataccama ONE are divided into two categories: Data-based detection rules and Metadata-based detection rules. These rules serve different purposes and can be accessed and created from the "Data Quality > Rules" section. Let's take a closer look:

Data-based Detection Rules: These rules evaluate attribute data and are used to assign terms to attributes based on predefined conditions. For example, you can create a rule to assign the term "First Name" to attributes that contain values found in a specific lookup file. To implement this rule:

1. Create a detection rule with the condition that attributes matching the values in the lookup file should be detected as the "First Name" term.
2. Assign the newly created detection rule to the "First Name" term in the term settings tab.
3. The rule will be applied to attributes that meet the rule's condition, effectively detecting the "First Name" term.
    

Metadata-based Detection Rules: These rules help assign terms based on attribute names. They are particularly useful when you want to apply terms based on metadata rather than the attribute's data.

It's important to note that detection rules are separate from AI-powered term detection and are used for rule-based term detection during profiling. They can be created and configured in the "Data Quality > Rules" section, but most of the interaction with detection rules occurs in the Glossary and Catalog sections.

Data Quality (DQ) Rules

DQ rules in Ataccama ONE are used to evaluate the quality of catalog items and their attributes. They can be applied to terms, attributes, and catalog items within the Data Catalog and also in monitoring projects. Let's understand how DQ rules work:

• DQ rules are organized into different categories called DQ dimensions, which define the aspects of data quality being evaluated.
• Each DQ dimension affects the rules in two ways:
  • The available results in the rule condition builder depend on the selected dimension.
  • The results of rules from contributing dimensions are included in the calculation of the Overall Quality metric.
     

The predefined DQ dimensions include Validity, Uniqueness, Completeness, Accuracy, and Timeliness. Each dimension has predefined results and can be customized to fit your specific requirements. For example, the Validity dimension evaluates if data is valid or invalid based on defined conditions.
 

To implement DQ rules:

1. Create DQ rules based on the desired dimension and conditions.
2. Map the rules to terms, which can then be applied to attributes and catalog items.
3. The results of the DQ rules can be viewed in the Data Quality tab of catalog items, attributes, or the term itself.

   

    

DQ rules play a vital role in profiling, DQ evaluation, and monitoring projects, ensuring data quality is assessed accurately.

By understanding the concepts of Detection rules and Data Quality rules, you can effectively manage and evaluate data quality in Ataccama ONE.

Stay tuned for the next post and let us know if you have any questions in the comments below