Enhancing Data Quality Evaluation with Component Rules in ONE Web

Speaking about data quality rules, we usually refer to DQ rule configuration in ONE web application. However, when it comes to referencing external sources or performing preliminary work or combining different sources, we may find that the capabilities within ONE Web are insufficient for our needs. For example, consider a scenario where we need to validate whether our data matches an external data source, such as ABN Lookup on https://abr.business.gov.au/. This is where component rules come in handy.

The Task at Hand: Validate ABN Information.

To implement this data quality (DQ) check, we require a component rule.

Here's how you can configure the ABN validation component and use it for DQ evaluation:

​​​​​​​Configure a placeholder for the component

Navigate to DQ section in ONE Web application and create a Validation component with the following parameters:

• Name the input attribute as 'ABN.'

• Specify a list of explanations
  • ABN_EMPTY: will be used for the case when ABN is empty. 
  • ABN_WRONG_LENGTH: ABN has the wrong length (not 11 digits).
  • ABN_INVALID: ABN is invalid.
  • ABN_VALID: ABN is valid.

• Choose Ready for Implementation as the implementation state.

• Publish the validation component.

Configuring Logic for the Component using ONE Desktop application

To configure the logic for the component using the ONE Desktop application, follow these steps:

• Connect to the web application to ensure a proper connection. Right-click on your component and choose "Edit Component"

• Configure the logic for your component adding steps between integration input and integration output steps.
  • By and large, the flow has two possible paths, depending on ABN value:
    • If ABN meets the 11-digit requirement and the format is valid, it follows one path.
    • If ABN doesn't meet the requirement or is empty, it takes a different path.
  • The workflow allows for data to be processed and scored accordingly, based on the quality and validity of the ABN being checked. In this step, we are calculating a score or a measure of the quality or validity of the ABN. A higher score may indicate a more valid or accurate ABN, while a lower score signifies potential issues with the ABN.

  • 

    Publish your component by right-clicking on the 'ABN Validation' and choose 'Implemented.'

  • 

    Navigate to the ONE Web application and check the status of your component.

    

Adding Changes to Runtime Configuration in ONE Web Application

• Create a connection to ABN resource.
  • Name: AbrXmlSearch
  • URL: https://abr.business.gov.au/ABRXMLSearch/AbrXmlSearch.asmx

• Add information into runtime config in you ONE Web Application

  

Configuring the DQ component rule

To configure DQ rule with a component perform the following steps:

1. Navigate to the Data Quality section.
2. Create the rule 'ABN Validation.'
   1. Set 'ABN' as the input attribute.

      

       

   2. Choose 'Component' in the Rule Logic section and select the 'ABN validation' component.

      

       

   3. Match the component's attributes and logic with DQ rule results.

      

       

   4. Test the rule.

      

      

       

Applying the Rule

There are several options for using this rule in ONE Web Application (v14.4): 

1. Apply it directly to an attribute in a Catalog Item.
2. Use the rule assigned to an attribute in monitoring projects.
3. Assign the rule to a term.

DQ component rules can significantly enhance data quality options for managing and validating data. The step-by-step guide provides a practical framework for configuring, implementing, and testing these rules, ultimately leading to improved data accuracy and reliability, which is crucial for data-driven decision-making across various applications and industries.

The material was prepared in collaboration with @Alexander Kislyakov and @Adrian Anderson