Skip to main content
Question

Can I create a rule that doesn't consider Datatype?

  • May 21, 2025
  • 1 reply
  • 20 views
tziad
Data Voyager

When creating a DQ rule, I need to specify an attribute and define it’s datatype. But what if I don’t care what data type something is?

 

I want to create a rule that checks a combination of NULL/NOT NULL for two columns. So if column A is NULL and column B is not, this would result in an INVALID test.

 

But I want to do this for various metrics. Sometimes I want to check if a string column is null against a float column, or a date against a string etc. Do I need to create every permutation of every datatype combination as a rule in order to accomplish this?

 

Also, even if it is required to make all these duplicate tests. Is there a way to configure the rules so that they don’t show up as “Applied DQ Checks” for both columns selected in the configuration? Example. I care whether or not Column A is populated given column B, but I don’t care whether or not Column B is populated. Currently when making a rule like this, the tests gets affixed to both columns even if I only add it to a specific one.

    Lisa Kovalskaia
    Ataccamer
    Forum|alt.badge.img+3

    ​@tziad hi, I can definitely see how “typeless” rules could be useful. At the moment there isn’t really a solution I can think of, you’d have to create multiple rules to make sure that the rule input type matches to the data column type. To streamline the process a bit you can use the Duplicate rule feature, so you would only need to adjust the datatypes and then the new rule would be good to go. We do have a similar idea in our product feature requests pool, and upvoted it on your behalf. Hope we’ll see some traction there!

    Regarding your second point, showing the rule on some but not all of the data attributes that are being evaluated by the rule would also require some additional logic in the product. “DQ evaluation target” vs “supporting data” attributes would have to be distinguished which would add some complexity to rule configuration or mapping process. How do you imagine this enhanced functionality could work in an ideal world? You’re very welcome to describe it on our Ideas space here: https://community.ataccama.com/ideas What I would suggest in the meantime is filtering for relevant attributes in your DQ reports/dashboards. You can use terms (or a custom field if necessary) to label the key data attributes you’re evaluating in each catalog item, to make that filtering easier downstream. Will this work for you? Happy to discuss further!

    Reply


    Community Terms of Use & ConditionsCookie settings

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

     
    Cookie settings