Skip to main content

I would like to setup a generic data quality rule to check string length. Requirements for length checks differs per attribute. Say 
column1 needs to have max length of 10 
column2 needs to have max length of 15 and so on
Is it possible to create a generic rule to check string length, passing value of length as parameter?
I am aware of attribute length insights available as part of DQ profiling results but I would like to make the check as part of monitoring project. Thanks!

Hi @Adilhussain, for the moment rules can only take variables that are derived from either the input values or some logical expressions. Metadata can be used in detection rules and I can see how this could be a nice enhancement for DQ rules, too - I've raised your idea internally for our product team consider.

On a side note, it may be a good practice to have separate rules for different columns with different expected value lengths:

  • You might need to apply some transformation logic to the input value before you check its length, e.g. for a card number you might want to drop all non-digits, while for an alphanumeric ID you might evaluate the record as is.
  • It might also be useful to have multiple conditions evaluated in a rule, a length check being one of them, and the set of conditions applicable to different columns may vary.

Thank you for raising an interesting topic - let us know if you have additional comments!

I just read you can do this now in 15.x .. at least the documentation says so.. 

Create DQ Evaluation Rule :: Ataccama ONE

Reply


Community Terms of Use & ConditionsCookie settings