Solved

Need DQ Rule

  • 7 September 2022
  • 4 replies
  • 141 views
K
Userlevel 3
Badge +4

How to 

Check whether the target column has only DATE values or not.

?

icon

Best answer by anna.spakova 9 September 2022, 08:41

View original

4 replies

anna.spakova
Rank
Userlevel 5
Badge +8

Hi,

if you have the data with the column (Catalog Item) in a catalog, you can check the profiling results to see what the data in the column look like.

 

Does that answer your question?

K
Userlevel 3
Badge +4

Hello @anna.spakova I need to write DQ Evaluation rule to check whether the data in the column is in intended data type. Eg., Is the column defined as integer and all contents of the column as integer?

anna.spakova
Rank
Userlevel 5
Badge +8

Hello @KarthikaSPillai , thanks for clarifying. I don’t think there is some native check for the data type. You see the data type after the Catalog Item is imported into Catalog and as I mentioned above, you can verify in the Profiling results, if the values correspond to what you expect.

From the DQ rules perspective, you can write rules that will check if the values are e.g. only numbers (example would be to use trashNonDigits function and compare to the original value) etc., but I can imagine this can get pretty complicated for DATE as you can have different date formats. You may need to use some regex expressions… Also important is to keep in mind that the rule can get assigned only to the attribute with the corresponding data type as the input attribute of the rule (so if you have an INTEGER column, you are not able to assign rule with STRING as an input attribute).

S
Userlevel 2
Badge +3

You can assoicate a term to this column and have DQ rules associated to TERM, this will ensure the underlying rule is executed whenever you reprofile and validate dq evaluation results

Reply


Terms & ConditionsCookie settings