Use "reference" files to validate data?
Is there a way, with the free DQ Analyzer, to compare/validate the values in fields on one file against a list of values in other separate "reference" files?
-
Hello,
there isn't support for Dictionary files in DQ Analyzer, but you may use Foreign Key analysis feature with Drill-through to find the values, which are and are not in the reference file.
You can do that by Creating a profile using both files as an input and creating a plan. After opening the Profiling step, there is a Foreign keys tab on the root node, where you add columns for analysis between the sources.
Also in a case you do not have many values in reference files, you may validate the data using expression "is in {}". For example:
src_gender is in {'F', 'M', 'FEMALE', 'MALE'}
This won't help you if you have a lot of enumerated values in the reference file, unless you want to rewrite them manually. -
-
In the DQC, we are using a Lookup files for validation against reference files. The Lookup files can be used in many ways (in many different steps) and usage of them is described in a DQC Lookup Video Tutorial.
-
Loading Profile...



Twitter,
Facebook, or email.

EMPLOYEE
