I'm working with a database table that contains a column storing complicated JSON/XML data structures. I need to implement data quality rules that will search for and validate specific values within these structures and compare them with values in other columns.
My specific requirements:
-
Need to search for particular patterns and values nested within JSON/XML structures
-
Considering using regular expressions for this search capability
-
Concerned about the performance impact on database resources
-
Want to ensure this approach is sustainable as data volume grows
Has anyone implemented data quality rules for searching/validating values within JSON or XML columns? Are regular expressions an appropriate approach, or are there better alternatives within Ataccama (Desktop?) for handling hierarchical data structures?
I'm particularly concerned about performance impacts and would appreciate insights from those who have solved similar challenges.