Skip to main content
Solved

How to limit number of each DQ rule in invalid sample exports from One Desktop


  • Universe Traveller

Sometimes we have monitoring project results where a DQ check returns 1000 records because of so many rule failures. We want to limit the amount of records in the monitoring project export file for each DQ rule. If I only want to show X invalid samples per DQ check, how can I go about doing this? I initially thought of using a sequence step then filtering the final result, but this does not work because each rule needs its own sequence or count. Basically, I think I may need to group each DQ rule and apply a sequence to each group separately. This is how I am thinking of solving the problem. Any suggestions on what I can do? Thank you in advance for your help!

Best answer by AKislyakov

Are you sure it is Group Selector and not Group Aggregator.

Here is a sample Group Selector configuration. It selects up to 3 first (by Id) records from each group defined by the value in the ruleName column. 


 

And test result.
 

 

View original

Forum|alt.badge.img+2

Hi ​@WesT 
I think Group Selector step might work for you.

Step description

This step creates groups of records based on specified key definitions. The step will select records from each group based on specified criteria. Records can be selected multiple times based on different criteria. The resulting records of selection criteria for each group may be dependent on sort order. For example when selecting the first N records, the sort order of the group determines which records will be selected.

 

Though depending on configuration records failing multiple rules at once might be considered a separate group and will get their own count.


  • Universe Traveller
  • June 11, 2025

Hi ​@AKislyakov ,

Thank you for the response! I’m not sure if Group Selector will work in this case. Unless I’m doing something incorrectly. See the image for what I’m trying to accomplish. Group Selector allows me to group things with a filter/expression. What I need is to have a sequence for each rule. So for rule1 it starts a sequence, but once it hits rule2, it should reset and so forth. Then the final step will be to add a filter so I can limit the amount of records per rule.
 

 


Forum|alt.badge.img+2

Are you sure it is Group Selector and not Group Aggregator.

Here is a sample Group Selector configuration. It selects up to 3 first (by Id) records from each group defined by the value in the ruleName column. 


 

And test result.
 

 


  • Universe Traveller
  • June 11, 2025

@AKislyakov I followed your instructions and it worked! Thank you so much, I’ve been stuck on this for a few days. I wasn’t using the Group Selector properly. I didn’t know how to get to the Result portion with the Selections and Sorting.


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings