Iterating rows of a table (Record-by-record - SQL Iterator) in a workflow

  • 24 November 2022
  • 13 replies
  • 142 views

Userlevel 4
Badge +3

At Stater we needed to iterate a table in a database and perform some action on it. This needed to be orchestrated from a workflow. In our case trigger a set of MP's and write the output to a DQ Data warehouse.

So I created a ‘pattern’ which we use for our cookbook. With thanks to Brian and Matej!

 

 

Now if somebody can tell me how to add a PDF and ZIP with the sample, I'd be happy ;)


13 replies

Userlevel 4
Badge +3

Okay, so here is the PDF with the explanation and a sample in a ZIP.

The sample was created in 13.7. 

Let us know if we could have done things differently. Thank you.

Userlevel 1
Badge +1

I have exact similar situation. following this solution and the tutorial sample 07_04_Iterator_SQL_Parent.ewf, I am following exactly same setting but somehow the parameters is not getting recognized . Please refer to following screenshots and help me find my mistake:

  1. This is the sample sql output on which I want to iterate my actual DQ plan/comp, it has only 1 row output.

     

  2. there is nothing other than this to pass to the child workflow/ actual DQ plan, hence I am mapping only output column as process_date:
     
     
    1. but the child workflow doesn't recognize this process_date in parameter section unlike the samples I am referring to:

      Can you help me understand why this plan struggles to find defined parameter ?
Userlevel 1
Badge +1

I am getting this error while trying run:
30.08.2023 10:39:07 [FATAL]    The following variables are declared but not defined: process_date [Workflow]

 

Userlevel 4
Badge +3

I think that in the properties of the Run DQC you need to define the paramater with quotes. Am I correct I see a dollar-sign now? So the expression is "”"”+process_data+"”"”. 

Userlevel 1
Badge +1

Thanks for the response , Ataccama provided sample plan has $ sign to read a parameter , I have tried with “”””+pocess_date+””””  as well but its not recognizing 

 


 

 

Userlevel 4
Badge +3

Strange. The sample plan is mine. I'm sure you should have 4 opening and 4 closing double quotes. (But this editor is a bit strange with quotes.)

"”"”+var+"”"”. (Maybe take out the underscore. Atac is sometimes a bit strange with those?) But I also notice your var isn't pink/purple. When you created the name, was it selected from the drop-down? (CTRL-space dropdown) If not then it is not yet declared in the Global Properties of the workflow-canvas. Can you share the global properties of the canvas with the RunDQC?

 

Userlevel 1
Badge +1

yes that's true its not turning pink/purple , removing underscore doesn't help either., see nothing comes matching starting with ‘p’

 

 

Userlevel 1
Badge +1

I have both parent(iterator) and child (DQC) plans in same folder , I hope this is the correct referencing.

 

 

Userlevel 4
Badge +3

Can you share the global properties of the canvas to run the DQC? Click on a white space in the canvas (right-click) and 'edit properties’. Did you declare the input parameters?

 

Userlevel 1
Badge +1

Ah got it , I am able to pass parameter to child workflow and iterate it. 

 

How do I edit actual plan/comp parameter mapping ? its blank and on right click its not allowing to add anything.

 

Userlevel 4
Badge +3

Hi, I think all that is in the PDF. You have to use the alter format and map the variables:

 

Userlevel 1
Badge +1

Yes , I am able to successfully do it. Many thanks 

Userlevel 4
Badge +3

You are welcome!

Reply