Skip to main content

Iterating rows of a table (Record-by-record - SQL Iterator) in a workflow


Forum|alt.badge.img+2

At Stater we needed to iterate a table in a database and perform some action on it. This needed to be orchestrated from a workflow. In our case trigger a set of MP's and write the output to a DQ Data warehouse.

So I created a ‘pattern’ which we use for our cookbook. With thanks to Brian and Matej!

 

 

Now if somebody can tell me how to add a PDF and ZIP with the sample, I'd be happy ;)

Did this topic help you find an answer to your question?

13 replies

Forum|alt.badge.img+2

Okay, so here is the PDF with the explanation and a sample in a ZIP.

The sample was created in 13.7. 

Let us know if we could have done things differently. Thank you.


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

I have exact similar situation. following this solution and the tutorial sample 07_04_Iterator_SQL_Parent.ewf, I am following exactly same setting but somehow the parameters is not getting recognized . Please refer to following screenshots and help me find my mistake:

  1. This is the sample sql output on which I want to iterate my actual DQ plan/comp, it has only 1 row output.

     

  2. there is nothing other than this to pass to the child workflow/ actual DQ plan, hence I am mapping only output column as process_date:
     
     
    1. but the child workflow doesn't recognize this process_date in parameter section unlike the samples I am referring to:

      Can you help me understand why this plan struggles to find defined parameter ?

Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

I am getting this error while trying run:
30.08.2023 10:39:07 [FATAL]    The following variables are declared but not defined: process_date [Workflow]

 


Forum|alt.badge.img+2

I think that in the properties of the Run DQC you need to define the paramater with quotes. Am I correct I see a dollar-sign now? So the expression is "”"”+process_data+"”"”. 


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

Thanks for the response , Ataccama provided sample plan has $ sign to read a parameter , I have tried with “”””+pocess_date+””””  as well but its not recognizing 

 


 

 


Forum|alt.badge.img+2

Strange. The sample plan is mine. I'm sure you should have 4 opening and 4 closing double quotes. (But this editor is a bit strange with quotes.)

"”"”+var+"”"”. (Maybe take out the underscore. Atac is sometimes a bit strange with those?) But I also notice your var isn't pink/purple. When you created the name, was it selected from the drop-down? (CTRL-space dropdown) If not then it is not yet declared in the Global Properties of the workflow-canvas. Can you share the global properties of the canvas with the RunDQC?

 


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

yes that's true its not turning pink/purple , removing underscore doesn't help either., see nothing comes matching starting with ‘p’

 

 


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

I have both parent(iterator) and child (DQC) plans in same folder , I hope this is the correct referencing.

 

 


Forum|alt.badge.img+2

Can you share the global properties of the canvas to run the DQC? Click on a white space in the canvas (right-click) and 'edit properties’. Did you declare the input parameters?

 


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 30, 2023

Ah got it , I am able to pass parameter to child workflow and iterate it. 

 

How do I edit actual plan/comp parameter mapping ? its blank and on right click its not allowing to add anything.

 


Forum|alt.badge.img+2

Hi, I think all that is in the PDF. You have to use the alter format and map the variables:

 


Forum|alt.badge.img+1
  • Data Enthusiast
  • 8 replies
  • August 31, 2023

Yes , I am able to successfully do it. Many thanks 


Forum|alt.badge.img+2

You are welcome!


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings