Skip to main content

I have a plan file in which there are multiple flows which generate multiple result csv files having different  number/size of column .

I want to store the output of each flow into a single csv file by appending the rows.

I want to this happen every time I run the plan file 

Example - On 1st run of the plan file each result csv file have 10 rows and there are 5 flows . This should lead to a total of 50 rows in the appended file . Now on the 2nd run of the plan file the 10 rows are generated again for each flow , this should lead to a total of 100 rows( 50(1st run) + 50 (2nd run) )  in appended file .

Hi Kundan,

 

You can use the Union Same step from the Flow Control palette in ONE Desktop application to union the 50 records from the 1st run and the additional 50 records from the 2nd run.

Example of using Union Same to append additional source input(s)

It is not possible to read and write to the same file at the same time.

So we can use a temporary filename in the Text File Writer step, then wrap this DQC plan in a workflow that performs the execution of the DQC plan and the renaming (and overwrite) of the temporary file name to the original input file name.

 

Execute DQC Plan and rename/overwrite the filename.

I have attached the file I used to create the screenshots.

 

Danny

Hi Danny ,

 

Thank you for the response . 

In my case the structure of the files that I need to append are not same . In union same we can combine files that have same structure right ?

Can you please recommend a way where I can use files that have different structure .

 

Regards,

Kundan 

Reply


Community Terms of Use & ConditionsCookie settings