I have a plan file in which there are multiple flows which generate multiple result csv files having different number/size of column .
I want to store the output of each flow into a single csv file by appending the rows.
I want to this happen every time I run the plan file
Example - On 1st run of the plan file each result csv file have 10 rows and there are 5 flows . This should lead to a total of 50 rows in the appended file . Now on the 2nd run of the plan file the 10 rows are generated again for each flow , this should lead to a total of 100 rows( 50(1st run) + 50 (2nd run) ) in appended file .
Best answer by DannyRyan
Hi Kundan,
You can use the Union Same step from the Flow Control palette in ONE Desktop application to union the 50 records from the 1st run and the additional 50 records from the 2nd run.
Example of using Union Same to append additional source input(s)
It is not possible to read and write to the same file at the same time.
So we can use a temporary filename in the Text File Writer step, then wrap this DQC plan in a workflow that performs the execution of the DQC plan and the renaming (and overwrite) of the temporary file name to the original input file name.
Execute DQC Plan and rename/overwrite the filename.
I have attached the file I used to create the screenshots.
You can use the Union Same step from the Flow Control palette in ONE Desktop application to union the 50 records from the 1st run and the additional 50 records from the 2nd run.
Example of using Union Same to append additional source input(s)
It is not possible to read and write to the same file at the same time.
So we can use a temporary filename in the Text File Writer step, then wrap this DQC plan in a workflow that performs the execution of the DQC plan and the renaming (and overwrite) of the temporary file name to the original input file name.
Execute DQC Plan and rename/overwrite the filename.
I have attached the file I used to create the screenshots.
We use 3 different kinds of cookies. You can choose which cookies you want to accept. We need basic cookies to make this site work, therefore these are the minimum you can select. Learn more about our cookies.
