Merge Partitioned Parquet files

Question

Hello all,

i have datalake gen2 parquet files in containers and folders. In the folder are delta.log file and parquets. Is there some way to merge these parquets to one table? This table should represent one current(up to date) table contains data from parquets.

this is what i see now in ataccama. I want to merge these data to one table.

Thanks for replies, Matus

Samuel Muvdi · Accepted Answer

Hello! Please look into transformations on ONE web app as well as ONE Data tables. Through a combination of these 2 you should be able to union your parquet files in a transformation plan using their partition key, and then create a ONE data table out of the outcome of that union, to be able to see the results tabularized

Best regards-

Samuel :)

Cansu · Answer

Hi @Matus_Marinakwelcome to the community and thank you for posting! I’m closing this thread for now, if you have any follow-up questions please feel free to share them in the comments or create a new post 🙋‍♀️

Reply

Sign up

Login to the Ataccama Community

Scanning file for viruses.

This file cannot be downloaded