Solved

Merge Partitioned Parquet files


Hello all, 

i have datalake gen2 parquet files in containers and folders. In the folder are delta.log file and parquets. Is there some way to merge these parquets to one table? This table should represent one current(up to date) table contains data from parquets. 

this is what i see now in ataccama. I want to merge these data to one table. 

Thanks for replies, Matus

icon

Best answer by Samuel Muvdi 9 July 2024, 18:08

View original

2 replies

Userlevel 3
Badge +3

Hello! Please look into transformations on ONE web app as well as ONE Data tables. Through a combination of these 2 you should be able to union your parquet files in a transformation plan using their partition key, and then create a ONE data table out of the outcome of that union, to be able to see the results tabularized

 

Best regards-

Samuel :) 

Userlevel 6
Badge +4

Hi @Matus_Marinak welcome to the community and thank you for posting! I’m closing this thread for now, if you have any follow-up questions please feel free to share them in the comments or create a new post 🙋‍♀️

Reply