Solved

Data Quality Monitoring Project Result Download

  • 12 March 2024
  • 2 replies
  • 56 views
J
Badge +1

I have a data quality monitoring project to which I’ve deployed a post-processing component. Results are written to a CSV file, which I presume is stored within Ataccama’s object storage. When I navigate to the data quality monitoring project and select the Export tab, I see the below screenshot, which appears to include a file path along with a download button. 

I’m trying to determine how to automate download of the below-depicted CSV file. Is there a way to determine the fully-qualified path to the file and download from Ataccama server?

 

icon

Best answer by anna.spakova 12 March 2024, 10:39

View original

2 replies

anna.spakova
Rank
Userlevel 5
Badge +8

Hello @jbrant , the files are stored in Minio, which is a mandatory component installed with the platform. Usually, the UI is accessible by https://minio-console.<rest of the url to one>.  You should have the link and credentials from the platform onboarding. If not, please contact your Ataccama contact person or our support (in case of the PaaS).

You can then automate the download using e.g. the orchestration server. You would define the connection to Minio as a server connection and use that in a plan to read the data from the files.

You can define the connection to minio in ONE Desktop:

and then use it with resource://minio/export/<rest of the path to the data> in a file path in the Text File Reader step. The bucket in your case is the shared one.

Let me know if this helps or if you have further questions.

Kind regards,

Anna

maykwok_hamilton
Userlevel 2
Badge +2

@jbrant , if your organisation has s3 bucket, ADLS Gen2 storage, or google cloud storage, I’d recommend trying to save the file directly over to those locations. This way you have your own dedicated space, and you won’t need to store things alongside critical application components in minio.

I personally prefer to touch minio as little as possible - if you accidentally move / delete a component on minio it could have very bad consequences.

Reply


Community Terms of Use & ConditionsCookie settings