Skip to main content

Hello!

 

I am trying to think of a way through building a single post processing plan and or component that is able to be applied to every monitoring project in my ecosystem. It is a very simple plan, I will just have a filter for when invalid_rules_explanation is not null, and export to a default CSV location. The issue I am running into is that in all documentation, it is project / catalog specific when setting up a post processing plan. For a company with numerous projects and catalogs (around 500 unique configurations and more coming), I do not want to manually have to create the same post processing plan for every project and catalog. Is there a way to automate this, or have something within the UI of the web version that would allow me to do this more seamlessly?

Hello ​@ThomasPro ,

 

thank you for your question. At this moment, there is no simple way how to automatically create multiple post-processing components. In theory, it should be possible via a combination of API calls and uploading components to Minio, but the process is quite compliated and can result in inconsisten state.

Depepnding on your version, there is a new feature - Tranformation plans - that allows users to create simple post-processing plans via front-end application. There are some limitations, though, e.g. hybrid deployment has currently some limitations, but you could try on one example and see if it’s a viable option. https://docs.ataccama.com/one/latest/monitoring-projects/data-transformation-plans.html

Why I am suggesting this - from what I observed, it is using a standard graphQL API, so you should be able to get all the calls through e.g. the GraphQL inspector plugin in browser and prepare a script/plan that could automate it for all your projects. I haven’t tried myself, but might be worth a shot.

Also, it is one of the points our development team wants to address in the new cloud verions of Ataccama, so hopefully we will be able to resolve this issue with post-processing components going forward.

Kind regards,

Anna

Reply


Community Terms of Use & ConditionsCookie settings