Is it possible to run python code from one web-> transformation plans?

Hello ​@mp_ataccamauser ,

thank you for your question. This is not currently possible. There might be some options to run some external scripts using workflows (and Run Shell script task) but that is AFAIK possible only for self-managed clients that can install Python into their runtime servers. 

Can you tell me what is your usecase and what is your deployment type? Maybe we can suggest something else or raise it as a feature request with our product team.

Kind regards,

Anna

Well I’m not sure of the deployment type, all we know is the requirement of triggering a python file/code on completion of a monitoring project.

We wanted to know if we could use Ataccama workflows if not transformation plans in this case after we set up necessary pre-requisites that it needs.

Hi ​@mp_ataccamauser ,

in that case, it might be easier for you to actually create a Python script to subscribe to events (see example here: https://docs.ataccama.com/one/latest/one-apis/mmm-events-api.html#client-subscription-example) where the event would be the completion of a monitoring project (entityType = dqEvalJob). When any project is done, event would be triggered in your python script and you can then do some additional tasks (call another Python script etc.)

If you want to use Ataccama, you can create a Notification handler (it’s an Ataccama variation of the Python script mentioned above (https://docs.ataccama.com/runtime-server/latest/workflow-and-scheduler/notifications-handler.html) that would call a workflow that inside would call the Python script through the Run Shell script. But as mentioned above, that requires the Python script (and python itself) to be available from the runtime server - for cloud deployments, this is not the case (AFAIK), because python is not part of the runtime server - for this usecase I would suggest to reach out to your Ataccama contact or support and ask for the options you speficially have.

Kind regards,

Anna

Thanks for the suggestions ​@anna.spakova I would like to explore the first approach of subscribing to events. Does the python script start subscription on it’s first run itself, does it mean that every time we run the given MP ID, it triggers the python code shared here (see example here: https://docs.ataccama.com/one/latest/one-apis/mmm-events-api.html#client-subscription-example)

Hello ​@mp_ataccamauser ,

I am not a Python expert, but to my knowledge this script expects to run all the time to be able to receive the events (same as the notification handler in Ataccama). As part of the script there is the subsription part but also consuming the events - async for result in self.ws_client.subscribe_async(...): … which an infinite loop that waits for the server to push the next chunk of events and processes it when it arrives. The subscription needs to be run just once so in theory you can then disconnect and connect back again to start consuming the events later. You should never lose the events which are not consumed. But please note that the server side will eventually discard unused subscriptions if you don’t unsubscribe; retention is controlled by plugin.external-events.ataccama.one.externalevents.subscribers-retention-period.

Inside __process_events, the example just:

• prints the received events,

• counts how many it has seen,

• and once it hits ackLimit, it acknowledges the last event id so the server can safely move the “delivered” pointer forward.

You can replace this with your own logic (the python script you mention above).

Kind regards,

Anna