Skip to main content
Solved

Availability of API's in Data Quality to check results ?

  • 2 July 2024
  • 6 replies
  • 95 views

Hi We are planning to use Atacamma DQ as a service and would like to use its DQ Engine to query and provide us counts. We would like to build a API call to Atacamma DQ with necessary SQL Query and connection parameters as part of API call, and would like to know if this SQL query can be executed on the Atacamma DQ and provide the results as an output back to API as a response. Please let us know the possibilities.

 

Regards,

Uma

6 replies

S
Rank
Userlevel 4
Badge

Hi @Uma ,

We can achieve this by using DQ Engine and pull the table from the database using the jdbc reader and get the count. Once you have the count you can use the json call API and the count of the results back to API as a response any where you want to publish.

Hope this helps! If you have any questions please provide more details so I can help. 

Regards,

Srija Piratla

 

Cansu
Rank
Userlevel 6
Badge +3

Hi @Uma, welcome back to the community! I’m closing this thread for now, if you have any follow-up questions please do feel free to share them here in the comments or create a new post 🙋‍♀️

Check out Our Community Recommended Journey to get started on the community 🙋‍♀️
U
Rank

Hi Srija,

  Thanks for your response, im curious to know how i can pass a query as an input to the DQ Engine.

Given the circumstances is that for each API call, i would be passing below parameters..

  1. Data source of the connection to which i need to connect
  2. User Id/ Password  - How do i send this information, can i use a vault like mechanism which we use during scanning
  3. Query 

Regards,

Uma

S
Rank
Userlevel 4
Badge

Hi @Uma,

We can achieve this in below method

1st Method: (If the database that we are using is static)
1) In Ataccama, we have to create a database connection for which we are accessing if we have the connection details of the database along with the user name and password.
2) we can use the SQL SELECT step to query the database instead of USING the API.
3) The results of the query can be transferred to the desired location using the API. To achieve this we have to use, JSON call step and POST methodology


2nd Method: (If the database that we are using is dynamic)
1) We have to create all the database connections that we will be using prior in the Ataccama if we have the connection details of the database along with the user name and password.
2) We can use JSON Call to extract the data source name and the Query using the GET method.
3) We can use the SQL SELECT step to query the database instead of USING the API.
4) The results of the query can be transferred to the desired location using the API. To achieve this we have to use, JSON call step and POST methodology

Regards,

Srija Piratla

U
Rank

Hi Srija,

  2nd method is what seems to be a go to solution, but im not clear on step 2 onwards.

we will have all connections created in Atacamma prior to this invocation call.

the way we are looking at the trigger is as below..

Application will invoke a API Call ( This API Call will have the below parameters )

  1. Connection Details
  2. Query for doing the necessary validation (Basically a select query, where the output is a count)
  3. Output if count is > 0 False, count <0 is True some kind of boolean output is what we are expecting.

If i understand the flow you had written

we need to build a workflow which will take the connection id details as input and bring in all necessary parameters for making a DB connection to run the query ?

Please confirm.

 

Regards,

Uma

R
Rank

Hi @srija piratla ,

Can you please provide some more insights on implementation of above process.

 

Regards,

Rohith

Reply


Community Terms of Use & ConditionsCookie settings