Getting started with using Manta integration in Ataccama

  • 22 February 2023
  • 2 replies
  • 195 views
L
Rank
Userlevel 4
Badge +5

If you just configured the integration between Manta and Ataccama and have no experience using it, here’s your guide to getting started.

 

Integrating Data Sources

 

First, you’ll want to turn on the integration for the Data Sources in your Ataccama Knowledge Catalog.

  • If you are creating a new data source in ONE, simply mark the checkbox Enable Manta Integration in the Connection details. This will automatically propagate the new source to Manta.
  • If you want to integrate an existing source in ONE, go to Source >> Connection >> Edit and mark the Enable Manta Integration checkbox and Save. 

To confirm that the Connection is successfully propagated to Manta, check that it appears in the Manta Admin UI (Connections >> Databases). 

You might have some data sources connected directly to Manta - it’s fine to keep them alongside the ones propagated from Ataccama. Note that when you trigger Calculate Lineage from the Ataccama ONE app, the Manta workflow will execute lineage revision for all connections enabled in Manta. The lineage information will only appear on a catalog item in ONE if the parent Connection was propagated from ONE to Manta.

You may further edit connections propagated from Ataccama ONE to Manta. Go to the Manta Admin UI and, for example, specify extracted or excluded databases/schemas to limit the scope of lineage revisions.

Once you’re happy with the connections setup, you may trigger lineage revision. Go to Ataccama ONE >> Sources >> Calculate Lineage in the top right corner of the screen. The Ataccama Processing Center will show the status of the Manta lineage revision process.

 

Browsing lineage

 

Once the lineage revision process has finished, basic lineage information will appear in the Lineage tab of catalog items imported from integrated Connections.

From the Lineage tab, you will also have the option to click View Detailed Lineage. Manta Flow Lineage viewer UI will open up with the preloaded lineage visualization of this catalog item.

 

If you see no lineage and/or no button to View Detailed Lineage for a catalog item from an integrated Connection, there're a few things to check:

  • Do you have the MMM module configured properly?
  • Are you looking at an imported Catalog Item? (Lineage is not shown for SQL CIs and VCIs.)
  • Do you have some other catalog items related to the catalog item you’re viewing? If the catalog item you are currently viewing has related tables in the source Connection, but they have not been imported into the Knowledge Catalog, then Ataccama cannot display the lineage. If you import these related tables the lineage will be shown. Alternatively, you can click on View Detailed Lineage and see the full lineage in Manta UI, including tables that aren’t represented in the ONE Catalog.
  • Did you trigger revision calculation from Ataccama ONE? It is possible to trigger the workflow from Manta Admin, but the revision calculated that way will not be imported to Ataccama.

Are you planning to use the integration between Ataccama and Manta? Let us know if you have questions about it!

2 replies

V
Badge +1

Hi @Lisa Kovalskaia thank you for sharing the article, it was helpful to get started, but we have the following additional questions:

  1. We have noticed the DB connection done to SQL data source using jTDS does not successfully propagated to Manta, is this a limitation or something that is missing probably in our set up
  2. If we delete a source in Ataccama, it does not directly delete the source in Manta, I presume we need to do this manually
  3. Can the calculate lineage be scheduled to run at a specific time
  4. Can any source that supports JDBC connection like SAPHANA be used for Lineage calculation? 
  5. Is cross system lineage supported? eg. SQL server and Power BI?
L
Rank
Userlevel 4
Badge +5

Hi @Vikram! Glad to hear you found the article helpful.

Here's some more info with regards to your questions:

  1. We have noticed the DB connection done to SQL data source using jTDS does not successfully propagated to Manta. -- It is indeed a limitation, the integration doesn't support jTDS at the moment. I think this connection type can be created directly in Manta, in that case you'd be able to view its lineage withing the Manta Viewer UI, but it's best to verify through their documentation.
     
  2. If we delete a source in Ataccama, it does not directly delete the source in Manta. -- Hm, that doesn't sound right. The integration should be picking up delete/update/create events and making the changes in Manta accordingly. I would suggest troubleshooting directly with our Support team, could you please open a ticket about the issue and let them know which versions of ONE and Manta you're using? It may be useful to create a couple of test sources and see if the deletion is consistently not propagating to Manta?
     
  3. Can the calculate lineage be scheduled to run at a specific time -- There's no scheduling from the web app, but if you have a bit of experience with our API and the orchestration server, that's doable. You could trigger a Manta revision job with an API call to the Ataccama platform. To schedule it, you can implement an Orchestration Workflow, and use the Scheduler component to set the cadence. 
     
  4. Can any source that supports JDBC connection like SAPHANA be used for Lineage calculation? -- Lineage calculation in itself depends on Manta, so if it supports a given connection type then it can extract the lineage for you. Then, the ability to display the lineage in Ataccama ONE would depend on the integration, which currently supports only a subset of technologies that Manta can handle. In other words, for some technologies the lineage can be calculated and shown in Manta but won't be automatically imported into the ONE web app. There're ways to set up custom import of lineage data, too. The list of supported technologies and custom options varies in different versions and we have a host of updates in the works to expand the integration capabilities. I would encourage you to chat wit your Account Manager at Ataccama, they will be happy to help discover the right approach to your case!
     
  5. Is cross system lineage supported? eg. SQL server and Power BI? -- It's hard to give a short answer to this one, lineage extraction capabilities vary between Manta vs ONE native features vs integration features, so I would suggest exploring this with your Ataccama AM, too. The integration with Manta currently doesn't import that kind of lineage import, but if you're on the latest version of Ataccama and looking for high level details in ONE such as which report is derived from which dataset, that's actually shown natively in ONE when you import PowerBI reports. 

Let me know if you have more questions!

Reply


Community Terms of Use & ConditionsCookie settings