Hello fellow data enthusiasts,
We’re starting to look into the lineage feature of Ataccama and we want to get it right. Calling other customers who already use lineage, and also if a representative from the Product team could chime in, that’d be amazing!
I’m reviewing how our data catalog sources / connections are set up, and how that needs to / doesn’t need to relate to how the scan plans are configured.
For the catalog, for example:
- Sources: A source system. I’m in the insurance business, we’d have Policy Admin systems, Claims systems, Accounting systems, Datawarehouse etc. I’d put each one of them separately
- Connections: Possibility of multiple connections for a given system, as each system might have separate databases. We use MS SQL and we do have a database server that has multiple databases sitting in it, and data moves between them.
- SQL CIs / VCIs: These go under the source into the workspace / folders
Does this have any bearing as to how the lineage scan plans are set up?
- One scan plan for the database server
- Lineage import one scan plan into multiple connections
- One scan plan for each database, so there will be multiple scan plans
- Lineage import each scan plan into multiple connections
Trying to balance a good user experience for my team, neatly organised on the platform so it doesn’t look chaotic, easy to manage and maintain, and of course reasonable performance.
Any ideas and input greatly appreciated!