Master Data Management & Reference Data Management 🛰️

Hello Ataccama gurus--- Has anyone here integrated with OneTrust before? It’s something that’s come up at 2-3 potential projects at different customers recently, and I’m curious of any previous experience people may have nad, or not with this? (What type of data would be kept at each side, methods of synchronisation (assuming API & metadata readers/writers. Any re-usable components available?)Cheers! 

I need the party entity (MDC -ataccama one) to send a notification to kafka when insert or update

When I am publish message in kafka, it throw exception error I am using ataccama 12.7 com.ataccama.nme.core.NmeException: java.lang.NullPointerException at com.ataccama.nme.engine.event.handler.SimpleEventProcessor$NmeSimpleEventProcRuntime.publish(SimpleEventProcessor.java:66) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.mainLoop(NmeEventPublisherBase.java:210) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.run(NmeEventPublisherBase.java:172) at java.base/java.lang.Thread.run(Unknown Source)Caused by: java.lang.NullPointerException at com.ataccama.dqc.streaming.io.writer.KafkaWriter.validate(KafkaWriter.java:130) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursiveInternal(ValidationUtils.java:158) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursive(ValidationUtils.java:127) at com.ataccama.dqc.tasks.common.config.ValidatingStepConfigBase.validateElement(ValidatingStepConfigBase.java:39) at com.atacca

 This article will explain how you can build a relationship between different entities in Gen 2. Out-of-the-box you can have relationship between the same entities - e.g. Catalog Items. But what if you would like to define a relation between different entities? E.g. Source and Report (custom entity)? Following are steps that will explain how to do it. AssumptionsYou have your two entities between which you would like to have a relationship. For this article we will use source (default entitiy) and report (custom). Create the relationship entityFirst, go to the Global Settings (or Organization) → Metadata Model and create a new entity that will store the relationships. Name it e.g. sourceReportRelationship. It will have following properties:order (Integer) source (Referenced Object: source) target (Referenced Object: report) type (Referenced Object: relationshipType). Put the new relationship entity in metadata entityUnder Metadata model, navigate to the entity metadata. Add a new prope

Hi Team, I have few data loads that read data using API’s and writing to Snowflake tables. Often the data is incomplete & inconsistent. Im checking if I can use Attacama to compare the data between different snowflake schema’s : eg) Raw vs Curated? I understand we can write SQL Execute (select col1 from Tab A minus select col1 from Tab b and create a plan. But i have 500+ jobs and needs a lot of plans to be created. Looking for correct way of doing this?  

Tested on v13.3Steps overview:Download all components Install ONE Desktop Launch ONE Desktop and create the MDM Banking example project Place the files in the right structure Modify a bunch of files so they are working on MacOS Configure the Mac permissions to run things Launch ONE Desktop and create the MDM Banking example project Launch it all (keycloak, postrgres), one by oneDownload:Builds (Latest versions here): MDM: MDM-Assembly-13.3.2-final-linux.zip ONE Desktop: Ataccama-one-desktop-13.2.1.final-macos.dmg ONE Runtime: Server-assembly-13.3.2-final.tar Plugins (Latest versions here, the official Postgres version for Linux missing for now): Postgres: pgsql.zip Keycloak: https://ataccama.s3.amazonaws.com/Frances/Ataccama-one-desktop-13.0.0-GA_keycloak_12_0_4_plugin.zip Your license (.plf file)For latest versions of the data (optional):ataccama.json permissions file, found hereInstall ONE Desktop:Install One Desktop (Ataccama-one-desktop-13.2.1.final-macos.dmg) Rename

Hi Team, I have bunch of queries sitting in my Oracle tables(queries in one of the column). I am trying to see if I can read the oracle table as source and use the column values(which has sql queries ) to be sent to database & get return value. Is there a component that can help me achieve that? jdbc reader, sql execute components expect you to hard code the query. But in my case the query will come as value from one the table.  eg) Col1 Col2 1 select count(*) from tab A 2 select count(*) from tab B In the above example I wanted to use col2 as input and pass the values from Col2 to DB and assign the output to next transformation. Hope its clear.

Copy Columns (Instance Layer)Copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, there must exist a relationship definition between entities. After it is defined, it is possible to define the direction of copy columns: parent to child or child to parent.In the following example we will create a matching rule (firstn+lastn+bd+con) for Party entity (Parent entity) with same First Name, Last Name, Birthday, Contact (Child entity). Notice that the contact information is stored in the Contact entity which is a child to Party entity. In that case we would need to transfer the information (contact) into the party entity in order to use it in the matching on that particular entity. In that case we will use Copy Columns functionality to move and store associated contacts of the party record directly in the party entity.The relationship of Party (Parent) to Contact (Child) is 1:N, which means that a particular party can have 1 or m

First part of the tutorial - Ataccama Admin Centre: ServerConfig and RuntimeConfig for beginners (Self-managed installations) 1/2Use cases Admin Centre Security 1. ServerConfig 2. RuntimeConfig Online service endpoint 1. ServerConfig for OnlineServicesComponent 2. RuntimeConfig for pathvar and rdmapp 3. ServerConfig for Http dispatcher Workflow and Scheduler ServerConfig for Scheduler component configuration ServerConfig for Workflow component configuration RuntimeConfig for workflow and components Security for workflow by role Deploy your files and test them locally Check the UI and Server Health Status Test User Permissions Deploy to serverUse casesNow let’s consider some use cases and see how we can set up the Admin Centre to meet our needs.Security on the Admin Centre UI Online service endpoint Schedulers, workflows and plans / componentsAdmin Centre SecuritySecurity on the Admin Centre UI itself is taken care of by the Http dispatcher Component. The one-executor.serv

We are trying to connect Ataccama with Python code. Could you please help us with what are the possible ways to connect with python, it would be really great if you can share us with any reference links or docs. Thanks in advance.

Hi  Folks, I’m super-new to Ataccama - in fact I haven’t even had a hands-on yet. Curious  to know whether we can embed content from Ataccama onto Confluence pages via an API or  something? Anyone done this before? Mick

As part of the continues integration development, I would like to implement Unit testing automation, this topic is to discuss some ideas of how to do that in Atacama, I have some specific doubts like:Architecture suggested for unit testing automation  Database approach for unit testingThanks in advance

When using Run Shell Script task in a Workflow, be careful that this task uses a different expression markup: %expression()% Link to documentation: Run Shell Script

Hello there,Please, when sending data about a person to MDM, can we differentiate this data between VERIFIED and UNVERIFIED?I mean, unverified DATA should be stored, but it should never overwrite the existing verified data in MDM.Example:User ELON MUSK, 60 years old, elon@tesla.com, California, etc is created as verified data in MDM and master-ID ABC is assigned to his record. Months later someone tries to create a profile and the data is not yet verified, that data would be ELON MUSKK, 58 years old, elon@tesla.com, Texas.   The end result coming from from a “GET” should be ELON MUSK, 60 years old, elon@tesla.com, California, master-ID ID ABCIt would be OK to MDM store that UNVERIFIED Data somewhere, but it should NEVER overwrite VERIFIED data nor consider UNVERIFIED information as correct data to be returned to MDM callers/consumers.I know that other MDMs keep unverified data separated of VERIFIED data, I would like to know what is the recommended way to do this in Ataccama.Any help w

I am getting the error “Column Name cannot be empty” under StringValue. When I click on the drop down, there’s no options available. 

What is LDAP?LDAP is directory that stores companies usernames, passwords, email addresses, connection information and much more. It is usually used due to it's ability to allow hundreds of active connections at any given point.In terms of configuration and connectivity, in Ataccama we authenticate/authorize through a 3rd party tool called Keycloak. Items to consider prior to connecting to LDAP - Best PracticeYou first need to understand how your organizations Active Directory is setup. Some pre-requirements needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, LDAP groups is a way where we can map Ataccama Roles to a single group. This will heavily assist in ease of maintenance going forward and not leave your devops team stuck doing manual work such as adding/removing new users each time someone is added/removed from LDAP. Once you have dec

Hi, is there a method to make a “wait n seconds” between 2 steps?Thanks a lot

GeneralIt is strongly recommended not to place an Eclipse Project directly inside the git repository.That means, avoid having .git directory and .project file at the same level - instead, place the project into a sub directory.On the other hand, there are particular config files which you typically keep at the same level as .git directory:  .gitattributes and .gitignore. We use them to control some processes when working with Git. Note: settings in these files will override any git global settings on your computer.Example of recommended setup is as follows: .gitattributesThe file provides a possibility to control certain operations in the git repository. Especially it is important to keep the line endings (prevents Git from normalizing them) as provided in the Ataccama default Eclipse Project content because:Prevents the "The project signature is invalid" issue resulting into a state when you are not able to see/work with the Project at all (entire project tree is not visible). Prevent

Hi,Am exploring Ataccama MDM and would like to know if Ataccama MDM supports Multiple language in matching and data maintenance. Appreciate your help. Thanks.Vinoth

Hi,When importing a flat file, if the data has double quotes or single quotes or some special characters then they are automatically being replaced with ‘?’s. This is not as expected. This is the same issue for all encoding formats. Could you please suggest how to fix this? Thanks,Kranthi 

Simply explained, reference data is data that serves as the model for all other data throughout an organization. It is contained within tables and databases, which help bridge one source system to another, and provides the foundation for the entire data architecture. Together with the initiative set forth by the overall data governance program, reference data maintains consistent data across all departments and allows for source system inter-communication. Reference Data tables are entities that compose the framework of the reference data architecture, providing lists of values for each department and the company as a whole. They contain either the backbone data in its base form (central authority) or data that is a derivative of the base-form data (tables used in the various branch systems throughout the enterprise).Each department within a company works with table data fields in a particular format that suits their processes. Although department A might deal with the same product as

We would like to have a user group in AtaccamaOne so that when new people join, we do not have to manually add them for access of rules, catalogs or projects. Simply adding them to a user group should give them all the necessary access. How to create such a user group?

Hi there,Imagine Ataccama is the primary source of truth (transactional) for PII information inside a Financial Service company. Now, we would have different ONBOARDING systems (one for each Financial Line of Business that we offer). When a new person tries to create a profile, the Onboard system would (while the user is still looking into the APP) SYNCHRONOUSLY call an Ataccama API with meaningful data that we collected from this person, say: (Full Name, DoB, Insurance Number / Any other ID document, Email(s), Phone(s))m then Ataccama would quickly do the “Match” and return if this person (according to the information we provided) is likely to ALREADY exist. If the probability is higher than some threshold (Ie. 95%), it would also return the MASTER UUID for this person in Ataccama (Lookup key). Then the onboarding would ask this person to log in, as this person ALREADY exists in our Ataccama Database.Can Ataccama do the MATCH and return the % probability in near real-time? Instead of

1 entity Party→ 2 entities (Person/Organization)or 1 entity Contact → 2 entities (Phone/Email) etc Pros Clean Data Model (on attribute level)In general, we have only columns that are needed for a particular entity (Person) or (Organization). This means that, for example we do see the company_name attribute on the Person records, etc. So in general we have fewer columns in the tables. This could be especially important for the generated MDM native services format (WSDL in SOAP, or all attributes used in the REST API). The consumers of the web services only see attributes per definition of the given model entity and don’t see any irrelevant attributes (similar to the data hiding/encapsulation principle in the Object-Oriented approach).This can also help minimize the area of impact in case of a required model change (the impact to online API change can be countered with custom .online services exposed as wrappers to the MDM native services keeping the API stable with a fixed set of attrib

IntroductionWe did an internal performance PoC for Streaming. We are reading messages (“msg”) from Kafka or Java messaging service. Parameters In order to tune the performance of the streaming we need to modify 2 main parameters on MDM side:settings.count = how many messages we need to consume in one single batch settings.timeout_s = what is the maximum timeout to wait for this number of messages Average duration of the task vs Records per second We are not aiming for the throughput (records per second) but we are aiming for the average duration of the task. The goal is to run it under 1 sec (1000ms). In various scenarios we have to increase the settings.count parameter starting from 1, 20, 30 etc. Scenarios Below are the different scenarios of using different settings for various tasks/transactions in MDM within 10 mins (600sec)For example of timeout of 1 sec and settings.count = 1000 (scenario , that means that we try to consume 1000 msg at once within this 1 sec. We will consume wha