Master Data Management & Reference Data Management 🛰️

Let’s say I have a table that has 500,000 rows.I need to query this table using REST API (because I want to get both published records and records still in edited/waiting for publish states).I have some params to filter the data with, but even with params, the result set could be say, 10000.I tried using the REST API using JSON call step in a plan, setting “count” to 10000, and the API says maximum is 1000.I see that the REST API provides the offset param as well. Is there an example of how I can use that offset param to paginate and query all the records that I will need, but in one plan/component?This component will be a part of a complex set of components as a larger operation, and I’d prefer to not have to create an extra .ewf and iterator for it, otherwise this whole thing gets too complicate for anyone else to understand… Any ideas?

Need help with graphQL query. I need to bring in All the metrics like DQ scores, Overall quality for CIs that are part of monitoring projects. Is there a way to get it? Also, do we have document which tells us more about when to use edges and nodes appropriately.

Hello,Is it possible to self-join tables in RDM to populate some information? I would like to try something like shown in the screenshot. Attribute 4 and 5 values are selected from RDM UI. So, dropdown should only show values from the same table where att2 = CAT2 for Att4 and att2 = CAT3 for Att4Table1                 Selection from RDM UI   Att1 Att2  Att3 Att4 (where att2 = CAT2) Att5 (where att2 = CAT3)   ID1 CAT1 val1 ID3 ID5   ID2 CAT1 val2 ID4 ID6   ID3 CAT2 val3       ID4 CAT2 val4       ID5 CAT3 val5       ID6 CAT3 val6                    I somehow tried to create self join in relationship like below. I was able toi upload configuration successfully on RDM with these relationships.  However, when edit the record on RDM the lookup is blank. it does not show any value although table is populated.   (this can also be understood as employee info vs manager info relationship.

We have multiple Realm roles created for our data domains in the below fashionData Domain 1 Realm Role - Data Owner Realm Role - Data Steward Data Domain 2 Realm Role - Data Owner Realm Role - Data Steward How can I merge Data Domain 1 and Data Domain 2 into one single New Domain and merge the corresponding Realm Roles?I.e.Data Domain 1 Realm Role - Data Owner Realm Role - Data Steward                                            = New Data Domain Data Domain 2                                                                                           New Realm Role - Data Owner Realm Role - Data Owner                                                              New Realm Role - Data Steward Realm Role - Data Steward

How can we retrieve or read the workflow status (running, success, or failed. Etc) using a GET API after triggering the workflow via an Http service?

Hello, I wanted to know that how do we implement matching options in the master layer of the MDM project.I have used in the columns in the instance layer and then used the same in the master layer in the matching step just as used in the cdi example. and then I have used the same name in the matching step but still  nothing is  showing the UI part. Can someone help me with that.

Hello everyone,My organization will soon be upgrading to a higher version of Ataccama, and I want to make sure the transition goes as smoothly as possible. I'm reaching out to ask:What should I be documenting or looking out for before, during, and after the upgrade?Some specific things I'm wondering: Are there key areas (like DQ rules, workflows, configurations, etc.) that I should definitely back up or document? What kind of testing should I plan for in UAT? Should I be validating things both before and after the upgrade? Are there any common issues or changes I should be aware of that might affect how things work in the new version? If anyone has experience with a recent version upgrade, I’d love to hear what worked well for you—or what you wish you had done differently!Also, if there are any official resources or checklists from Ataccama that you found useful, feel free to share.Thanks in advance!

07.05.2025 16:24:30 [INFO]     Using Ataccama DQC configuration file D:\ataccama\workspace1\General MDM project - CDI example\Files\engine\trans\address\address_match.comp07.05.2025 16:24:30 [INFO]     Using runtime configuration file C:\Users\AS001086183\AppData\Local\Temp\runtimeCfg31836356917877677.tmp07.05.2025 16:24:30 [INFO]     Ataccama DQC engine initialized.07.05.2025 16:24:30 [INFO]     Creating runtime...07.05.2025 16:24:31 [INFO]    [Address deduplication] parallel processing of partitions: false07.05.2025 16:24:31 [INFO]    [Address deduplication] parallel processing of key rules: false07.05.2025 16:24:31 [INFO]    [Address deduplication] parallel processing of matching rules: false07.05.2025 16:24:31 [INFO]     Starting runtime...07.05.2025 16:24:31 [INFO]     Running runtime...07.05.2025 16:24:31 [FATAL]    Internal error occurred during run of the plan: Persistence is not set (the standalone mode may not be configured)[STEP Address deduplication[Md Unify]]07.05.2025 16:

Hello everyone,I’m working on a project that integrates a data masking tool with Ataccama, and I’m encountering some issues when trying to retrieve actual catalog item data (the row-level data) via the REST API. I’ve been testing both GraphQL and REST approaches, and here’s what I’ve found so far:GraphQL vs. REST for Data Retrieval:The GraphQL API appears to focus on metadata – attributes, versioning, policies, etc. However, based on my understanding, the actual table data (the rows and cells) isn’t available through GraphQL; it’s only accessible via a REST endpoint. Based on available documentation for MDM and RDM REST APIs, the REST endpoints follow a pattern using a fixed base path (/api/rest) on the default port (8051) and use basic HTTP authentication.Endpoint URL Confusion:Initially, I attempted to use an endpoint like:  https://teranet.test.ataccama.online/catalog/data/catalogItem/<gid>/datawhich returned a 404 error. Later, based on the docs, I switched to the documented

 I am trying to do validation on Tin , on which score and explanation are getting populated but the records which are invalid are not getting highlighted in red and i have enabled validation in the Tin columnAttaching the snapshots of the steps which i have done while configuring the validation:  How should i fix this.

We made changes into the clean operation of an entity. it will take effects for new transactions.How can we apply this to the current records that we already have into the database? We are exploring use a full reprocess of this entity:but we have some doubts about the setup. For example:What is the difference between a reprocess vs rematch checkbox? We tried to read the documentation but there is not detail explanation about it. Thanks in advance!

I have been trying to explore the Reprocess and Rematch options in MDM.From this post I understand that Reprocess is used to reprocess the records. For example, lets say, after an initial load where the email values were not being modified in contact_clean.comp, we are now to determine which emails have the domain company-name.com and are now being updated to have domain companyname.com . So, I need to run the reprocess job.Rematch is to run the match step on top of the reprocess since the email values have been modified. What I do not understand is do I run a rematch job only if the email value participates in determining the master record or do I need to run it whenever the reprocess jobs is run irrespective of the changes that took place due to a reprocess?There is another question I have. When we run a load job, the change detection is done right. Does this change detection run on all the records or just the delta records. Post the change detection, matching and merging is run only

Hello all,I’m using the Ataccama v14.5 there we have created the custom entity for classifications and created the classification types (Internal, Public etc) and added them with tables and attributes through the One Desktop plan. But when i am using the same plan to delete it from table or attributes just by changing the One Metadata Writer property in General tab “workflow state as deleted” and in the Columns tab providing the “Id Column Name as table_id” then it is not deleting anything. Any guidance or suggestions is really helpful. Thanks.General tab Screenshot:Columns tab screenshot: 

I have MDM setup and records are loaded using a Full Laod configuration. I need to create a load such that all the records that were added into the source are moved into to MDM hub on a daily basis. The approach being used is to identify all the records updated today and load them into MDM. Can you please suggest if I need to create a Delta Load for this or Full Load works. Also, I need to schedule a run of the load file on daily basis.How can I do it?

The Workflow task Run MDM Export does not recognize the export component and and gives error message.I am assuming it is because there is no path variable set for the Export folder.Can someone please help me identify the issue?

I have a workflow with a file iterator step. It runs without any issues in the IDE on my local machine, but whenever I deploy it my company’s RDM server, I get the following error:2025-01-07 09:05:31 INFO: starting...2025-01-07 09:05:31 ERROR: Problem during initialization of iterator valuejava.io.IOException: An unexpected network error occurred    at java.io.WinNTFileSystem.canonicalize0(Native Method)    at java.io.WinNTFileSystem.canonicalize(Unknown Source)    at java.io.File.getCanonicalPath(Unknown Source)    at com.ataccama.dqc.resources.local.FileResource.getCanonicalPath(FileResource.java:40)    at com.ataccama.dqc.resources.impl.DelegatingResourceBase.getCanonicalPath(DelegatingResourceBase.java:41)    at com.ataccama.adt.task.iterators.FileIterator$FileIteratorInstance.iterate(FileIterator.java:133)    at com.ataccama.adt.task.iterators.TaskIteratorInstance.initAndIterate(TaskIteratorInstance.java:36)    at com.ataccama.adt.task.exec.EwfForEachTaskInstance.run(EwfForEachTas

I  have a full load already configured for a source system. This load component has 3 different entities(party, contact, address).I have used this to load data into mdm-hub as well.However, now we want to use a different source table for party entity and want to load only the records for party entity. I am planning on using a new full load with mappings only for the party entity. Will thiscreate new instance party records and make the old records inactive? bring any change to the records for contact and address entities? 

Security Assertion Markup Language (SAML) allows users to use one set of credentials to log into many different websites. It is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers(SP). SAML transactions use Extensible Markup Language (XML) for standardized communications between the identity provider and service providers. SAML is the link between the authentication of a user’s identity and the authorization to use a service. Items to consider prior to connecting to SAML- Best PracticeIt is important to understand how your organization’s Active Directory is setup. Some pre-requisites needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, with AD Groups we can map Ataccama Roles to a single group. This benefit of doing this is, your internal dev-ops team won’t have to manually add/remove u

Our Incremental load operation is taking huge time (2 hrs to process 5-7K records). We have applied all performance tuning things which is there in document. Apart from this we thought of loading data from DW to PostgreSQL (MDM DB) stage table and then will use that stage table as a source in Incremental load. Idea is when the server will be same for both source and target the computation will be faster.But this direct load job is failing with below error. Kindly help on fixing this. Or please recommend on any other approach.com.ataccama.nme.core.NmeExecutionException: Plan "C:\Users\EYKU4F\Ataccama\workspace\OmniChannelMDM_Latest\Files\engine\load\EPRISM_stg_eprism.comp" execution failed: runtime.error.jdbc_reader_error - nullcom.ataccama.dqc.model.environment.AbortedExecutionException at com.ataccama.dqc.model.elements.data.flow.RecordQueue.putBatch(RecordQueue.java:173) at com.ataccama.dqc.model.elements.data.flow.RecordQueue.access$100(RecordQueue.java:35) at com.ataccama.dqc.model

Hi  We are noticing behaviour in our model where a source system will split from its golden id group when updated on matching criteria fields ( in this example address). However it only does this when the group of records was manually merged onto the golden id. It wont “jump ship” and form a new golden id when the records previously auto merged based on the matched criteria we assigned in the model.Is there a way we can stop this splitting from happening when the source system is updated? For example we can see here the new golden id created with one SF record  the previous group the SF record belonged to, shows a lot of manual merging in the Match Rule Name column Our rematch if changed section shows these fields are used for rematching:  We want to keep rematching switched on, to bring other source systems into the group if they meet criteria, but we dont updates on the SF record itself to allow it to break out of the group. (We dont want “auto-splits”) thanks