Skip to main content

Master Data Management & Reference Data Management 🛰️

Discuss, learn, ask, suggest, share all things MDM and RDM 🔍

  • 256 Topics
  • 590 Replies
258 Topics
A
Star Blazer L1
aish_TFStar Blazer L1
posted in Master Data Management & Reference Data Management 🛰️
Usage of Rematch and Reprocess jobs

I have been trying to explore the Reprocess and Rematch options in MDM.From this post I understand that Reprocess is used to reprocess the records. For example, lets say, after an initial load where the email values were not being modified in contact_clean.comp, we are now to determine which emails have the domain company-name.com and are now being updated to have domain companyname.com . So, I need to run the reprocess job.Rematch is to run the match step on top of the reprocess since the email values have been modified. What I do not understand is do I run a rematch job only if the email value participates in determining the master record or do I need to run it whenever the reprocess jobs is run irrespective of the changes that took place due to a reprocess?There is another question I have. When we run a load job, the change detection is done right. Does this change detection run on all the records or just the delta records. Post the change detection, matching and merging is run only

376
Cansu
Community Manager
A
Data Voyager
A_NData Voyager
asked in Master Data Management & Reference Data Management 🛰️
How to delete the classification types from the tables and attributes level through One Desktop Plan?

Hello all,I’m using the Ataccama v14.5 there we have created the custom entity for classifications and created the classification types (Internal, Public etc) and added them with tables and attributes through the One Desktop plan. But when i am using the same plan to delete it from table or attributes just by changing the One Metadata Writer property in General tab “workflow state as deleted” and in the Columns tab providing the “Id Column Name as table_id” then it is not deleting anything. Any guidance or suggestions is really helpful. Thanks.General tab Screenshot:Columns tab screenshot: 

192
Cansu
Community Manager
K
Data Pioneer L2
KundanData Pioneer L2
asked in Master Data Management & Reference Data Management 🛰️
Schedule load in MDM

I have MDM setup and records are loaded using a Full Laod configuration. I need to create a load such that all the records that were added into the source are moved into to MDM hub on a daily basis. The approach being used is to identify all the records updated today and load them into MDM. Can you please suggest if I need to create a Delta Load for this or Full Load works. Also, I need to schedule a run of the load file on daily basis.How can I do it?

466
A
Star Blazer L1
A
Star Blazer L1
aish_TFStar Blazer L1
asked in Master Data Management & Reference Data Management 🛰️
Run MDM Export workflow task does not recognize the export component

The Workflow task Run MDM Export does not recognize the export component and and gives error message.I am assuming it is because there is no path variable set for the Export folder.Can someone please help me identify the issue?

103
A
Star Blazer L1
H
Data Voyager
HMJohnData Voyager
asked in Master Data Management & Reference Data Management 🛰️
File iterator task fails when workflow deployed to server

I have a workflow with a file iterator step. It runs without any issues in the IDE on my local machine, but whenever I deploy it my company’s RDM server, I get the following error:2025-01-07 09:05:31 INFO: starting...2025-01-07 09:05:31 ERROR: Problem during initialization of iterator valuejava.io.IOException: An unexpected network error occurred    at java.io.WinNTFileSystem.canonicalize0(Native Method)    at java.io.WinNTFileSystem.canonicalize(Unknown Source)    at java.io.File.getCanonicalPath(Unknown Source)    at com.ataccama.dqc.resources.local.FileResource.getCanonicalPath(FileResource.java:40)    at com.ataccama.dqc.resources.impl.DelegatingResourceBase.getCanonicalPath(DelegatingResourceBase.java:41)    at com.ataccama.adt.task.iterators.FileIterator$FileIteratorInstance.iterate(FileIterator.java:133)    at com.ataccama.adt.task.iterators.TaskIteratorInstance.initAndIterate(TaskIteratorInstance.java:36)    at com.ataccama.adt.task.exec.EwfForEachTaskInstance.run(EwfForEachTas

254
H
Data Voyager
OGordon100
Ataccamer
OGordon100Ataccamer
published in Master Data Management & Reference Data Management 🛰️
📚 Guide: How to run GraphQL requests - Part II Working with MMM

Hi everyone! I’m back with the second and last part of the GraphQL best practice series. First part was all about authentication types & which one you should use and how to authenticate GraphQL requests. This post will focus on how to work with MMM in GraphQL - so let’s get into it.If you’d like to check out the first part I’m sharing it here 👇 ONE Metadata Readers Or GraphQL? MMM is the name for the linked list that holds all metadata and the connections between them. Pretty much everything in MMM can be explored via GraphQL - as GraphQL is what happens behind the scenes in most steps/actions in ONE Desktop.As far as ONE Desktop is concerned, while ONE Metadata Readers are often all you need, highly nested queries and associated joins can become quite tricky to handle via ONE Desktop plans, and can sometimes be simpler to build by just running a specific GraphQL query. In addition to that, some things, such as raw profiling information can only be extracted via GraphQL. Exploring

250
OGordon100
Ataccamer
A
Star Blazer L1
aish_TFStar Blazer L1
asked in Master Data Management & Reference Data Management 🛰️
New Full Load for an already loaded source data

I  have a full load already configured for a source system. This load component has 3 different entities(party, contact, address).I have used this to load data into mdm-hub as well.However, now we want to use a different source table for party entity and want to load only the records for party entity. I am planning on using a new full load with mappings only for the party entity. Will thiscreate new instance party records and make the old records inactive? bring any change to the records for contact and address entities? 

405
A
Star Blazer L1
john.efechaobor
Ataccamer
john.efechaoborAtaccamer
published in Master Data Management & Reference Data Management 🛰️
Connecting Azure AD to Keycloak (SAML)Best practice

Security Assertion Markup Language (SAML) allows users to use one set of credentials to log into many different websites. It is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers(SP). SAML transactions use Extensible Markup Language (XML) for standardized communications between the identity provider and service providers. SAML is the link between the authentication of a user’s identity and the authorization to use a service. Items to consider prior to connecting to SAML- Best PracticeIt is important to understand how your organization’s Active Directory is setup. Some pre-requisites needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, with AD Groups we can map Ataccama Roles to a single group. This benefit of doing this is, your internal dev-ops team won’t have to manually add/remove u

91933
David Mecca
Ataccamer
F
Universe Traveller
fshahinUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
Direct Load Operation failure

Our Incremental load operation is taking huge time (2 hrs to process 5-7K records). We have applied all performance tuning things which is there in document. Apart from this we thought of loading data from DW to PostgreSQL (MDM DB) stage table and then will use that stage table as a source in Incremental load. Idea is when the server will be same for both source and target the computation will be faster.But this direct load job is failing with below error. Kindly help on fixing this. Or please recommend on any other approach.com.ataccama.nme.core.NmeExecutionException: Plan "C:\Users\EYKU4F\Ataccama\workspace\OmniChannelMDM_Latest\Files\engine\load\EPRISM_stg_eprism.comp" execution failed: runtime.error.jdbc_reader_error - nullcom.ataccama.dqc.model.environment.AbortedExecutionException at com.ataccama.dqc.model.elements.data.flow.RecordQueue.putBatch(RecordQueue.java:173) at com.ataccama.dqc.model.elements.data.flow.RecordQueue.access$100(RecordQueue.java:35) at com.ataccama.dqc.model

223
Cansu
Community Manager
jduha
Ataccamer
jduhaAtaccamer
published in Master Data Management & Reference Data Management 🛰️
Spice Up Your MDM with Emojis! 🎉

Hi community 👋 Every now and then, we hear from our users wondering if it’s possible to spice up their Master Data Management (MDM) interface with something a little more visual.Recently, I worked with one of our members who wanted to mark one of the records in a group as having a special meaning, so that it would instantly stand out to data stewards. While the logic behind selecting this record was complex, the outcome boiled down to a simple boolean flag, like cmo_preferred_flag = {true | false}. However, displaying such an attribute on the screen isn’t exactly thrilling:In earlier versions, it was possible to load an image file to represent things like this, but it required customization that was tricky to maintain. Plus, it often got wiped out when regenerating the MDM project. In our current versions (14.5.x / 15.x), we’ve moved to SpringBoot, and for security reasons, loading external static content is no longer supported.So, can we solve this in a fun and practical way?The emoj

330
jduha
Ataccamer
M
Data Voyager
mauriceKData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Updates creates a record to split from its golden id

Hi  We are noticing behaviour in our model where a source system will split from its golden id group when updated on matching criteria fields ( in this example address). However it only does this when the group of records was manually merged onto the golden id. It wont “jump ship” and form a new golden id when the records previously auto merged based on the matched criteria we assigned in the model.Is there a way we can stop this splitting from happening when the source system is updated? For example we can see here the new golden id created with one SF record  the previous group the SF record belonged to, shows a lot of manual merging in the Match Rule Name column Our rematch if changed section shows these fields are used for rematching:  We want to keep rematching switched on, to bring other source systems into the group if they meet criteria, but we dont updates on the SF record itself to allow it to break out of the group. (We dont want “auto-splits”) thanks

284
Cansu
Community Manager
F
Universe Traveller
FionaFUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
Error during manual Merging

While trying to manually merge 2 records, facing an error stating permission issue. Followed above mentioned step to merge records. Error message 

476
Cansu
Community Manager
alexAguilarMx
Data Pioneer
alexAguilarMxData Pioneer
asked in Master Data Management & Reference Data Management 🛰️
how to apply new changes into clean operation for records already in the db?

We made changes into the clean operation of an entity. it will take effects for new transactions.How can we apply this to the current records that we already have into the database? We are exploring use a full reprocess of this entity:but we have some doubts about the setup. For example:What is the difference between a reprocess vs rematch checkbox? We tried to read the documentation but there is not detail explanation about it. Thanks in advance!

734
A
Star Blazer L1
T
Data Voyager
TylerData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Set MDM records to inactive based on source flag and not delete

Hello, from one of our source systems we were previously excluding any inactive addresses, contact details, etc. Our users have requested we load these records into MDM for visibility. My issue is that while we can add these records as is, they will not show as inactive on MDM. We could add a source active flag but for consistency with other sources we would like them to show as inactive on MDM (Greyed out). We have considered doing a one time load of all the inactive instances and then excluding them from the daily load but because it is possible to create address and contact details as inactive from the source that would not work. I have explored using the delta load as it has a change type column that could be appropriate for our use case.Has anyone had to set records in MDM as inactive as part of the load not using the deletion strategy? What method did you use?I have also looked into creating new instance entities to store inactive records but that would be a large undertaking as

644
Cansu
Community Manager
R
Data Voyager
ritesh.ranjanData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Apply Last Changed Date Field Delta Change only on Instance Layer and not on Master Layer

Hi @Cansu and AllIn the event if there is no change in the delta records coming in MDM except one of the Sequence Generated column, is their a way that the date field(Last Changed) gets updated with the date of the new row only on INSTANCE layer and not on the MASTER layer. So, if MDM ingests the delta record in Instance Layer and sees that there are no changes detected on any columns except the sequence column and it has nothing to update the Golden Record already existing then it should update the date field on Instance Layer only and not on Master Layer.Please let me know if you have any additional questions.Thank YouRitesh Ranjan

673
Cansu
Community Manager
A
Star Blazer L1
aish_TFStar Blazer L1
asked in Master Data Management & Reference Data Management 🛰️
Unable to start keycloak locally on a Azure DevBox

Unable to start keycloak locally on a Azure DevBox, When I run the keycloak.bat file from OneDesktop, it is immediately terminated as per console logs. Can anyone help with the diagnosis?

192
Cansu
Community Manager
D
Data Pioneer
debashishghoshData Pioneer
asked in Master Data Management & Reference Data Management 🛰️
Update flow for Delta Export

We are getting errors in the delta update flow for party and policy master and this is the error message. “com.ataccama.nme.core.NmeExecutionException: Plan "/opt/ataccama/one/mdm-server-config-13.9.1.221107-11165-6e37fd24/Files/engine/export/Party_Master_Delta.comp" execution failed: runtime.error.sql.instance.processError - Transaction (Process ID 901) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim.” Please advice if any help can be provided.Thanks !!

132
Cansu
Community Manager
F
Universe Traveller
fshahinUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
MDM Performance Issue

Hi All,We are facing performance issue while loading and exporting data in MDM. We have around 30 million records. We are loading data in chunks and initial few execution were very quick. After processing 20M we are observing severe degrade in performance. Now even for 10K records the job is taking 8-10 hours for both load and export. We have applied parallelism wherever its possible. Is there any other thing which we need to consider/implement which will enhance the performance?TIA.

302
Cansu
Community Manager
J
Universe Traveller
jbrantUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
Updating Lookup Item via GraphQL Mutation

I have an ETL process that’s run on-demand and refreshes master data referenced by several Ataccama catalog items. There’s about 60 lookup items that are created from catalog item attributes that I need to update immediately following the master data refresh. I know there’s a lookup item “Update” button in the Ataccama web interface and a method to schedule refreshes; however, I haven’t been able to find a way to update lookup items on-demand, such as through a GraphQL mutation.Is there an available mutation for updating lookup items on-demand, or another method that could be utilized?

331
OGordon100
Ataccamer
A
Star Blazer L1
aish_TFStar Blazer L1
asked in Master Data Management & Reference Data Management 🛰️
MDM Instance output to represent Ids from each system

I want to convert the instance export table values to a new table/view.The instance export contains 3 columns that I want to utiliseSharing an example-I have the following instance table structure :MASTER_ID SOURCE_SYSTEM SOURCE_ID 001 sys1 1234 001 sys2 12232 001 sys3 1292 001 sys4 13432 001 sys2 12334  5 records from 4 different source system make a master record, I want to group it in such a manner that there is a single row for each source_system id. If a source system participate twice(with two unique records) in formation of master record, I want two rows representing the different source_ids and a column with the source system that participates twice. Here is how I intend the output table to be: MASTER_ID SYS1_ID SYS2_ID SYS3_ID SYS4_ID MULTIPLE_REC 001 1234 12232 1292 13432 SYS2 001 1234 12234 1292 13432 SYS2  Can someone help me with the ONE Desktop step I can use for this?I have explored using Aggregating Column A

151
A
Star Blazer L1
A
Star Blazer L1
aish_TFStar Blazer L1
posted in Master Data Management & Reference Data Management 🛰️
Duplicate instance record IDs with different contact values

Taking reference of CDI Example, I have multiple contact values for the same person and I want all of these contacts to be a part of the contact master entity. I am not sure how the same instance record id will affect this. Will ataccama load both these records? Will both these records partivipate in match logic(same master_party_id, same_contact_type,same contact_value) or will it consider just the first one?Also, if someone can please confirm if the master_party_id is id for master record derived on processing of all party records?

5411
Cansu
Community Manager
A
Star Blazer L1
aish_TFStar Blazer L1
posted in Master Data Management & Reference Data Management 🛰️
source_id column value and matching step

Hi All,I have a few observations regarding the source_id column in matching step of an entity in a MDM project.For context, source_id is a column with unique values (Primary Key in most cases) that we define to uniquely identify each record in a source system, connected to mdm hub.I was recently trying to test the match logic on OneDesktop using plan files. I used a very small subset of records from each source.Load ComponentIn the project, source_id column(defined in load component) for one of these source system is “SOURCE_SYSTEM_NAME~^~pk_id”where SOURCE_SYSTEM_NAME is as string value and pk_id is the primary key. match componentWhen we defined a match component in mdm project, we do not define standalone bindings but assumably “Id Column” would be mapped to source_id column if we are testing on OneDesktop as a plan.So, when I created a plan to test the match logic, I mapped “Id Column” to source_id (with source_id having string values as mentioned above). But when I run the plan wi

212
A
Star Blazer L1
Cansu
Community Manager
CansuCommunity Manager
published in Master Data Management & Reference Data Management 🛰️
Efficient Bulk Profiling for Large Databases 💾

Hi everyone, In this post, we will cover how to do Bulk Profiling 💾Profiling vast databases with numerous tables can be a daunting task, but Ataccama's Bulk Profiling extension streamlines the process. This extension simplifies the profiling of database data sources with multiple tables. Instead of configuring analyses individually for each table, it enables you to import data structures directly from the database dictionary.Bulk Profiling then automatically generates a profiling plan for each table. To top it off, the tool creates an execution batch file that can run the individual plans. This means you can perform the entire profiling process unattended, whether overnight or at a scheduled time using a task scheduler. Step 1: Creating a Bulk Profiling ProjectIn the Model Explorer view, click on New Model Project in the toolbar. This will open the New Model Project dialog. In the dialog, select the Bulk Profiling model template. Click Finish.   This initiates the process of creating

1665
M
Data Voyager
J
Universe Traveller
Jason.SupticUniverse Traveller
published in Master Data Management & Reference Data Management 🛰️
Programming Languages

Howdy community!Today’s post is a very quick thought on some language knowledge that seems to help me when searching for answers about a project I might be working on in Ataccama. When I search online for an answer about an expression, or maybe an error I am seeing, I typically search in the following order. In this example we will assume I want to change a date to a string in a Column Assigner.I always start with SQL. So my search might be “PGSQL to turn a date into a string” Next I usually go to JAVA. Same search but with JAVA instead of PGSQL. Finally (and I usually reserve this one more for troubleshooting), I go to XML. While it might not be super useful for this example, there have been a couple times where searching for certain XML. One example is from another of my post on making a field nillable (able to accept a null value). The way I was helped by looking up XML functions, was that I was actually able to view the XML for the Enrichment I was creating and search it piece by p

763
Cansu
Community Manager
F
Universe Traveller
FionaFUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
Load data in ERP Systems

How to load result data from Ataccama ONE MDM to ERP systems like SAP?

392
Ales
Ataccamer

Badge winners

  • First idea 💡
    BarbaraBhas earned the badge First idea 💡
  • First idea 💡
    jbinghas earned the badge First idea 💡
  • First idea 💡
    samsTmobilehas earned the badge First idea 💡
  • First idea 💡
    ViLashas earned the badge First idea 💡
  • First idea 💡
    ScottAndersonhas earned the badge First idea 💡
Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings