Discuss, learn, ask, suggest, share all things MDM and RDM 🔍
Recently active
The Workflow task Run MDM Export does not recognize the export component and and gives error message.I am assuming it is because there is no path variable set for the Export folder.Can someone please help me identify the issue?
I have MDM setup and records are loaded using a Full Laod configuration. I need to create a load such that all the records that were added into the source are moved into to MDM hub on a daily basis. The approach being used is to identify all the records updated today and load them into MDM. Can you please suggest if I need to create a Delta Load for this or Full Load works. Also, I need to schedule a run of the load file on daily basis.How can I do it?
I have a workflow with a file iterator step. It runs without any issues in the IDE on my local machine, but whenever I deploy it my company’s RDM server, I get the following error:2025-01-07 09:05:31 INFO: starting...2025-01-07 09:05:31 ERROR: Problem during initialization of iterator valuejava.io.IOException: An unexpected network error occurred at java.io.WinNTFileSystem.canonicalize0(Native Method) at java.io.WinNTFileSystem.canonicalize(Unknown Source) at java.io.File.getCanonicalPath(Unknown Source) at com.ataccama.dqc.resources.local.FileResource.getCanonicalPath(FileResource.java:40) at com.ataccama.dqc.resources.impl.DelegatingResourceBase.getCanonicalPath(DelegatingResourceBase.java:41) at com.ataccama.adt.task.iterators.FileIterator$FileIteratorInstance.iterate(FileIterator.java:133) at com.ataccama.adt.task.iterators.TaskIteratorInstance.initAndIterate(TaskIteratorInstance.java:36) at com.ataccama.adt.task.exec.EwfForEachTaskInstance.run(EwfForEachTas
Hi everyone! I’m back with the second and last part of the GraphQL best practice series. First part was all about authentication types & which one you should use and how to authenticate GraphQL requests. This post will focus on how to work with MMM in GraphQL - so let’s get into it.If you’d like to check out the first part I’m sharing it here 👇 ONE Metadata Readers Or GraphQL? MMM is the name for the linked list that holds all metadata and the connections between them. Pretty much everything in MMM can be explored via GraphQL - as GraphQL is what happens behind the scenes in most steps/actions in ONE Desktop.As far as ONE Desktop is concerned, while ONE Metadata Readers are often all you need, highly nested queries and associated joins can become quite tricky to handle via ONE Desktop plans, and can sometimes be simpler to build by just running a specific GraphQL query. In addition to that, some things, such as raw profiling information can only be extracted via GraphQL. Exploring
I have a full load already configured for a source system. This load component has 3 different entities(party, contact, address).I have used this to load data into mdm-hub as well.However, now we want to use a different source table for party entity and want to load only the records for party entity. I am planning on using a new full load with mappings only for the party entity. Will thiscreate new instance party records and make the old records inactive? bring any change to the records for contact and address entities?
Security Assertion Markup Language (SAML) allows users to use one set of credentials to log into many different websites. It is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers(SP). SAML transactions use Extensible Markup Language (XML) for standardized communications between the identity provider and service providers. SAML is the link between the authentication of a user’s identity and the authorization to use a service. Items to consider prior to connecting to SAML- Best PracticeIt is important to understand how your organization’s Active Directory is setup. Some pre-requisites needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, with AD Groups we can map Ataccama Roles to a single group. This benefit of doing this is, your internal dev-ops team won’t have to manually add/remove u
Our Incremental load operation is taking huge time (2 hrs to process 5-7K records). We have applied all performance tuning things which is there in document. Apart from this we thought of loading data from DW to PostgreSQL (MDM DB) stage table and then will use that stage table as a source in Incremental load. Idea is when the server will be same for both source and target the computation will be faster.But this direct load job is failing with below error. Kindly help on fixing this. Or please recommend on any other approach.com.ataccama.nme.core.NmeExecutionException: Plan "C:\Users\EYKU4F\Ataccama\workspace\OmniChannelMDM_Latest\Files\engine\load\EPRISM_stg_eprism.comp" execution failed: runtime.error.jdbc_reader_error - nullcom.ataccama.dqc.model.environment.AbortedExecutionException at com.ataccama.dqc.model.elements.data.flow.RecordQueue.putBatch(RecordQueue.java:173) at com.ataccama.dqc.model.elements.data.flow.RecordQueue.access$100(RecordQueue.java:35) at com.ataccama.dqc.model
Hi community 👋 Every now and then, we hear from our users wondering if it’s possible to spice up their Master Data Management (MDM) interface with something a little more visual.Recently, I worked with one of our members who wanted to mark one of the records in a group as having a special meaning, so that it would instantly stand out to data stewards. While the logic behind selecting this record was complex, the outcome boiled down to a simple boolean flag, like cmo_preferred_flag = {true | false}. However, displaying such an attribute on the screen isn’t exactly thrilling:In earlier versions, it was possible to load an image file to represent things like this, but it required customization that was tricky to maintain. Plus, it often got wiped out when regenerating the MDM project. In our current versions (14.5.x / 15.x), we’ve moved to SpringBoot, and for security reasons, loading external static content is no longer supported.So, can we solve this in a fun and practical way?The emoj
Hi We are noticing behaviour in our model where a source system will split from its golden id group when updated on matching criteria fields ( in this example address). However it only does this when the group of records was manually merged onto the golden id. It wont “jump ship” and form a new golden id when the records previously auto merged based on the matched criteria we assigned in the model.Is there a way we can stop this splitting from happening when the source system is updated? For example we can see here the new golden id created with one SF record the previous group the SF record belonged to, shows a lot of manual merging in the Match Rule Name column Our rematch if changed section shows these fields are used for rematching: We want to keep rematching switched on, to bring other source systems into the group if they meet criteria, but we dont updates on the SF record itself to allow it to break out of the group. (We dont want “auto-splits”) thanks
While trying to manually merge 2 records, facing an error stating permission issue. Followed above mentioned step to merge records. Error message
We made changes into the clean operation of an entity. it will take effects for new transactions.How can we apply this to the current records that we already have into the database? We are exploring use a full reprocess of this entity:but we have some doubts about the setup. For example:What is the difference between a reprocess vs rematch checkbox? We tried to read the documentation but there is not detail explanation about it. Thanks in advance!
Hello, from one of our source systems we were previously excluding any inactive addresses, contact details, etc. Our users have requested we load these records into MDM for visibility. My issue is that while we can add these records as is, they will not show as inactive on MDM. We could add a source active flag but for consistency with other sources we would like them to show as inactive on MDM (Greyed out). We have considered doing a one time load of all the inactive instances and then excluding them from the daily load but because it is possible to create address and contact details as inactive from the source that would not work. I have explored using the delta load as it has a change type column that could be appropriate for our use case.Has anyone had to set records in MDM as inactive as part of the load not using the deletion strategy? What method did you use?I have also looked into creating new instance entities to store inactive records but that would be a large undertaking as
Hi @Cansu and AllIn the event if there is no change in the delta records coming in MDM except one of the Sequence Generated column, is their a way that the date field(Last Changed) gets updated with the date of the new row only on INSTANCE layer and not on the MASTER layer. So, if MDM ingests the delta record in Instance Layer and sees that there are no changes detected on any columns except the sequence column and it has nothing to update the Golden Record already existing then it should update the date field on Instance Layer only and not on Master Layer.Please let me know if you have any additional questions.Thank YouRitesh Ranjan
Unable to start keycloak locally on a Azure DevBox, When I run the keycloak.bat file from OneDesktop, it is immediately terminated as per console logs. Can anyone help with the diagnosis?
We are getting errors in the delta update flow for party and policy master and this is the error message. “com.ataccama.nme.core.NmeExecutionException: Plan "/opt/ataccama/one/mdm-server-config-13.9.1.221107-11165-6e37fd24/Files/engine/export/Party_Master_Delta.comp" execution failed: runtime.error.sql.instance.processError - Transaction (Process ID 901) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim.” Please advice if any help can be provided.Thanks !!
Hi All,We are facing performance issue while loading and exporting data in MDM. We have around 30 million records. We are loading data in chunks and initial few execution were very quick. After processing 20M we are observing severe degrade in performance. Now even for 10K records the job is taking 8-10 hours for both load and export. We have applied parallelism wherever its possible. Is there any other thing which we need to consider/implement which will enhance the performance?TIA.
I have an ETL process that’s run on-demand and refreshes master data referenced by several Ataccama catalog items. There’s about 60 lookup items that are created from catalog item attributes that I need to update immediately following the master data refresh. I know there’s a lookup item “Update” button in the Ataccama web interface and a method to schedule refreshes; however, I haven’t been able to find a way to update lookup items on-demand, such as through a GraphQL mutation.Is there an available mutation for updating lookup items on-demand, or another method that could be utilized?
I want to convert the instance export table values to a new table/view.The instance export contains 3 columns that I want to utiliseSharing an example-I have the following instance table structure :MASTER_ID SOURCE_SYSTEM SOURCE_ID 001 sys1 1234 001 sys2 12232 001 sys3 1292 001 sys4 13432 001 sys2 12334 5 records from 4 different source system make a master record, I want to group it in such a manner that there is a single row for each source_system id. If a source system participate twice(with two unique records) in formation of master record, I want two rows representing the different source_ids and a column with the source system that participates twice. Here is how I intend the output table to be: MASTER_ID SYS1_ID SYS2_ID SYS3_ID SYS4_ID MULTIPLE_REC 001 1234 12232 1292 13432 SYS2 001 1234 12234 1292 13432 SYS2 Can someone help me with the ONE Desktop step I can use for this?I have explored using Aggregating Column A
Taking reference of CDI Example, I have multiple contact values for the same person and I want all of these contacts to be a part of the contact master entity. I am not sure how the same instance record id will affect this. Will ataccama load both these records? Will both these records partivipate in match logic(same master_party_id, same_contact_type,same contact_value) or will it consider just the first one?Also, if someone can please confirm if the master_party_id is id for master record derived on processing of all party records?
Hi All,I have a few observations regarding the source_id column in matching step of an entity in a MDM project.For context, source_id is a column with unique values (Primary Key in most cases) that we define to uniquely identify each record in a source system, connected to mdm hub.I was recently trying to test the match logic on OneDesktop using plan files. I used a very small subset of records from each source.Load ComponentIn the project, source_id column(defined in load component) for one of these source system is “SOURCE_SYSTEM_NAME~^~pk_id”where SOURCE_SYSTEM_NAME is as string value and pk_id is the primary key. match componentWhen we defined a match component in mdm project, we do not define standalone bindings but assumably “Id Column” would be mapped to source_id column if we are testing on OneDesktop as a plan.So, when I created a plan to test the match logic, I mapped “Id Column” to source_id (with source_id having string values as mentioned above). But when I run the plan wi
Hi everyone, In this post, we will cover how to do Bulk Profiling 💾Profiling vast databases with numerous tables can be a daunting task, but Ataccama's Bulk Profiling extension streamlines the process. This extension simplifies the profiling of database data sources with multiple tables. Instead of configuring analyses individually for each table, it enables you to import data structures directly from the database dictionary.Bulk Profiling then automatically generates a profiling plan for each table. To top it off, the tool creates an execution batch file that can run the individual plans. This means you can perform the entire profiling process unattended, whether overnight or at a scheduled time using a task scheduler. Step 1: Creating a Bulk Profiling ProjectIn the Model Explorer view, click on New Model Project in the toolbar. This will open the New Model Project dialog. In the dialog, select the Bulk Profiling model template. Click Finish. This initiates the process of creating
Howdy community!Today’s post is a very quick thought on some language knowledge that seems to help me when searching for answers about a project I might be working on in Ataccama. When I search online for an answer about an expression, or maybe an error I am seeing, I typically search in the following order. In this example we will assume I want to change a date to a string in a Column Assigner.I always start with SQL. So my search might be “PGSQL to turn a date into a string” Next I usually go to JAVA. Same search but with JAVA instead of PGSQL. Finally (and I usually reserve this one more for troubleshooting), I go to XML. While it might not be super useful for this example, there have been a couple times where searching for certain XML. One example is from another of my post on making a field nillable (able to accept a null value). The way I was helped by looking up XML functions, was that I was actually able to view the XML for the Enrichment I was creating and search it piece by p
