Discuss, learn, ask, suggest, share all things MDM and RDM π
Recently active
While trying to manuallyΒ merge 2 records, facing an error stating permission issue.Β Followed above mentioned step to merge records.Β Error messageΒ
Hello, from one of our source systems we were previously excluding any inactive addresses, contact details, etc. Our users have requested we load these records into MDM for visibility. My issue is that while we can add these records as is, they will not show as inactive on MDM. We could add aΒ source active flag but for consistency with other sources we would like them to show as inactive on MDM (Greyed out).Β We have considered doing a one time load of all the inactive instances and then excluding them from the daily load but because it is possible to create address and contact details as inactive from the source that would not work.Β I have explored using the delta load as it has a change type column that could be appropriate for our use case.Has anyone had to set records in MDM as inactive as part of the load not using the deletion strategy? What method did you use?I have also looked into creating new instance entities to store inactive records but that would be a large undertaking as
HiΒ @CansuΒ and AllIn the event if there is no change in the delta records coming in MDM except one of the Sequence Generated column, is their a way that the date field(Last Changed)Β gets updated with the date of the new row only on INSTANCE layer and not on the MASTER layer. So, if MDM ingests the delta record in Instance Layer and sees that there are noΒ changes detected on any columns except the sequence column and it has nothing to update the Golden Record already existing then it should update theΒ date field on Instance Layer only and not on Master Layer.Please let me know if you have any additional questions.Thank YouRitesh Ranjan
Unable to start keycloak locally on a Azure DevBox, When I run the keycloak.bat file from OneDesktop, it is immediately terminated as per console logs.Β Can anyone help with the diagnosis?
We are getting errors in the delta update flow for party and policy master and this is the error message.Β βcom.ataccama.nme.core.NmeExecutionException: Plan "/opt/ataccama/one/mdm-server-config-13.9.1.221107-11165-6e37fd24/Files/engine/export/Party_Master_Delta.comp" execution failed: runtime.error.sql.instance.processError - Transaction (Process ID 901) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim.βΒ Please advice if any help can be provided.Thanks !!
Hi All,We are facing performance issue while loading and exporting data in MDM. We have around 30 million records. We are loading data in chunks and initial few execution were very quick. After processing 20M we are observingΒ severe degrade in performance. Now even for 10K records the job is taking 8-10 hours for both load and export. We have applied parallelism wherever its possible. Is there any other thing which we need to consider/implement which will enhance the performance?TIA.
I have an ETL process thatβs run on-demandΒ and refreshesΒ master data referenced by several Ataccama catalog items. Thereβs about 60 lookup items that are created from catalog item attributes that I need to update immediately following the master data refresh. I know thereβs a lookup item βUpdateβ button in the Ataccama web interface and a method to schedule refreshes; however, I havenβt been able to find a way to update lookup items on-demand, such as through a GraphQL mutation.Is there an available mutation for updating lookup items on-demand, or another method that could be utilized?
Taking reference of CDI Example, I have multiple contact values for the same person and I want all of these contacts to be a part of the contact master entity.Β I am not sure how the same instance record id will affect this. Will ataccama load both these records? Will both these records partivipate in match logic(same master_party_id, same_contact_type,same contact_value) or will it consider just the first one?Also, if someone can please confirm if the master_party_id is id for master record derived on processing of all party records?
Hi All,I have a few observations regarding the source_id column in matching step of an entity in a MDM project.For context, source_id is a column with unique values (Primary Key in most cases) that we define to uniquely identify each record in a source system, connected to mdm hub.I was recently trying to test the match logic on OneDesktop using plan files. IΒ used a very small subset of records from each source.Load ComponentIn the project, source_id column(defined in load component) for one of these source system is βSOURCE_SYSTEM_NAME~^~pk_idβwhereΒ SOURCE_SYSTEM_NAME is as string value and pk_id is the primary key.Β match componentWhen we defined a match componentΒ in mdm project, we do not define standalone bindings but assumably βId Columnβ would be mapped to source_id column if we are testing on OneDesktop as a plan.So, when I created a plan to test the match logic, I mapped βId Columnβ to source_id (with source_id having string values as mentioned above). ButΒ when I run the plan wi
Hi everyone,Β In this post, we will cover how to do Bulk Profiling πΎProfiling vast databases with numerous tables can be a daunting task, but Ataccama's Bulk Profiling extension streamlines the process. This extension simplifies the profiling of database data sources with multiple tables. Instead of configuring analyses individually for each table, it enables you to import data structures directly from the database dictionary.Bulk Profiling then automatically generates a profiling plan for each table. To top it off, the tool creates an execution batch file that can run the individual plans. This means you can perform the entire profiling process unattended, whether overnight or at a scheduled time using a task scheduler.Β Step 1: Creating a Bulk Profiling ProjectIn the Model Explorer view, click on New Model Project in the toolbar. This will open the New Model Project dialog. In the dialog, select the Bulk Profiling model template. Click Finish. Β This initiates the process of creating
Howdy community!Todayβs post is a very quick thought on some language knowledge that seems to help me when searching for answers about a project I might be working on in Ataccama. When I search online for an answer about an expression, or maybe an error I am seeing, I typically search in the following order. In this example we will assume I want to change a date to a string in a Column Assigner.I always start with SQL. So my search might be βPGSQL to turn a date into a stringβ Next I usually go to JAVA. Same search but with JAVA instead of PGSQL. Finally (and I usually reserve this one more for troubleshooting), I go to XML. While it might not be super useful for this example, there have been a couple times where searching for certain XML. One example is from another of my post on making a field nillable (able to accept a null value). The way I was helped by looking up XML functions, was that I was actually able to view the XML for the Enrichment I was creating and search it piece by p
How to load result data fromΒ Ataccama ONE MDM toΒ ERP systems like SAP?
I have a scenario where I do not want to deactivateΒ the source party records in Ataccama, even thoughΒ it no longer exists in the source. Is there a way to achieve this?
Hello,I am trying to use MPP step to derive some values from some free text field.I am struggling to parse any regex pattern for string like βA/Bβ.It seems to work but whenever there is / sign it just doesnβt parse it.I tried to escape it using \/ and also some other different options but nothing seems to work.Β What am I doing wrong?EDIT:Tried to parse similar REGEX against Regex Matching step and here it seems to work:Β Could it be possible that MPP step is parsing incorrectly?EDIT 2:I might have found the issue. I took Tokenizer part from our another component and it started working correctly. However, I still donβt understand exactly how Tokenizer works and the docs arenβt providing with great examples on how to understand it.Β Szymek
Please help/guide me on how to create parameters to enter values for a workflow in admin centre. I have an already configured workflow with parameters in Admin centre but I want to remove it. I am not aware of this change as it was made previously by another person. If there is some documentation in ataccama docs that someone could point me towards, that would also be great.TIAΒ
Can we have duplicate source_id column values in mdm full load?
I created twoΒ Complex Delta Load file and this led to addition of relatedΒ details inΒ .metadata/md/coβnnected_systemsβBut these two filesΒ donβt show up in the Admin Centre. Please tell me what can be done?
I have created a workflow to call a component which is selecting data based on a value provided by user in the admin center workflow input parameter.Β To make it error free so that the user doesnβt input any wrong value for the parameter, I was wondering ifΒ it is possible to create a dropdown or bulletΒ with accepted parameter values in the admin center so we could just choose the value and run the workflow?Please let me know if anyone has tried this.Thank you,
Hi All,How can I use the file us_ca_firstname_synonym.lkp to create a match? Like Robert Smith should match with Bob Smith. I checked the file and for one name it has multiple synonyms. I can fetch the result in clean component but not sure how do I use that in Match component to generate match. Your help is much appreciated. Thanks in advance.ThanksFS
Reasons for deletionThere are several reasons why you might want to delete data from your MDM.Legal reasons e.g. GDPR Incorrect load operation happened The data is simply not relevant anymore Performance issues while keeping redundant data etc.Truncate tables?People usually ask if a TRUNCATE operation worked. The simple answer is NO. It might work only in isolated edge cases. You typically need to consider MANY other things and at least some of them need to be resolved.uncommitted transactions related tables/records x... tables historical table(s) direct deletion will not be properly removed from the history and must be removed separately master table(s) matching repository and matching proposals repository (if matching is enabled) think about copyColumns - are there any data copied from that specific entity to a related one export operations (truncated data will not be exported) event handlers (truncated data will not be published) auditing direct deletion will not be logged and will
If my MDM instance isΒ up and running with the model same as the example project. But there were standardization changes that need to be implemented now. Example:Β ph no. (+91)22343 24532 is being standardised to +91234324532.Will this affect matching for contact.master records? If so, will it create new master ids in any scenario?Β
Could you please help to share steps to setup connection from ataccama to Snowflake
Is there a way that I can use to make an API call to trigger a workflow?We want to start a workflow to Load Data into MDM after the data is refreshed in all source systems, each time it is refreshed.Please suggest if there is a way do define a service for this.Thanks in Advance :)
Hi All,I have recently made created new export delta operation along with made minor tweaks in the full load export operation too.After making changes,once i save and generate,trying to up the MDM server,getting the below erro.Please suggest,what is could be the missing dots here.Β Sharing the error log received in console for start_mdm_Server.bat file.2024-09-27 12:23:30,456 β[1;31mERRORβ[0;39m [][][main] β[36mo.s.b.SpringApplication:835β[0;39m - Application run failedorg.springframework.beans.factory.BeanCreationException: Error creating bean with name 'workflowServerComponentService': Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'nmeEngine' defined in class path resource [com/ataccama/nme/services/config/NmeBeansConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.ataccama.nm
I would like to know what will be the effects in mdm web application if I make the following changes.Here goes the scenario:Letβs say I am using email attribute in my match logic of contact(silver) entityΒ and have already loaded the email records into my mdm-hub.But now, I do not want to use email information from 1 of 3 source systems while determining the contact(silver) master. I think the option is to not load email information from the said system.WillΒ the email information (from the records that were loaded previously from system 1 )still show up after I run a reprocess job( after loading the records from system 1 without any email information.)?
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.