Master Data Management & Reference Data Management 🛰️

Discuss, learn, ask, suggest, share all things MDM and RDM 🔍

144 Topics
239 Replies

144 Topics

asked in Master Data Management & Reference Data Management 🛰️

Monitoring Project - Integer and Dates

When I use Monitoring Projects and try and apply a rule to a DATE or INTEGER it is not allowed. For Date Rule, I want to check for NULL or a DATE range. For Integer, I want to check from a referenced lookup table. Why is this check only allowed for Strings ? Will DATE and INTEGER be added at a later time ? Rene Requenes

6 days ago

OjaswiniRocket Pioneer II

asked in Master Data Management & Reference Data Management 🛰️

Too many components / steps in a single onPublishHandler.plan file.

Hello Ataccamer,I have a question regarding the "onPublishHandler.plan" file. This file contains components and steps for on-publish synchronization for all the RDM tables, which results in numerous components corresponding to each RDM table. As the project progresses, the number of RDM tables keeps increasing, causing this file to continuously grow in complexity.I'd like to inquire whether it's possible to segregate these components based on projects example diff file for domain or any other way. Is there a way to create separate "onPublishHandler.plan" files for different project domains? This could help reduce confusion, especially when multiple developers are working simultaneously. Just a thought. Thanks and regards,Ojaswini

13 days ago

OjaswiniRocket Pioneer II

asked in Master Data Management & Reference Data Management 🛰️

How to append data into Parquet file in RDM on-publish event.

Hello,I have a requirement to append the records in parquet file in the event of RDM on-publish synchronization. Does anyone know, how can we append data into Parquet file in RDM on-publish event?I believe parquet file writer will write data in the file, but if the new record created in the RDM, will it append the data in the same file or will replace the file completely and create new one.Thanks for any help.-Ojaswini

14 days ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

Permission/Security setup via the OneDesktop IDE (RDM)

Setting permissions for groups via the IDE.Ensure your groups exists in KeyCloak first In the IDE go to your RDM Project and complete the following under "Security" Right click on Roles Select New role Add the name and description of the role (be sure to use the same names that you setup in KeyCloak) then click Apply and OK Expand the new role, and right click on Entities then select Edit… Add the Tables and the Table level permissions in the next window NOTE: Table permissions do not give column level permissions. Double click the number next to the table that you wish to setup permissions for Notice that there are options for "View for All Columns" and "Modify for All Columns." However, if you want to select only certain columns, you can do that below with the following steps. Add the column names you want permissions to be given for, and select View and/or Modify for the given column When complete, click Apply then OKTo ensure these permissions are applied to the Web Admin do

14 days ago

OjaswiniRocket Pioneer II

asked in Master Data Management & Reference Data Management 🛰️

How to write data to Parquet file when it is a record deletion event on RDM UI. (in on-publish)

Hello Ataccamer,I am dealing with Parquet file writing in on-publish mode. Could you please suggest how to write data to Parquet file when it is a record deletion event on RDM UI (in on-publish) . As in case of Parquet file it is always first delete the file and then created the file again.How the CHANGE_TYPE is in {'DELETED'} will work in this case. Any suggestions, please share.Thanks and regards,Ojaswini

19 days ago

GersonSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Property with name 'KafkaResources' doesn't exist in com.ataccama.dqc.streaming.KafkaProviderComponent!

Hello Ataccama Community,I'm currently using Ataccama One Desktop version 13.9.1, and I'm encountering an error while trying to execute a plan that involves a Kafka Reader component. I have configured a Kafka server(kafka integration) with the following properties:ssl.truststore.location=LOCATION_TO_TRUSTSTOREssl.truststore.password=TRUSTSTORE_PASSWORDssl.protocol=TLSsecurity.protocol=SASL_SSLsasl.mechanism=GSSAPIsasl.kerberos.service.name=kafkasasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \ useKeyTab=true \ storeKey=true \ keyTab="LOCATION_OF_THE_KEYTAB" \ principal="USERNAME_HERE@EUROPE.XYZXYZ.CORP"; The error message I'm seeing is: "Property with name 'KafkaResources' doesn't exist in com.ataccama.dqc.streaming.KafkaProviderComponent!"I've reviewed my configuration, but I'm still unable to resolve this issue. Can someone please provide guidance on how to troubleshoot and resolve this error? Any insights or suggestions would be gre

27 days ago

vcostaSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Ways to get related entities in transformation proccess

Hi, I have below use case:A party can have multiple phones related to it, when any of them change, we need to update a flag on every related phones (same party_id) I am having problems to get those related phones during the merge process, what would be the better way to do it? I have tried it using Mdc Read Step, but I'm not able to start the server due below errorjava.lang.IllegalStateException: Cannot find INmeEngine: Failed to create step 'Mdc Read' (com.ataccama.nme.dqc.steps.MdcRead) (Cannot find INmeEngine) at com.ataccama.nme.internal.dqc.PlanExecutor.createRunner(PlanExecutor.java:137) at com.ataccama.nme.internal.dqc.PlanExecutor.<init>(PlanExecutor.java:83) at com.ataccama.nme.internal.dqc.PlanExecutor.<init>(PlanExecutor.java:69) at com.ataccama.nme.internal.dqc.pool.PlanExecutorCreator.createResource(PlanExecutorCreator.java:34) at com.ataccama.nme.internal.dqc.pool.PlanExecutorCreator.createResource(PlanExecutorCreator.java:17) at com.ataccama.dqc.commons.reso

27 days ago

Marnix WisselaarRocket Pioneer II

posted in Master Data Management & Reference Data Management 🛰️

Iterating rows of a table (Record-by-record - SQL Iterator) in a workflow

At Stater we needed to iterate a table in a database and perform some action on it. This needed to be orchestrated from a workflow. In our case trigger a set of MP's and write the output to a DQ Data warehouse.So I created a ‘pattern’ which we use for our cookbook. With thanks to Brian and Matej! Now if somebody can tell me how to add a PDF and ZIP with the sample, I'd be happy ;)

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

Greyed out fields appear to change with Enrichment... but they do not (thankfully)!

Howdy Ataccama Community:If you are needing to know how to grey out fields for edit, please navigate here: Grey out fields for edit | Community (ataccama.com)Problem: After greying out fields, I noticed that fields an Enrichment acts on appeared to change in the User Interface, even if greyed out.Solution: Working with our Ataccama professional support person, it was found that although the fields change in the UI, when saved the greyed out fields are not updated in the record being edited. This is great news, as the purpose (at least for our team) of greying out a field, is so that it cannot be changed once created.So that’s it, probably the shortest post I will put on here, but an important one for record reliability.Thank you,Jason R.S.

1 month ago

alexAguilarMxRocket Pioneer II

asked in Master Data Management & Reference Data Management 🛰️

Multi-load operation :: Why data acquisition is slow?

Hi Every one, Why the data acquisition in a multi-load operation could be so slow?Checking the database, the query to get the data is taking less than 10 minutes. Do you have any guidance about it?

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

Errors in LKP file with multiple columns

Howdy Ataccama community:Today I am creating an LKP file with multiple columns for the first time, and came across a couple rookie mistakes I was making, and wanted to share the warnings and errors I am getting.The file I am making is a simple mapping file going from the key to a field “PH1”Rookie mistake number 1: The file being used to make the lkp is a csv, and I made the assumption the Text File Reader in the lookup builder plan would recognize this from the file type. It did not, it had the separation of columns as \t (tab) still. This created the warning [EHS:SHORT_LINE][step:Text File Reader][line:1] with a row for ever row in my file. To fix this one was simply setting the “Line Separator” field in the Text File Reader to a comma instead of \t.Rookie mistake number 2: After correcting my first mistake, I was now getting the following error “There were logged 1 runtime warnings and 1 runtime errors.” With the details being “Error sending response” with no info on why. I got frus

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

Grey out fields for edit

Howdy Ataccama Community!Have you ever wanted to have a user see a field while in edit mode, but want to have it greyed out for editing? This problem arose while we were working with a Primary Key that needed to be seen in edit mode but never edited. The only way for a field to be seen in edit mode, is to set it to Modify in the permissions, but we did not want the user to modify. Problem: Show all fields in Edit mode, but only have certain ones editable.Solution: Begin by deploying your project and populating any tables with initial loads as needed. Next go back to OneDesktop IDE, and open the table you are wanting to make some fields non-editable. For the fields you want to make non-editable select Generated. While this sounds like it will auto generate the field, I assure you it does not.Now generate and deploy your project again (remember to generate at the App Configuration level).Note: You might need to stop and restart services for the change to take effect. While I was testing

1 month ago

Mayank VermaSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Looking for a training on Ataccama MDM tool

Hi All,I want to learn the implementation of ataccama MDM. Is there any learning module or instructor who can provide training is available in India.If yes, please do let me know. Thanks,Mayank Verma

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

Screen flashing when user tries to log in

Howdy all:Earlier this week we started having an issue when end users were trying to log in to our DEV environment. When they would navigate to the URL, their browser would show Ataccama splash screen, then instantly (and continuously) refresh. When asked to try to ping the URL via the command line, the correct IP was being hit, and there was a low latency reply from the server.Note: This issue occurred using Azure AD.Problem: User navigates to a known good URL for Ataccama RDM Web Application. When the application is reached, Ataccama splash screen shows in browser, then browser starts refreshing every 2 to 3 seconds.Solution: It was found that someone accidentally change the setting in KeyCloak called “First Login Flow.” This should have been set to “First Broker Login,” but was switched to “Browser.” Switching back to “First Broker Login” fixed the issue for our users.I am unsure if this is specific to Azure AD, but it is worth a try, if you are seeing a similar issue.Thank you,Jaso

1 month ago

qingyuSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Supported Data Sources

Hi Team, I want to know if ataccama support the connection to Apache Druid and TigerGraph? If yes, is there any Docs for that?

1 month ago

Jason.SupticSpace Explorer II

published in Master Data Management & Reference Data Management 🛰️

Programming Languages

Howdy community!Today’s post is a very quick thought on some language knowledge that seems to help me when searching for answers about a project I might be working on in Ataccama. When I search online for an answer about an expression, or maybe an error I am seeing, I typically search in the following order. In this example we will assume I want to change a date to a string in a Column Assigner.I always start with SQL. So my search might be “PGSQL to turn a date into a string” Next I usually go to JAVA. Same search but with JAVA instead of PGSQL. Finally (and I usually reserve this one more for troubleshooting), I go to XML. While it might not be super useful for this example, there have been a couple times where searching for certain XML. One example is from another of my post on making a field nillable (able to accept a null value). The way I was helped by looking up XML functions, was that I was actually able to view the XML for the Enrichment I was creating and search it piece by p

121

1 month ago

BhumiStarblazer

asked in Master Data Management & Reference Data Management 🛰️

DQI SK stage sample demo

Is there a doc/demo available on how to use DQI SK stage (Analyze category) in Thick client please?

1 month ago

mahesh ArRocket Pioneer

asked in Master Data Management & Reference Data Management 🛰️

parse multi arrays response value from JSON call

I have to parse the below JSON GET response to get “Headers” and "Name” as columns in Json Call step with Data Streams, How can I perform this? [ { "Headers": [ “Id”, “Name”, “Email” ], "ID": “test_111”, "Rows": [ [ "100”, “John”, “John@gmail.com” ]}]

1 month ago

mahesh ArRocket Pioneer

asked in Master Data Management & Reference Data Management 🛰️

How can we integrate python script with Ataccama plan?

My requirement is that I want to read a file in Atacama plan which is an output of a python script .I would like to integrate python script then plan to run sequentially.

1 month ago

sandipilagSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Login issue in ataccama web URL (http://localhost:8050/)

Hi Team,When trying to login in ataccama web URL, receiving below error:Could you please help me here?As all compeonts like DB, leyloack, MDM server, MDM are up and running successfuly.-----------------------------------------------------------Error in MDM server ------------------------------------------------------------------WARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by retrofit2.Platform (file:/D:/Ataccama/mdm/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)WARNING: Please consider reporting this to the maintainers of retrofit2.PlatformWARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future release2023-08-07 15:26:23,797 ←[34mINFO ←[0;39m [][][restartedMain] ←[36mn.r.s.b.l.a.LogbackAccessContext:149←[0;39m - Configured the Logback-access: context=[default], config=[classpat

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

How do I view Enrichment Online XML

Howdy all!I said in another post today that it was going to be short. It turned a little longer than I expected. This one might be a little shorter.Question: How do I find the XML that makes up my “.online” files for Enrichments?Steps:Start all your services, and ensure you have deployed your Configuration in the Admin consol (localhost:8060/admin) Go to localhost:8061 Under Server, select Applications Click on the Application “.online” file you are wanting to see the XML for Click the service you want to view the XML for in the next screen Then select the link for the XML SchemaThe data in here can help to find errors in the setup of the service! Enjoy!Jason R.S.

1 month ago

sandipilagSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

Ataccama login issue

Hi All,Receving below error when trying to login in ataccama URL: Need help on this? WARNING: An illegal reflective access operation has occurredWARNING: Illegal reflective access by retrofit2.Platform (file:/D:/Ataccama/mdm/lib/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)WARNING: Please consider reporting this to the maintainers of retrofit2.PlatformWARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operationsWARNING: All illegal access operations will be denied in a future release2023-08-07 15:26:23,797 ←[34mINFO ←[0;39m [][][restartedMain] ←[36mn.r.s.b.l.a.LogbackAccessContext:149←[0;39m - Configured the Logback-access: context=[default], config=[classpath:logback-access.xml]2023-08-07 15:26:25,274 ←[34mINFO ←[0;39m [][][restartedMain] ←[36mc.a.l.s.s.e.KeyProvidersSpringBootAutoConfiguration:41←[0;39m - messageId=createDefaultKeyProvider encryption=INTERNAL, reason=the 'internal.encryption.keyst

1 month ago

sandipilagSpace Explorer

asked in Master Data Management & Reference Data Management 🛰️

.metadata folder not visible

Hi All,Require quick help on ataccama IDE, I am not able to see .metadata folder in IDE. What is issue here?Please find attached screen shot ?Please help me here?

1 month ago

Jason.SupticSpace Explorer II

posted in Master Data Management & Reference Data Management 🛰️

EXTRA_DATA error or warning

The EXTRA_DATA error/warning was a thorn in my side for the last couple of days. I added a column to a dataset I was loading to a model from a Text File Reader, and I could not figure out for the life of me what I was doing different from before the column add. It turned out, where I was trying to unselect data was too early.Problem: Suppose you are bringing in a delimited file to use in a Text File Reader that has an extra field at the beginning, which you don't want to have in your output. Finding the correct place to disable this data can difficult. If you disable it before your Text File Reader, or simply don't bring it into your Text File Reader, then you run the risk of getting an error or warning that states EXTRA_DATA for every row you try to bring in, as seen below.I have seen the EXTRA_DATA error in 3 scenariosDeselecting the “Use” box in the Metadata definition on columns you don’t want. Selecting all columns in the Metadata definition, but not bringing the column into your

2 months ago

bazSpace Explorer II

asked in Master Data Management & Reference Data Management 🛰️

Displaying a list in a drop-down

Hi, How do I present a selection of values in a drop-down in the RDM create/edit screens.E.g. I have an entity called Door with a column called state that can have a value of either Open or Closed. Can this be presented as a drop down to a user?baz

2 months ago

Page 1 / 6

Master Data Management & Reference Data Management 🛰️

144 Topics

Monitoring Project - Integer and Dates

Too many components / steps in a single onPublishHandler.plan file.

How to append data into Parquet file in RDM on-publish event.

Permission/Security setup via the OneDesktop IDE (RDM)

How to write data to Parquet file when it is a record deletion event on RDM UI. (in on-publish)

Property with name 'KafkaResources' doesn't exist in com.ataccama.dqc.streaming.KafkaProviderComponent!

Ways to get related entities in transformation proccess

Iterating rows of a table (Record-by-record - SQL Iterator) in a workflow

Greyed out fields appear to change with Enrichment... but they do not (thankfully)!

Multi-load operation :: Why data acquisition is slow?

Errors in LKP file with multiple columns

Grey out fields for edit

Looking for a training on Ataccama MDM tool

Screen flashing when user tries to log in

Supported Data Sources

Programming Languages

DQI SK stage sample demo

parse multi arrays response value from JSON call

How can we integrate python script with Ataccama plan?

Login issue in ataccama web URL (http://localhost:8050/)

How do I view Enrichment Online XML

Ataccama login issue

.metadata folder not visible

EXTRA_DATA error or warning

Displaying a list in a drop-down

Badge winners

Let's shape the future of Ataccama together

Master Data Management & Reference Data Management 🛰️

Badge winners

Let's shape the future of Ataccama together

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded