Learn & Share

Hi everyone,I currently have a plan that is creating a main task and populating it with sub tasks that each link to individual tables. My issue at the moment is that sometimes this creates tasks with a huge amount of sub tasks that may seem daunting for people to try to work through.Currently the tables and their attribute ids are all stored in one excel sheet, so I was wondering if it would be possible to break it into separate excel files each containing 20 rows. If this is possible I would then be able to create additional tasks rather than just grouping everything in one.Thank you for any help!

Hi all! I am working on a monitoring project. I have a catalog item consisting 130 columns. I want to create a custom catalog item selecting only few attributes that I need to have them in my monitoring project.However, based on my previous experience: Once I have created a catalog item with selected attributes. Then I had to add new attributes according to my project needs. I have run the monitoring project again with new attributes of catalog item, then I lost my monitoring project. It showed errors for all DQ rules because as I understood from error message, DQ rules need to have attribute key ids, and with adding new attributes I have changed attribute key ids. Now, again I have a huge catalog item and I want to reduce the volume with selecting few attributes but I don’t want to face with the same problem again. If you have any suggestion it would be so great for me. Thanks !

Hello all, i have datalake gen2 parquet files in containers and folders. In the folder are delta.log file and parquets. Is there some way to merge these parquets to one table? This table should represent one current(up to date) table contains data from parquets. this is what i see now in ataccama. I want to merge these data to one table. Thanks for replies, Matus

I want to create another column using the filter step on the IDE. How do I concatenate foreign characters and fields with different data types into one field.Fields to concatenate is date field, special character (/), string and integer fields

Hi We are planning to use Atacamma DQ as a service and would like to use its DQ Engine to query and provide us counts. We would like to build a API call to Atacamma DQ with necessary SQL Query and connection parameters as part of API call, and would like to know if this SQL query can be executed on the Atacamma DQ and provide the results as an output back to API as a response. Please let us know the possibilities. Regards,Uma

Hello, is it possible to export a catalog item along with its associated terms?  As you can see in the screenshot below, I set the attribute called “last_update_hostname”  as confidential temporarily for an example, but when I export it to a file, I was only able to see the data values not the terms associated with it.   

Hello, how to run the plan below (created using ONE Desktop) on Ataccama Web ?  

Hi,I have a .csv file containing country codes that looks like:countrycode.csvI want to use this file as a lookup. In ONE Web it is possible to use a .lkp file as a lookup (why not just import an excel file there is beyond me). In order to do this I understoond I had to create a lookup builder:Lookup builder​​In the Text File Reader I selected the documented with the encoding (I hope this is the right one?). As for the columns I created nm_lnd and kd_lnd (both string).Text File Reader - GeneralText File Reader - ColumnsAs for the Lookup Builder:Lookup BuilderWhen I run it a .lkp is created but also a rejected.txt with everything that is inside the .csv meaning I'm not able to create the desired .lkp file. What can I do to resolve this? Is there also documentation on how to create lookup using .csv somewhere? I can't find it.

The email notifications starts with “Dear Collegue”… and i was wondering - is there a mail merge kind of thing - that can be done to replace the generic addressing with speicific name of the individual it is being emailed to? And - if it is a role - may be we need to get into role membership to expand into names. Has anyone done that? 

Hi DQC online service experts,I have built an online service that writes data into RDM using Rdm importer. I put some validation logic into the component so that it would reject bad records from being written (e.g. value supplied does not exist in parent table). The rdm importer is going straight to confirm mode as we need the data to be automatically published and available. I've configured the integration output to provide the primary key and my explanation in the case that the component rejects the record. The app that is sending the data, are now saying that even if I provide an error explanation, the endpoint still provides a 200 response code. They have some in built error handling capability that they want to use, but they currently can't use it because the server responds 200. Is it possible for me to set a different response code e.g. 400, based on the data that is provided in the integration output?

I need to understand the difference between Full and Complex Delta Load in MDM.As per this document on Complex Delta Load,  Delta Dependency Mode  needs to be setup for each entity. I see there are two options for it -Autonomous and Central. Can someone please explain the difference between these and in what scenario these should be used?I also see that The engine expects an additional column in the input interface denoting the type of record change (change_type: I - insert, U - update, D - delete). Where shall we define these and what how can we identify which records needs which change_type?TIA 😊

Hello Community! I am looking to make a bit of an adjustment to my MMM in DQG version 14.5. Specifically, I would like to add a field for AD groups, so that they are validated at an entity level. I know I can write in AD groups and add them to the entity description (see image below), but I am also looking to enforce that idea. The entity summary includes a field to list an AD group, but this does not have any sort of legitimate ties to the actual external group.Any insight on how others have approached this topic or otherwise accomplished my goal is greatly appreciated. Thank you for your time.  Willoh

Hello! I am interested in hiding the global settings from users, at least some of them. I know this question has been asked before for version 13, but it was mentioned the approach is quite different in 14.5. Would appreciate any insight you all have. 

Hi everyone!Im not sure if this is possible, but I have not been able to find a query to find who the stewards are of a particular catalogItem. Specifically I am interested in getting the data owner and then using this in another part of my project.I do have the Attribute Id of the catalog item and my atacamma playground is working I just cant seem to find specifically what query I need the make.Thank you for any help!

Hi Team,we are just trying to convert the string timestamp format values to dateTime format in the expression transformation, when we tried to use  toDateTime(<column_name>,'YYYY-MM-DD HH:mm:ss','en') returning null- can someone help.

Hi Team,I am reading from collibra using collibra reader, some data has html tags like - <p> , <div> in it. Is there a way where we can filter out the html tags if its present in the data. like strip html tags like that. Let me know if more information is needed.Thanks.

Hi team,I am working on a story where a user needs to see the results of the monitoring project which is running.Like where the results are getting saved in the backend,in the tables.Need to create a pipeline for that like when we click the run monitoring button in Ataccama Web,a workflow in Ataccama desktop should run and the results should be saved in the database to look at to see how many records are passing and failing.So how can we create that pipeline?

Hello all, i have datalake gen2 parquet files in containers and folders. In the folder are delta.log file and parquets. Is there some way to merge these parquets to one table? This table should represent one current(up to date) table contains data from parquets. this is what i see now in ataccama. I want to merge these data to one table. Thanks for replies, Matus

Hi Team,I am trying to find how we can achieve completeness DQ control by comparing data in the file and the table irrespective of the position of primary key in either of them. Also, need to know how we can achieve the same by using the composite keys.

Hello,I hoped somebody has already worked this out so I don’t have to 🤣. I have a Epoch Unix Timestamp value that I want to convert into normal DATETIME format in Ataccama.e.g. 1711015200 I was expecting converted value to be 2024-03-21 10:00:00 (GMT).But when I do a toDateTime() on it, it gives me 1970-01-20 20:16:55. So I guessed wrong.I thought about using dateadd() but that one does not support seconds. Any ideas?

Hi We would like to publish the profiling results of a scan on to Azure service so that we can pick up those results and update it back to another reporting systems via Kafka or Azure Pubsub.What we are mainly interested is in how to achieve sending publish results to Azure post scanning in runtime without impacting the performance on PostgreSQL DB where profiling results are getting logged. Did any one encounter similar scenario and how did you achieve it.Regards,Uma

Hi everyone, Just wondering if Ataccama provides any best practice guides on naming conventions when working with the platform. This includes best practice for rule names, error names, sql catalogue item names etc.  Kind Regards,  Vishy