Learn & Share

Hello, I am trying to understand better where timeliness dimension can be applied with all the possible case scenarios. The documentation explains about the configuration of the dimension but with no real time examples. Could you please help with some use cases from the real world?For something like below , what do we need to take into consideration when implementing the rule? Attribute must be updated Daily ; effective date= Today-1. Load dates are Sun-Sat excluding Holidays.

Hi everyone,We have created several monitoring projects in Ataccama, each dedicated to a specific catalog item. Our approach is to create special tables per clientId and measure those within the monitoring project-effectively one monitoring project per catalog item.Could you please share your thoughts on this approach? Is this a best practice or are there more efficient ways to handle multiple client-specific data quality checks?Additionally, we want to create individual postprocessing plans for every rule in a monitoring project. For example, if a catalog item has 20 rules, we want 20 separate postprocessing outputs (Excel files) listing the bad records per rule. This is to share these sheets with data stewards who will then fix the data and upload it back.However, we are encountering a limitation: Ataccama currently does not allow more than 10 postprocessing plans per monitoring project.What are your recommendations or best practices to handle this? Should we split monitoring project

I need to create a complex duplicate check rule.  I am familiar with the aggregation rule; however, in this case it won’t work for me.  Here is the logic to determine duplicates:If the name, unit of measure, location, and type are all the SAME but the code (not a PK) is DIFFERENT then it is considered a duplicate.These would be considered duplicates:Code Name UOM Location Type 123 Trek LBS Houston Bike 234 Trek LBS Houston Bike            This is not considered a duplicate because we can extend products to different locations instead of creating a new row: Code Name UOM Location Type 678 Razor LBS Houston Scooter 678 Razor LBS San Antonio Scooter            When I write the rule, I use the aggregation rule to select name, uom, location type is not unique; however, that can give me false positives for the scenario above (Code = 678).  I don’t see a way to say these fields are the same while this field is different. 

Hi,I have created a simple monitoring project with 12 catalog items for which I do only one dq check per catalog item + anomaly detection for this one attribute. However it seems that for this the system runs a full profile of every catalog item. Therefor this project is very heavy for the system to process. Can we adjust some settings so that the profiling is only run for this one attribute per table? Seems like a waste of resources this way.kind regards, Jur Dördregter

I have a requirement in which I am supposed to check completeness and data freshness on a catalog item at row level and at the same time I have business rules that are table level validations like - checking maximum sales value for previous batch/load is in allowed limits (95 to 105%)e.g., if aggregated Sept value is e.g, sum(sale) = 100 when month = max(month_field) then next load e.g., October value can have value sum(sale) = 105 or 95 anything other than this is fail of my dq check.I am using Ataccama web v15.4 Issue here is I cannot store the max and second max values in same catalog item and apply the rule, any suggestions? I am thinking of creating a sql catalog item to get the subset of data and then apply business rules that need aggregated values.Please suggest the right approach/alternative here?

Hii folks,We have a requirement to do DQ evaluation on a table that has few billions of records (> 10 billions). We have to perform various checks on this table that includes aggregations for uniqueness check, completeness, data format etc. Making a subset of data also doesn’t help in reducing volume to desirable/executable range as it still has total count in billions.What would be the ideal way to to DQ on such tables? Your inputs are highly appreciated here. Thanks! ​@anna.spakova ​@Lisa Kovalskaia  any suggestions here?

I want to create a sql catalog item by joining one table from snowflake schema and other from one data file uploaded as catalog item. I don’t see the table available for selection from SCI creation screen. Is it not possible to do?

Hi,  is it possible to remove a specific run of the data observability? We had to rerun data observability today, but now we see a lot of anomalies because part of the data had remained the same between two runs. I want to trow away both runs and then rerun again to keep things clean. kind regards, Jur Dördregter

Can we connect Ataccama to Sharepoint xl file? if yes, how can I achieve this? I am new this forum and new to ataccama.

I have an Excel file that contains the SCI names, corresponding queries, folder name(workspace name), source connection name for each catalog item. I want to create a desktop plan which will help to generate the SCI though ONE Metadata writer or graphql query. Is there any possibilities to do that?

Hello,I want to integrate my Ataccama monitoring results with Teams notification.I checked the below link but Office 365 connectors within Teams are retiring, that's may be why I do not see Incoming webhook on my Teams. Is there any way to set up Teams with Atacamma using workflows instead?https://docs.ataccama.com/one/latest/notifications/ms-teams-integration.html#add-incoming-webhook-to-ms-teams-channel Please suggest.

Is it possible to create and assign DQ rules from one desktop without using/taking any rules from one Web. I also want to know if we could assign dimension information to rules in desktop.

The ProblemI need to connect to a Collibra API that has a tricky two-step login process: First, I get a temporary access token using a standard Client ID and Secret. Then, to actually get data, every API call must include two headers: the temporary token and a second, custom header called XFeedAuth that acts like a static password. The issue is that the standard Ataccama connectors (like the Collibra Reader or a generic server connection) don't seem to have a way to add that second, custom XFeedAuth header. My Plan & My Question My plan is to build the connection manually using two chained Json Call steps: the first step gets the token, and the second step uses that token and manually adds both the required headers.My question for you is: Does this sound like the right way to do it? Or is there a simpler, more built-in way in Ataccama to handle an API that requires both a bearer token and a second custom authentication header?

Is there a way to disable the “Create Task” option for Data Consumers (view only users)?  

 Can we restrict Admins to view sample data in Catalog or Invalid records in Monitoring Projects. 

Hi ​@anna.spakova My manager wanted to get a metric on how many rules each person created. Is it possible to take the metric from ONE Desktop Plan?

Is there a way to implement data slices or leverage other techniques on Salesforce tables to monitor only specific attributes, rather than profiling the entire table?Our Salesforce tables are quite large, and attempting to profile the full dataset is causing our on-prem DPM service to crash. We're looking for a more targeted approach to reduce the load and improve stability.Any guidance or best practices you can share would be greatly appreciated.

Hi I am using v15.4.0 on windows 11 cloud pc and trying to run a small metadata reader project from Advanced DQ training projects that come along with ataccama installation.I’m getting the warning -’Status Tracking Issue’ when tried running it with correct Ataccama Environment selected. The result of the plan are not rendered but the job status is shown as completed on Admin Server. What could be the issue here? Any resolution that you can think of pls advise?cc: ​@Ataccama Community Admin 

Dear Fellow Users,I have a certain issue using the Matching step in workflow editor.I composed a Matching Function, and assigned it to a data field as Matching Rule.Now I want to use the result of the Matching Rule further in the next steps as value.Please see the attachment for exact presentation of the case.First attachment demonstrates how the Matching step is structured - the details in question are highlighted in green frames.Second attachment demonstrates the function applied, and the result I want to take as value to be processed further. Any solution is keenly welcome.Thank you, Laszlo

Hi ​@anna.spakova,Would Ataccama provides REST APIs to read metadata of rules, catalog items and monitoring projects?We would like to use python API capabilities and read the metadata from API in the form of JSON. Please let us know if this is a possibility.

hi Team,I would like to understand a couple of things related to implementation of slices directly on catalog item v/s filtering out records in sql or virtual catalog items and then using these items in monitoring projects (MPs) I am adding a data slice on a data catalog item X having e.g., 300 millions of records in total, sliced subset is having records around 30-35 millions -Question: when I run the MP, will it first pull all 300 records and then filter out those 30 million records based on slice and then do the DQ evaluation? Also, In case I use SQL catalog item (SCI), I am already cutting down undesired records, so I’m directly applying DQ evaluation on 30 million records, will it be faster in this case than above scenario?In any case, Slice / SCI/ VCI - how are we calculating the overall quality of the data catalog item here? Are there any guidelines on calculation of DQ on the whole dataset using output/score from DQ of subset of data.Really looking forward for your inputs here!

Hey Community!I would like to create an overview with One Desktop that provides me insight in which assets have been shared with which groups/persons with which type of access. I have been checking the metadata model, but cannot exactly locate the relevant entities that I could use for this purpose. Does anyone have an idea? Maybe only possible with GraphQL?Kind regards,Albert

Hi All,Does anyone have a best practice for creating mappings between related reference data?   We need to be able to relate various datasets used by different systems that have the same "entity" but the codes for that entity are different. Some examples:Entity:                     StateTable 1 Value for State:        NYTable 2 Value for State:        034Table 3 Value for State:        34Table 4 Value for State:        New York  Entity:                     AutoTable 1 Value for Auto:        AATable 2 Value for Auto:        AUTable 3 Value for Auto:        01Table 4 value for Auto:        Auto We need to be able to "map" these values within RDM, so we know that NY, 034, 34 and New York are all "State", and that AA, AU, 01 and Auto are all "Auto".  Additionally, these mappings may need to be hierarchical.  In Informatica, these objects are called “crosswalks”, and they exist as base entities within the tool. We need to duplicate that type of behavior. If you’ve implemented this, or have

I need some assistance creating a AQL query for finding a table within a database and schema. How would this be written? 

I have few monitoring projects in web but I would want to add them as components in workflows and run from desktop on schedule.Is it possible and if so what are the steps to do?  Are there any pre-requisites for this?Thanks in advance!