Data Quality & Catalog ⚙️

Hello, I am trying to understand better where timeliness dimension can be applied with all the possible case scenarios. The documentation explains about the configuration of the dimension but with no real time examples. Could you please help with some use cases from the real world?For something like below , what do we need to take into consideration when implementing the rule? Attribute must be updated Daily ; effective date= Today-1. Load dates are Sun-Sat excluding Holidays.

Hi everyone,We have created several monitoring projects in Ataccama, each dedicated to a specific catalog item. Our approach is to create special tables per clientId and measure those within the monitoring project-effectively one monitoring project per catalog item.Could you please share your thoughts on this approach? Is this a best practice or are there more efficient ways to handle multiple client-specific data quality checks?Additionally, we want to create individual postprocessing plans for every rule in a monitoring project. For example, if a catalog item has 20 rules, we want 20 separate postprocessing outputs (Excel files) listing the bad records per rule. This is to share these sheets with data stewards who will then fix the data and upload it back.However, we are encountering a limitation: Ataccama currently does not allow more than 10 postprocessing plans per monitoring project.What are your recommendations or best practices to handle this? Should we split monitoring project

I need to create a complex duplicate check rule.  I am familiar with the aggregation rule; however, in this case it won’t work for me.  Here is the logic to determine duplicates:If the name, unit of measure, location, and type are all the SAME but the code (not a PK) is DIFFERENT then it is considered a duplicate.These would be considered duplicates:Code Name UOM Location Type 123 Trek LBS Houston Bike 234 Trek LBS Houston Bike            This is not considered a duplicate because we can extend products to different locations instead of creating a new row: Code Name UOM Location Type 678 Razor LBS Houston Scooter 678 Razor LBS San Antonio Scooter            When I write the rule, I use the aggregation rule to select name, uom, location type is not unique; however, that can give me false positives for the scenario above (Code = 678).  I don’t see a way to say these fields are the same while this field is different. 

One of our teams need to use ataccama as a self service tool and catalog the excel file. How to do that?​@anna.spakova 

Hi,I have created a simple monitoring project with 12 catalog items for which I do only one dq check per catalog item + anomaly detection for this one attribute. However it seems that for this the system runs a full profile of every catalog item. Therefor this project is very heavy for the system to process. Can we adjust some settings so that the profiling is only run for this one attribute per table? Seems like a waste of resources this way.kind regards, Jur Dördregter

I have a requirement in which I am supposed to check completeness and data freshness on a catalog item at row level and at the same time I have business rules that are table level validations like - checking maximum sales value for previous batch/load is in allowed limits (95 to 105%)e.g., if aggregated Sept value is e.g, sum(sale) = 100 when month = max(month_field) then next load e.g., October value can have value sum(sale) = 105 or 95 anything other than this is fail of my dq check.I am using Ataccama web v15.4 Issue here is I cannot store the max and second max values in same catalog item and apply the rule, any suggestions? I am thinking of creating a sql catalog item to get the subset of data and then apply business rules that need aggregated values.Please suggest the right approach/alternative here?

Hi I have tried looking into an article on how can we do data reconciliation on Ataccama, but I only found that can be achieve in Ataccama ONE Web, not on ONE Desktop. How can we achieve the same in ONE Desktop?My use case is, I want to know the number of rows between two tables from two different databases - Oracle and Snowflake. I tried doing that in ONE Web but it took a lot of time to load.

Hii folks,We have a requirement to do DQ evaluation on a table that has few billions of records (> 10 billions). We have to perform various checks on this table that includes aggregations for uniqueness check, completeness, data format etc. Making a subset of data also doesn’t help in reducing volume to desirable/executable range as it still has total count in billions.What would be the ideal way to to DQ on such tables? Your inputs are highly appreciated here. Thanks! ​@anna.spakova ​@Lisa Kovalskaia  any suggestions here?

I want to create a sql catalog item by joining one table from snowflake schema and other from one data file uploaded as catalog item. I don’t see the table available for selection from SCI creation screen. Is it not possible to do?

Hi,  is it possible to remove a specific run of the data observability? We had to rerun data observability today, but now we see a lot of anomalies because part of the data had remained the same between two runs. I want to trow away both runs and then rerun again to keep things clean. kind regards, Jur Dördregter

Can we connect Ataccama to Sharepoint xl file? if yes, how can I achieve this? I am new this forum and new to ataccama.

Adding more details to a list view and providing additional filtering options can help business users in their daily tasks as you scale Data Governance across multiple teams, geographies, or business functions.In this article we’ll customize the Business Glossary to show additional term attributes as columns in the list view and to enable filtering by those attributes. This approach generally works across the Ataccama ONE Web app, so you can use this article as a guide to customize other pages, for example Rules, Catalog Items, or Monitoring Projects.You don’t have to be a developer to follow this guide. If you come across any concepts you’re not familiar with, have a look at our documentation. Adding columns First, navigate to the Business Glossary’s Terms list. In the top right, find the 3-dot menu which features the option to Edit page. In the side bar that opens up, go to the Layout tab. You’ll see the JSON that describes how the objects are shown on this page when the web app load

I have an Excel file that contains the SCI names, corresponding queries, folder name(workspace name), source connection name for each catalog item. I want to create a desktop plan which will help to generate the SCI though ONE Metadata writer or graphql query. Is there any possibilities to do that?

Hello,I want to integrate my Ataccama monitoring results with Teams notification.I checked the below link but Office 365 connectors within Teams are retiring, that's may be why I do not see Incoming webhook on my Teams. Is there any way to set up Teams with Atacamma using workflows instead?https://docs.ataccama.com/one/latest/notifications/ms-teams-integration.html#add-incoming-webhook-to-ms-teams-channel Please suggest.

Hello folks,I am creating a new connection on web application for OneDrive/SharePoint using Microsoft Entra ID; while doing so there's this field (Drive Id) which is optional if you have multiple drives and you want to limit the connection to a specific drive, we enter the ID of the drive you want to connect to.Without Drive Id I am able to access the root folders and their content but I want to target a specific child folder.We tried the actual drive ID, folder name, web URL for that particular folder but faced below mentioned errorError - com.ataccama.extension.onedrive.connector.api.OneDriveConnectorException: Unable to connect to OneDrive  This is the error message I get if we provide Drive Id 

Is it possible to create and assign DQ rules from one desktop without using/taking any rules from one Web. I also want to know if we could assign dimension information to rules in desktop.

The ProblemI need to connect to a Collibra API that has a tricky two-step login process: First, I get a temporary access token using a standard Client ID and Secret. Then, to actually get data, every API call must include two headers: the temporary token and a second, custom header called XFeedAuth that acts like a static password. The issue is that the standard Ataccama connectors (like the Collibra Reader or a generic server connection) don't seem to have a way to add that second, custom XFeedAuth header. My Plan & My Question My plan is to build the connection manually using two chained Json Call steps: the first step gets the token, and the second step uses that token and manually adds both the required headers.My question for you is: Does this sound like the right way to do it? Or is there a simpler, more built-in way in Ataccama to handle an API that requires both a bearer token and a second custom authentication header?

Is there a way to disable the “Create Task” option for Data Consumers (view only users)?  

 Can we restrict Admins to view sample data in Catalog or Invalid records in Monitoring Projects. 

Hi ​@anna.spakova My manager wanted to get a metric on how many rules each person created. Is it possible to take the metric from ONE Desktop Plan?

Is there a way to implement data slices or leverage other techniques on Salesforce tables to monitor only specific attributes, rather than profiling the entire table?Our Salesforce tables are quite large, and attempting to profile the full dataset is causing our on-prem DPM service to crash. We're looking for a more targeted approach to reduce the load and improve stability.Any guidance or best practices you can share would be greatly appreciated.

Hi I am using v15.4.0 on windows 11 cloud pc and trying to run a small metadata reader project from Advanced DQ training projects that come along with ataccama installation.I’m getting the warning -’Status Tracking Issue’ when tried running it with correct Ataccama Environment selected. The result of the plan are not rendered but the job status is shown as completed on Admin Server. What could be the issue here? Any resolution that you can think of pls advise?cc: ​@Ataccama Community Admin 

Hi ​@anna.spakova,Would Ataccama provides REST APIs to read metadata of rules, catalog items and monitoring projects?We would like to use python API capabilities and read the metadata from API in the form of JSON. Please let us know if this is a possibility.

hi Team,I would like to understand a couple of things related to implementation of slices directly on catalog item v/s filtering out records in sql or virtual catalog items and then using these items in monitoring projects (MPs) I am adding a data slice on a data catalog item X having e.g., 300 millions of records in total, sliced subset is having records around 30-35 millions -Question: when I run the MP, will it first pull all 300 records and then filter out those 30 million records based on slice and then do the DQ evaluation? Also, In case I use SQL catalog item (SCI), I am already cutting down undesired records, so I’m directly applying DQ evaluation on 30 million records, will it be faster in this case than above scenario?In any case, Slice / SCI/ VCI - how are we calculating the overall quality of the data catalog item here? Are there any guidelines on calculation of DQ on the whole dataset using output/score from DQ of subset of data.Really looking forward for your inputs here!

Hey Community!I would like to create an overview with One Desktop that provides me insight in which assets have been shared with which groups/persons with which type of access. I have been checking the metadata model, but cannot exactly locate the relevant entities that I could use for this purpose. Does anyone have an idea? Maybe only possible with GraphQL?Kind regards,Albert