Skip to main content

Data Quality & Catalog ⚙️

Discuss, learn, ask, suggest, share all things DQ&C 🔍

  • 590 Topics
  • 1,464 Replies
590 Topics
S
Data Voyager
sumishaData Voyager
asked in Data Quality & Catalog ⚙️
Data Protection classification

I have a column in my catalog item which contains sensitive data. I have tagged this column with a Business Term. This business term is tagged as PII in Data Protection Classification section in Glossary terms. Is there a way to show at catalog item level that this table contains PII data?

232
S
Data Voyager
A
Universe Traveller
apejkoUniverse Traveller
asked in Data Quality & Catalog ⚙️
Assigning Data Usage Licensing Agreements (Policy) Through One Desktop

Hello everyone! I currently have a large group of catalog items that I am trying to assign a Data Usage Licensing Agreement too. I know that I can manually add these through Atacama online, however because of the large number I was wondering if it would be possible to create a plan to do this for me? I have all the catalogItem ids already compiled I simply do not know how to assign the policy.While I do see the option to write to policy, I cant seem to set the parent as a catalog item. Any help would be appreciated. Thank you!!! 

40
A
Universe Traveller
J
Data Voyager
jaygskData Voyager
asked in Data Quality & Catalog ⚙️
Creating Source in Ataccama One for Snowflake

I am creating new source connection from Ataccama ONE and I need help on how to connect Snowflake.From below screenshot I am trying to understand JDBC connection looks like.For e.g. I am using jdbc:snowflake://myorganization-myaccount.snowflakecomputing.com/, but it is not working.Any help is greatly appreciated.  

101
A
Ataccamer
B
Universe Traveller
Bhuvaneshwari BalasubramaniamUniverse Traveller
asked in Data Quality & Catalog ⚙️
How to access “Monitoring projects” through API calls

Hi ,I am trying to access “Monitoring projects” through API calls. For this purpose , I have created service account in KeyCloak with ‘default” and “DQIT_admin” IAM role access. Able to generate tokens with the service account. But unable to access “Monitoring projects” API , receiving 401 “Unauthorized” response.2 questions:How to check whether this service account is able to access the Ataccama application or not ? Any other or what level of access rights need to be provided to access API calls for a service account?Please note I am using Ataccama web application version 14.4.  

243
B
Universe Traveller
B
Universe Traveller
Bhuvaneshwari BalasubramaniamUniverse Traveller
asked in Data Quality & Catalog ⚙️
DQ - Monitoring project "Export"

Hi ,Kindly share your inputs on below scenario.I am trying to place the “Monitoring Projects” export  “.csv” file in a shared path/folder with in the organization. Purpose of this “Report need to accessed by different set of stakeholders who do not have Ataccama access” Is there a way to access this export file by any API calls or we can directly configure the shared folder path for example “S:/SharedPath/Ataccama/Exports” in output location field ?  Note : This is on Ataccama ONE 14.4 version. Any input is appreciated. Thanks,Bhuvi

121
anna.spakova
Ataccamer
B
Universe Traveller
Bhuvaneshwari BalasubramaniamUniverse Traveller
asked in Data Quality & Catalog ⚙️
Apply rules only to newly created records through any filter

Hi,Could anyone help me on below scenario. I am using Ataccama ONE web 14.4. Millions of records are populated in testing environment from Production real time data. However , for testing purpose manipulated few 1K records. Is there any way in Ataccama where we can apply rules and perform “Monitoring Projects” only to the new records 1K created by giving any date created filter? Regards,Bhuvi

252
B
Universe Traveller
R
Data Voyager
RadziahData Voyager
asked in Data Quality & Catalog ⚙️
Fuzzy Checks with No Unique Identifier

HiI would like to ask whether we could do fuzzy check in Ataccama that involves two different datasets let’s say Set 1 and Set 2 from different data sources. Please note that both datasets have different number of columns but fuzzy checks involved only customer name and address. However, these datasets don’t have any unique identifier. Is it achievable in Ataccama and what would be the possible steps to achieve the results after the fuzzy checks? Thanks.

293
anna.spakova
Ataccamer
A
Data Voyager
Anurag choubeData Voyager
asked in Data Quality & Catalog ⚙️
Clearing all special characters expect defined special characters

In the above image I want to clear all special characters expect .(dot) As their can be any thing in place of  ! number may have *, &, %, $, etc. I tried it using trashnondigit() function but it will remove all the characters from the number, Is their a way to do it?

262
A
Data Voyager
T
Universe Traveller
TimBrown74Universe Traveller
asked in Data Quality & Catalog ⚙️
Confusing date data

The Business wants this date attribute validated.  I don’t know why, it’s a key field and I assume managed by the DB.  Nonetheless, I need to build a rule.But I think it’s strange that the Data Type is DATE while the date values are YYYY-MM-DD. I would expect to see DATE format data in the MM-DD-YYY HH:MM:SS AM format.I don’t know how to code it in the date functions since it doesn’t have the standard date format and it doesn’t work in any string functions.Any ideas?  

303
T
Universe Traveller
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Catalog ⚙️
Monitoring Data Quality – The key to trustworthy data

Hi everyone,  Ready for the second part of our end-to-end DQ guide? Monitoring DQ is essential for building trust in your data assets.This post explores how data profiling, observability, and automated monitoring help organizations detect and address data quality issues in real-time, ensuring data remains accurate, complete, and reliable. Before starting, missed the first post? Check it out here 👇 Once your data is documented, the next step is understanding its quality. Without proper monitoring, data issues can go unnoticed, leading to poor decision-making. Data profiling helps by analyzing patterns, completeness, and accuracy, while data observability provides real-time monitoring to detect anomalies before they cause harm​. A data quality monitoring system tracks key dimensions such as validity, completeness, and consistency, helping organizations demonstrate progress and ROI. Automate this process, detecting unexpected changes and flagging them for review​. By continuously assessi

50
Cansu
Community Manager
N
Data Voyager
NBalaguraData Voyager
asked in Data Quality & Catalog ⚙️
Configure the MMM Module for generating links in emails

Hi everyoneCould you please advice if there is any instruction how smpt server can be configured for e-mail notifications?Where we can perform this configuration? Is there any option in Ataccama ONE web? I have checked this part of documentation MMM Configuration :: Ataccama ONE but it is still not clear where mmm configuration can be done

222
N
Data Voyager
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
v15 Data slices - multiple slices in the same catalog item in the same project?

Hello all,I am on v14.5.3, so I haven’t seen the Data Slice feature hands on yet. Been reading the documentation on the Data Slice and its use in the monitoring project. Say for example, I have a table of transactions, and I have some statuses:Pending Active Inactive Cancelled  Let’s say I have the following rules and the respective slices they need to be applied on:Rule 1 Status = ‘Active’ Rule 2 Status in (‘Active’, ‘Pending’) Rule 3 Status = ‘Pending’  Is it then possible to create an ‘Active’ slice, and a ‘Pending’ slice?And then in my monitoring project, I add my catalog item, then I apply my Rule 1 to the Active slice, Rule 3 to the Pending slice, and Rule 2 to both the Active and Pending slices? Or something like that? Trying to gauge how we can use it, how much maintenance overhead there will be and the effort to try and start using this feature. Any insights from people who’s already using v15 much appreciated!

352
may_kwok
Star Blazer L3
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Catalog ⚙️
Data Reconciliation Best Practices 👾Best Practice

Hi Community! In the last few months the focus of our articles was majorly on features of ONE Web platform, from the basics to more complicated practices and now it’s time to move on to the next step. From next week we are moving on to ONE Desktop🖥️ Today we will focus on Data Reconciliation, and cover how you can utilize this feature effectively for seamless data consistency across multiple sources. Let’s get to it! In a nutshell, the Data Reconciliation feature confirms that your migrated data from one database to another is intact and accurate, without any data leaks or glitches. It conducts high-level checks on data consistency by comparing attribute profile statistics and even attributes fingerprints (except for Snowflake data). Step-by-Step Guide to Data Reconciliation 1. Create a Reconciliation Project in Data Quality:To kick off your data reconciliation journey, navigate to Data Quality and then select Reconciliation Projects. Here, you can either create a new project or edit

4867
anna.spakova
Ataccamer
Arno
Universe Traveller
ArnoUniverse Traveller
asked in Data Quality & Catalog ⚙️
Selecting multiple checkboxes

Hi everyone,Sometimes I'd like to select a large number of checkboxes in Ataccama, for example to profile a large number of Catalog items. Of course creating a plan is also an option, but not always the least time consuming.How do you do this with many checkboxes involved? There are browser extensions available for many browers, but not sure if those are secure enough. Any recommendations or tips?

282
Arno
Universe Traveller
Catherine
Data Pioneer
CatherineData Pioneer
asked in Data Quality & Catalog ⚙️
Data Source - BOX.COM Connectivity

Please advise if “BOX.COM” is a data source that is supported out of the box, being that ONE can integrate with “BOX.COM” using an existing JDBC or API connector. 

151
R
Ataccamer
Catherine
Data Pioneer
CatherineData Pioneer
asked in Data Quality & Catalog ⚙️
Atacama "Security Terms" written to

Use Case:  For our Data Lose Protection initiative, we are profiling and classify OneDrive, SharePoint, etc and we have a requirement to write the “security term (data classification - restricted, confidential, etc)” to the asset sensitivity labels (metadata) 

191
R
Ataccamer
R
Universe Traveller
RAZORXUniverse Traveller
asked in Data Quality & Catalog ⚙️
Notification on Policy Update or Creation

Hi Community,Is it possible to have a email notification to a group of people in case of an update in the existing policies in One Web?

203
A
Ataccamer
S
Data Voyager
Susan24usData Voyager
asked in Data Quality & Catalog ⚙️
Semi Structured Data

Hello, i have a report that we usually saved to hard drive that i would like to bring into Ataccama, the report is in semi structured format . does ataccama accept or work with semi structure data ?or do i need to convert the report first before connecting to source ?

262
OGordon100
Ataccamer
K
Data Pioneer
KarthikeyanData Pioneer
asked in Data Quality & Catalog ⚙️
Reading array of response using Json call

Hi Team, I am trying to json call to read the response of api and process it for further results. This is my example response { "Data": [ { "Name": "Amount", "Id": "12345", "ValueListAttribute_5d": [ { "term": "Attribute", "term_Id": "67" } ], "StringAttribute_00": [ { "definition": "Dollar amount", "definition_id": "5" } ], "BooleanAttribute_18e": [ { "indicator": "true", "indicatorid": "6" } ], }, { "Name": "Amount", "Id": "12345", "ValueListAttribute_5d": [ { "term": "Attribute", "term_Id": "67" } ], "StringAttribute_00": [ {

1139
A
Ataccamer
T
Universe Traveller
TimBrown74Universe Traveller
asked in Data Quality & Catalog ⚙️
Validate a string in the format YYYYMMDD

How would I validate a string in the format YYYYMMDD is a valid date?  I assume the range of valid years is 1940 to 2050.

566
T
Universe Traveller
K
Data Voyager
kellymremzilybData Voyager
asked in Data Quality & Catalog ⚙️
SQL Catalog Item Creation Issue

I am trying to create a SQL catalog item to filter out the inactive records.  When I create the SQL catalog item I get the following error message:SQL compilation error: Object 'DEV_SOURCE.SAPMDG.VTLFA1' does not exist or not authorized.I can see the data tab for this table.  I can run profiling/DQ insights tab so I’m not sure what is happening.  I have created the query with the schema name and without the schema name. Shouldn’t the credentials be the same to create the SQL catalog item?

142
K
Data Voyager
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Catalog ⚙️
Introducing the end-to-end Data Quality Guide: Laying the foundation – understanding your data landscape

Can you truly trust your data? If you or your team hesitates before answering, you’re not alone. Many organizations struggle with understanding what data they have, where it resides, and whether it’s reliable. Without proper documentation, data quickly becomes fragmented, outdated, or even misleading, and potentially leading to missed opportunities and poor decision-making. And that’s why we’re introducing our End-to-End Data Quality Series! 🚀Data is one of the most valuable assets for any organization, but without a strong quality framework, it can quickly become unreliable and untrustworthy. In this four-part series, we’ll walk through the essential steps of achieving high-quality, trustworthy data—from documenting your data landscape to preventing issues before they arise. Each post will break down a key stage of the framework and providing actionable insights. Please do join in the discussion in the comments, let us know if you have followed any of these tips, or what helped you i

200
Cansu
Community Manager
B
Universe Traveller
Bhuvaneshwari BalasubramaniamUniverse Traveller
asked in Data Quality & Catalog ⚙️
Creating rules for Name field

Hi ,Could someone guide me on creating rules for Name field.Rule : Name should not have any special characters but can have numbers and spaces. Ex: Jhon Leo - ValidJho1 Leo - ValidJho( Leo* - Not Valid  Regards,Bhuvi

182
B
Universe Traveller
A
Universe Traveller
Ajeesh_GUniverse Traveller
asked in Data Quality & Catalog ⚙️
Send files to sftp location

Hi, How can I send a file to SFTP folder using below task item? I am using Desktop version.Can someone help me how to get the required fields from the below screenshot? I have the location where I want to drop the file. 

331
Phil Holbrook
Ataccamer
B
Universe Traveller
Bhuvaneshwari BalasubramaniamUniverse Traveller
asked in Data Quality & Catalog ⚙️
Validating Date

Hi,I am trying to validate the date given between 100 years ago and 15 years ago are valid , everything else is invalid. I am able to achieve this validation in YEAR part. I am looking for validating complete date instead of only Year part.Below are few screenshots.  

262
B
Universe Traveller

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings