Data Quality & Data Governance ⚙️

Discuss, learn, ask, suggest, share all things DQ and DG 🔍

  • 398 Topics
  • 863 Replies

398 Topics

Albert de Ruiter
Albert de RuiterStar Blazer L1
asked in Data Quality & Data Governance ⚙️

Filtering in metadatareader on mutation datetime

Hi Community,Would there be a way in Ataccama Desktop to select records from for instance entity term that have been inserted/updated/deleted from a certain datetime, like the Published on datetime in the History tab? My intention is to use the metadatareader to read the details from term and then use this mutation datetime in the filter in the metadatareader.Kind regards,Albert

3
Albert de Ruiter
6 hours ago
M
Marcel-JanStar Blazer L1
asked in Data Quality & Data Governance ⚙️

How to connect to Hive Warehouse Connector?

We want to profile Hive managed tables in our Cloudera data lake. This is only possible if the Hive managed table is defined as non transactional. However to turn of that feature you need to rebuild the table. We had a discussion with our architecture team and they don't want to turn off this functionality.The other option then is using Hive Warehouse Connector. It is known that this has some performance costs. But we want to test this anyway.Is there any documentation on using this feature?

4
Cansu
Community Manager
1 day ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Getting Started with ONE DG: Workflows 🧑‍🏭Best Practice

Happy Wednesday Community! Today we will talk about workflows - let’s dive into it right away 🤿 Currently, workflows in Ataccama ONE are focused on the access request process. They offer additional automation for requesting changes to data access in ONE. Here are some key actions that workflows allow:If you have insufficient access rights on a node, you can request a higher access level from the node owner. The default assignee of the task can either approve or reject the request from the Kanban Board. For more information on tasks, check out our previous post on the community here.Before you can request access to entity data, make sure you meet the following prerequisites:The workflows:ableToStartRequestDataAccess trait must be enabled on the entity. Data access requests can only be enabled for the following entities: Catalog Item Source Location Connection Monitoring Project To view the list of workflows, navigate to Tasks And Workflows > Launched workflows tab. Here's what y

1865
Catherine
1 day ago
T
ThomasProData Voyager
asked in Data Quality & Data Governance ⚙️

TWS Integration

Hey all, Has anyone been able to integrate with TWS for kicking off Reconciliation or Data Quality Project runs? I have users asking if their DQ Projects can run after their jobs have run which are scheduled by TWS. I was thinking that there has to be a way that Ataccama could listen for when a TWS job runs and finishes, it could kick off a DQ or Reconciliation project to also begin running. Has anyone successfully set this up before? either in the Orchestrative server or within Ataccama Desktop?Thanks

1
L
Ataccamer
1 day ago
OGordon100
Ataccamer
OGordon100Ataccamer
published in Data Quality & Data Governance ⚙️

Customisation & Theming

Building upon the excellent work of our Flamingo design team, and a fun gradient ranging between purple and “electric” pink, I like to think that Ataccama looks pretty cool by default. However, we fully appreciate that many businesses have a need to customize their enterprise software to match their corporate brand and identity. This brings us to today’s topic… customisation!In this post we are going to cover how to set custom logos and colour themes in Ataccama DQG.  By navigating to Global Settings → Customization, it’s possible to perform a range of customisations in Ataccama DQG. As well as options for custom Translations (pirate, anybody?) and Navigation layouts to reorder and customise the sidebar, you can use the Theme section to fully customise the colours of the UI itself*, aswell as change the top logo to your own.Indeed, this last feature is as simple as drag-and-drop: You may have also noticed that this navbar is not purple! Here, we used the “Moon” theme, but there’s many

81
maykwok_hamilton
1 day ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Getting Started with ONE DG: Ataccama ONE Catalog 101 📖Best Practice

Hi Community! We are kicking off a new best practice series covering the basics of ONE DG. Each day for the next two weeks, we will dive into a new topic to level up or strengthen your ONE DG skills 💪 First things first, before starting our how-to articles let’s learn the essentials we will be using for the ONE Web Application. Knowledge Catalog is where you'll find the key tools for data management, including importing metadata from data sources, viewing, discovering, and profiling data. Let's explore the two main sections of the Knowledge Catalog: Catalog Items and Sources.Catalog items house your actual data, organized into attributes—think of them as columns in a table. These items are imported into the platform from various sources representing different storage types.Business Glossary serves as the central repository for business terms.Business terms are labels assigned to the data that help you understand and organize your data. You can also create hierarchical structures betwe

2293
Cansu
Community Manager
2 days ago
Albert de Ruiter
Albert de RuiterStar Blazer L1
asked in Data Quality & Data Governance ⚙️

Use case for Security terms

Hi Community!I was wondering if anyone has used the Security term (as extension of term) functionality in OneWeb and if so how. There seems no documentation available from the Documentation portal or the Community to clarify the modeling choices. The node has the following properties regulation is a string instead of a reference to node regulation and responsiblePerson is also a string. The effectiveDate is not of type date but string as well. Why would this be?Anyhow, I would appreciate if anyone can shine some light on this.Thanks and kind regards!Albert

0
Albert de Ruiter
2 days ago
V
vkumar192Data Voyager
asked in Data Quality & Data Governance ⚙️

Custom input for validation component

How can I map custom input parameter for Validation component in Ataccama One Web UI?   

3
J
Ataccamer
2 days ago
M
Marcel-JanStar Blazer L1
posted in Data Quality & Data Governance ⚙️

Passing a parameter from a workflow to a component with a parameterized SQL query

I have a workflow that runs multiple components. In the workflow I have a parameter called load_id that I would like to use to query all kinds of tables later in the flow. But if you use a JDBC Reader in your component you can't use the parameter in the query. You could if you were able to do an Alter Format before the JDBC Reader, but JDBC Reader allows no input.I've seen some tips here in the Community, like these: But initially I still ran into some issues before I got it working. So I decided to share a working example.This is my very simple workflow.It only has a Run DQC task that will run the component.But let me show the global variables in my workflow. I hardcoded the load_id, so my query would get rows back. Now lets look at the configuration of the Run DQC task: As you can see, I run the parameterinquery.comp component and pass on the load_id with the expression """"+load_id+"""" .On to my component:The Random Record Generator is the start of this, because with the JDBC Reade

0
M
8 days ago
Albert de Ruiter
Albert de RuiterStar Blazer L1
asked in Data Quality & Data Governance ⚙️

Layout change of preview pane in OneWeb

Hi Community,In a term's listing (overview) page you can click on the term's name after which the detail page for that term opens. If you click beside the term's name a kind of preview pane or side window opens, also showing details for that particular term. So like this:In the details screen it is possible to change the page layout, which is especially important after having added custom properties, so that a logical layout can be retained.It looks though that for this preview pane the layout cannot be changed. Does anyone of you know how to accomplish this?Kind regards,Albert

5
Cansu
Community Manager
8 days ago
Catherine
CatherineData Pioneer
asked in Data Quality & Data Governance ⚙️

Understanding the Access level for User Groups

Please advise if you have any documentation that provides a short description on each Ataccama functions pertaining to user access:  for exampleAtaccama Function Description  Administrator Data Owner Data Steward Custodian System   Full access       User Management   Full access Editing access View metadata access View metadata access Source   Full access Full access Full access View metadata access Connection   Full access Full access Full access View metadata access Folder Folders in Workspace are intended for storing SQL catalog items and virtual catalog items (VCIs).  Full access Full access Full access Full access Location   Full access Full access Full access View metadata access Catalog Item   Full access Full access Full access View metadata access Term   Full access Full access Full access View metadata access Rule   Full access Full access Full access View metadata acc

6
Catherine
9 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Policy Management 💡Best Practice

Hi everyone!In today’s post, I’ll walk you through the Policy Management module with best practices so that you can make the most of it in your organization. Let’s start with the basics: Understanding Policy ManagementPolicy Management is your one-stop hub for documenting and tracking policies and regulations that impact your entire organization. It's designed to help you stay compliant with regulations and internal policies. By using this module, you can easily define, organize, and monitor the rules and guidelines that guide your organization's operations. Navigating the Policy OverviewTo get started, head over to the Policy Management section on the left sidebar.Here, you'll find the All Policies tab that provides an overview of all your policies. Each policy is highlighted with different colors to indicate its status:No highlight: Published policy. Green highlight: Recently added policy. Orange highlight: Policy with requested changes. Red highlight: Policy with removal requested.C

421
M
9 days ago
F
FCB developerData Pioneer
asked in Data Quality & Data Governance ⚙️

Automatically add a Description field to Atacama ONE

Hello everyone!In the new version of Ataccama 14.5, Is it possible to add comments on tables and fields from the database automatically to the Description field in Ataccama ONE, if the Description is empty and the Comment from the database is not empty?I will be very grateful for your help🤗Thank you!

1
J
Ataccamer
9 days ago
M
Marcel-JanStar Blazer L1
asked in Data Quality & Data Governance ⚙️

Variables in Json Call debug filename?

I know that it is possible to use variables in filenames in a Text Writer (see here: Text File Writer: adding a datetime as suffix to an output file (ataccama.com)). But what about the Request and Response Debug File in a Json Call?I have a component for ServiceNow integration and I want to reuse another component to retrieve incident details. But when I run the main component I get this error:File file://d:/Ataccama/workspace/SNOW/distribution/debug/03_getincdetails_request_debug.txt must not be multiplicatively accessed in writing mode.The only way to get it all working, is by removing the Request and Response Debug File in the Json Call that gets incident details. A possible way around this would be if I could use variables in the filenames. But this does not seem to work in the same way that variables work in Text Writer steps (by putting the variable name between { } ).Is there another way?

3
J
Ataccamer
9 days ago
P
pdanpoonkijData Pioneer L2
asked in Data Quality & Data Governance ⚙️

Can we restrict Ataccama One User Management to Users' Stewardship group

Hello everyone,I’d like to set up access permissions in Ataccama One so that users within a particular stewardship group are limited to making changes only to that specific group (e.g., adding/removing users from certain roles within that group)?For example, HR users should only be allowed to make user related changes within the confine of the HR group.However, the user management seems to be a global page and does not have stewardship-specific access. If anyone has insights on how to achieve this restriction, could you kindly share your workaroundThank you in advance!Ben

2
P
10 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Workflows 🌊⭕Best Practice

Hi everyone, and happy Thursday!As I’ve mentioned starting this week, this post will be the last of the Ataccama ONE Web Best Practice Series for a while because…… we are moving on to ONE Desktop next week on! In this post, we’ll cover Workflows, and I highly suggest checking out the Tasks article from yesterday before reading it if you haven’t checked that out. It will help out with laying the foundation for workflows.Let's explore how these workflows supercharge your access and review request processes. Demystifying WorkflowsIn the realm of Ataccama ONE, workflows are your dynamic companions for handling access and review requests.Access Request Workflows automate data access change requests. Imagine needing higher access to an asset – this workflow empowers you to request and assigns the task to an approver, who can then give it the green light or a gentle decline.Review Request Workflows introduce automation to review changes made to data assets and get them published. If you hold

1973
P
10 days ago
Prasad Rani
Prasad RaniData Pioneer
posted in Data Quality & Data Governance ⚙️

How do i do data quality on inbound text file objects

Hello:I really appreciate the commnity helping me solve more use cases and get value out of the investment we made in Ataccama. Now - i have another area that i need help with.We have a vendor that drops a file every night - this is not comma delimited and not fixed length. Based on the first character of the line, it decides the format of that lineFor ex: if char(1) is - 1 - then the lenght of the file will be 10 characters with first 5 characters as identfier and next five char as status.But if the first character is 2 - that means it is a different type record (type 2), that has 100 characters in length - and each of them fixed length positions. Now - when i scan that file through Ataccama, it is thinking the file attributes based on the first row. But is there a way i can give the metadata to the profiler?And by the way, i need this to be a monitoring project that runs DQ on a demand basis (that i would like to trigger through Airflow when the file arrives)can some one give me some

1
anna.spakova
Ataccamer
10 days ago
Catherine
CatherineData Pioneer
asked in Data Quality & Data Governance ⚙️

Term Detection based on Metadata

Term Detection Rule: Based on metadataRules based on metadata use information about asset, e.g. description, Catalog Item or data source name.Is there an option to expand this function to detect our “add custom metadata”?  For example:  On the Catalog Item level: Data Classification, Criticality, and Data Asset Type  

4
Cansu
Community Manager
11 days ago
Prasad Rani
Prasad RaniData Pioneer
posted in Data Quality & Data Governance ⚙️

Vendor Data Source metadata changes

Hello Community. I have a vendor who shared their data through Snowflake Data Sharing. We added it as a data source and scanned the source to add the objects into my Data Catalog. I even have a sccheduled job that scans this source everyday (end of day) for sample profiling.Today, we had an issue that 10 of the tables that were previously present in the shared data source, were dropped, two new tables created, and some of the tables, have new fields added, existing fields removed, some fields renamed.This source is about 300 tables, but some of the tables ahve about 700+ fields. So, I would like to know best way for me to get a report/alert/notification onany new tables added to the source any existing tables dropped any existing table modified with new fields added any existing table modified with existing fields dropped/removed. What is the best way to accomplish this?For the dropped tables that previously existed, my end of day profile job is showing error in the DPM (because the ta

2
J
Ataccamer
13 days ago
Prasad Rani
Prasad RaniData Pioneer
posted in Data Quality & Data Governance ⚙️

Data Quality checks on catalog item can they automatically come into monitoring project when that item is added to MP?

Data Quality checks on catalog item can they automatically come into monitoring project when that item is added to MP? Here is my challenge:I have catalog items with 20 or more checks. If i add them to monitoring project - it does not automatically show as “Data Quality” for that catalog item itself. And when i add the checks to the Catalog item, they are not coming to monitoring project when i add that catalog item to the MP.Iam needing to add/configure all the rules separaely - both at Catalog item level (to get DQ percent score for that object), and also at the Monitoring project level. Is it possible to add the DQ rules - just to the Catalog item, and when I add that catalog item to the MP, it should automatically bring the existing rules on that object into the MP?​​​​

1
J
Ataccamer
13 days ago
V
Venkata RamaniData Voyager
asked in Data Quality & Data Governance ⚙️

Export list of Items to CSV

HiWe are interested to export metadata information into excel or CSV using “Export list of Items” option (see attached image).However, this is disabled in some screens.While there are ways to use desktop components to read metadata and export to CSV, this process is highly cumbersome and still doesn’t solve the purpose.Are there anyother alternatives?

2
J
Ataccamer
13 days ago
Prasad Rani
Prasad RaniData Pioneer
asked in Data Quality & Data Governance ⚙️

data source metadata changes - how to know if the field is newly added or existing field dropped?

Hello: I have a data source with 300+ tables. I have a daily profiling job (sample data).Sometimes, when a table gets changed, and a new field added/existing field dropped, when i open that table in Data Catalog, and in the “Data” tab, it shows that “N” number of metadata changes happened that are marked with a “yellow triangle” symbol (warning/alert).But, Iam able to see the list of fields that got changes, but iam not able to figure out if it is a new field added or existing field dropped or existing field changed??How can i get this data? 

3
Cansu
Community Manager
13 days ago
Prasad Rani
Prasad RaniData Pioneer
posted in Data Quality & Data Governance ⚙️

importing catalog item descriptions and attribute descriptions from dbT docs

We are using dBT for our data pipelines, and for the tables and fields, there is description maintained in dBT that we would like to export (json) and update Ataccama Data Catalog.Is there a known, trusted, supported way to automate this?I already do have a desktop plan i created that can update catalog objects and attributes based on a excel input file. I may be able to modify that if thats the only option. Iam reaching out to the community to see if there is a preferred way to get descriptions and documentation from dBT to Ataccama.

2
Cansu
Community Manager
14 days ago
V
vkumar192Data Voyager
asked in Data Quality & Data Governance ⚙️

Display Glossary/Business terms

How can I enable connection page to show glossary/business terms along with other properties. I tried editing the page template but not sure what details to add for fetching those details. 

2
Cansu
Community Manager
14 days ago
Albert de Ruiter
Albert de RuiterStar Blazer L1
asked in Data Quality & Data Governance ⚙️

Write multi-line Definition property with Metadata Writer

Hi Community,The purpose of the follwing scenario is to update multiple term names and definitions. I have created a plan that writes the terms names, definitions and GID to a csv file. In that file I can make updates on the names and definitions. With another plan I read that file and write it back into Ataccama.That works well, except in the following case. The text in the Definition can be entered on multiple lines by using the Enter button. So OneWeb will show:Text on line 1.Text on line 2.Text on line 3.In the export (the csv file) the text will look like this:"[{""type"":""paragraph"",""children"":[{""text"":""Text on line 1.""}]},{""type"":""paragraph"",""children"":[{""text"":""Text on line 2}]},{""type"":""paragraph"",""children"":[{""text"":""Text on line 3}]}]"If you import the text like this, the content will not appear on 3 lines, but literally as above and on one line.How can I import the text so it is on 3 lines again?Thanks for any suggestion.Kind regards,Albert

7
Albert de Ruiter
14 days ago

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Terms & ConditionsCookie settings