Data Quality & Data Governance ⚙️

Discuss, learn, ask, suggest, share all things DQ and DG 🔍

  • 309 Topics
  • 602 Replies

309 Topics

Albert de Ruiter
Albert de RuiterRocket Pioneer II
asked in Data Quality & Data Governance ⚙️

Text File Writer: adding a datetime as suffix to an output file

Hey all,I was wondering if you ever came accross the following requirement and has some tips maybe to implement it.The Text File Writer step can be used to create for instance log files. For tracking or governance reasons it may be necessary to keep all log files. But because the output file name is fixed, each time the file will be overwritten whenever a new one is created. Of course it is possible to manually add a datetimestamp to the file name after its creation, but that is a rather cumbersome approach. So I would like to be able to add a datetimestamp to the file name automatically. Does anyone have a suggestion?Kind regards,Albert

2
Albert de Ruiter
1 day ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Follow this Journey to Data Governance 👣

Hi everyone,  Are you just starting out with Data Governance or would like to brush off some of your skills? Check out our learning journey below and follow along to improve your Data Governance skills one article at a time⚡ First things first: Let’s start with the basics 📚 Continue with the essentials 🔑 Advance your skills 🔗 Need some brainstorming? Check out our digest issue 🧠 

70
Cansu
Community Manager
2 days ago
K
KundanSpace Explorer II
asked in Data Quality & Data Governance ⚙️

 standardization of volume data

I need to standardize volume data in ml .  I have data in galons,liters and mililiters . Can anyone please help me in this . Sample Volume Data 450ml2L5galon2.8L  Thanks in advance   

2
Ales
Ataccamer
3 days ago
OGordon100
Ataccamer
OGordon100Ataccamer
published in Data Quality & Data Governance ⚙️

Sending RDM Workflow Updates to Microsoft Teams & Slack

Hi, I’m Oliver. a Solutions Consultant here at Ataccama.Workflows and governance are integral to any form of modern data management, be it MDM or RDM. But with so much to do in a typical workday, how do you respect the time of your end users and ensure they are informed when their input is required?Out of the box, Ataccama MDM and RDM include useful functionality to send emails to defined roles (i.e. groups of users) whenever records are moved through C(R)UD workflows. But what if we need to:Create consolidated, routine summaries for cases where data changes often and frequently? Allow users (e.g. project managers) who are not directly involved in the workflow to see progress? Notify users in different ways (e.g. Microsoft Teams, Slack, or Amazon SNS)? Create dashboards of daily workflow progress to report against?Once again, we can turn to Ataccama Plans & Scheduled Workflows! Notification WorkflowsIn this example, consider a pretend client using RDM, with a custom workflow whenev

270
OGordon100
Ataccamer
3 days ago
Albert de Ruiter
Albert de RuiterRocket Pioneer II
asked in Data Quality & Data Governance ⚙️

Which entity and property applies to term type

Hey community,The term overview screen shows the entries of all types.I would like to implement a ‘term type’-filter (via Application Settings → Search Configurations) on that page. For that I need to know in which entity and property term type has been implemented, but I cannot find that. We have version 13.9.4.Does anyone have a clue?Kind regards,Albert de Ruiter

5
Cansu
Community Manager
3 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Read this if you are starting out with ONE Desktop 🔍📝

Hi everyone,  Learning new skills or starting something from scratch can be daunting sometimes. We may feel a bit lost in all the information coming at us or scroll endlessly in the search results. That’s why in this post, I want to create a how to get started with ONE Desktop journey, compiling all the articles we have published in the last months. According to your experience, you can jump a few articles or click on the one you find the most challenging at the moment. Please also do share your feedback and questions in the comments section 👇🏻 Beginner Friendly - Learn the essentials: How to work with plans? aka  your canvas on ONE Desktop:  Main elements of ONE Desktop 🔥🌍💧💨 Tips & Tricks:  If you got the basics let’s move on..  Integrations 🖇️ Best Practices from the community 💪🏻      

160
Cansu
Community Manager
7 days ago
A
aakhmetkanSpace Explorer II
asked in Data Quality & Data Governance ⚙️

To find all tables of attributes with no associated terms from the business term group

Hello everyone!I need to find all tables of attributes with no associated terms from the business term group: "FCB 360 Data attributes." The term "FCB 360 Data attributes" has multiple child terms.Is there a way to find tables that have no attribute of an unrelated term from the group of business terms: FCB 360 Data attributes. if it is possible to find at least with DesktopI will be very grateful for your help🤗Thank you!

4
anna.spakova
Ataccamer
8 days ago
F
FCB developerRocket Pioneer
asked in Data Quality & Data Governance ⚙️

To find attributes that are not linked to business terms.

Нello, everyoneI need to find attributes that are not linked business terms specifically with the name of tables and attributes. Using the "Filter" tab, I wrote such a query schema is 'cli_dwh' and attributes.any(termInstances.count() == 0) using step ONE Metadata Reader, but it seems it takes only the schema but skips the business term, gives a list of attributes that are already linked to the business term. Can you give advice on this problem?I will be very grateful for your help🤗Thank you!

3
Cansu
Community Manager
16 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Getting Started with ONE DG: Share Access 🔐Best Practice

Happy Thursday all! In this post, we will break down and go through all the how-to’s of a functionality we all frequently use in Ataccama - access & sharing. Sharing access is an essential feature that allows you to provide access to other groups and users at the asset level. By default, only admins or users with the appropriate rights can manage access to data assets. This can be configured through the manageAccess operation settings for each node for each access level. Understanding How Sharing Works When you share access to assets with other groups and users, it's important to understand how sharing functions based on group hierarchy, user roles, and node inheritance. Here are some key points to remember:Notifications: When you share a data asset or make changes to shared access levels, users or groups involved will receive notifications about the access changes. User Access: Sharing access with a user or group grants access to all data assets shared with that group. The access

781
R
17 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

ONE Desktop: Data Quality Results 🧑‍🔬

Hi Community! Are you looking to retrieve Data Quality (DQ) results from monitoring projects or catalogs using Ataccama's ONE Desktop? We have created a set of easy-to-follow steps to help you achieve this efficiently. These steps will mimic the API calls created for the same purpose and return DQ results. Let’s get into it 🔮 PrerequisitesBefore diving into the process, make sure you have the following prerequisites in place: Active Server Connection: Ensure you are connected to the ONE Web Application. Input Data: You need the data for which you want to retrieve results. You can use either the appropriate Input step or the MMM Reader step. You can add input data by dragging and dropping files onto the canvas or by configuring the steps manually. If multiple input steps are required, use the Join step and create connections accordingly.   Note: Depending on the step you wish to use to retrieve results, different parameters will be required in the input. Refer to the individual step

793
Cansu
Community Manager
21 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

How to create Post Processing Plans? 🖋️

Hi everyone, happy Monday! In this post, I’ll walk you through best practices for utilizing Post Processing Plans. After executing data quality (DQ) evaluations, the results are transferred to ONE Desktop.In the world of data management, the journey doesn't end with data quality evaluations. Once your data has undergone scrutiny and validation, the real magic happens in the post-processing phase. This is where you mold your data into the desired form, perform intelligent actions, and orchestrate its path to its next destination.What are Post Processing Plans?In the realm of Ataccama's ONE Desktop, post-processing plans are the key to unlocking the full potential of your data. These plans serve as bridges that connect the results of data quality evaluations with a world of endless possibilities. They allow you to decide what to do with the data next, whether it's isolating the problematic records, assigning tasks, sending out notifications, or exporting to an external database or custom

412
Cansu
Community Manager
21 days ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

ONE Desktop: Component Steps Part 1️⃣

 Happy Tuesday community 🙋‍♀️ This week we have a series of best practices on all things components on ONE Desktop. Component steps, often referred to as simply "components," are instrumental tools that come with unique capabilities. Here, we'll explore what makes components special and how to make the most of them in your data projects. What Sets Components ApartComponents are akin to plans but with distinctive attributes: Reusable: Components can be employed repeatedly, functioning like steps within other plan files. Parameterization: Components can have properties within their constituent steps mapped as parameters, allowing external plans to interact with them (learn more in the section on "Mapping Properties as Parameters"). Console Launch: Components can be executed directly from the console, with parameters for added flexibility. Example: The Component in ActionComponent files are identifiable by their ".comp" extension and can play pivotal roles within your data workflows.

592
Cansu
Community Manager
21 days ago
T
TamilSpace Explorer
asked in Data Quality & Data Governance ⚙️

Is it possible to install Ataccama free version in Azure ?

WE are trying to do some HandsOn by ourself. As part of it, we wanted the Atacama free version to be installed in Azure and our team can access it. Not sure whether it is possible and how to Install.

2
Cansu
Community Manager
21 days ago
N
NatarajanSpace Explorer
asked in Data Quality & Data Governance ⚙️

12.9.0 version ugrades

Hello folks,  I am new for working atacccama tool. we are planned into data migration from ataccama 12.4.1 into 12.9.0 version. If anyone please guide me for tool upgrading

1
Cansu
Community Manager
21 days ago
P
pdanpoonkijRocket Pioneer II
asked in Data Quality & Data Governance ⚙️

Retrive KeyCloak user information using Ataccama One plan

Hello community, I need to retrieve KeyCloak user information (e.g., email address) based on their username using Ataccama One Desktop. Has anyone done this before? If so, can you please share your experience or point me to some documentation?Thank you in advance for your help!

2
Cansu
Community Manager
24 days ago
A
Ajeesh_GSpace Explorer
asked in Data Quality & Data Governance ⚙️

Ataccama Server Environment

Hi I am not able to configure Ataccama Server. Can anyone help me on this?  I am getting below message when I double click on ‘start-server.bat’ Using java at: "C:\ataccama-one-desktop-14.5.0\jre\bin\java.exe"Using DQC  at: "C:\ataccama-one-desktop-14.5.0\runtime"Using java at: "C:\ataccama-one-desktop-14.5.0\jre\bin\java.exe"Using DQC  at: "C:\ataccama-one-desktop-14.5.0\runtime"Picked up _JAVA_OPTIONS: -Djavax.net.ssl.trustStore=C:\Windows\Sun\Java\Deployment\trusted.certsPicked up _JAVA_OPTIONS: -Djavax.net.ssl.trustStore=C:\Windows\Sun\Java\Deployment\trusted.certsWed Nov 01 16:25:19 IST 2023 : Could not listen on port 1529 on host localhost: java.net.BindException: Address already in use: bind2023-11-01 16:25:20 [INFO]    com.ataccama.dqc.server.bin.OnlineCtl messageId=generic_msg message=Starting server on localhost2023-11-01 16:25:21 [ERROR]   com.ataccama.dqc.commons.license.LicenseManager messageId=generic_msg message=DQC license not verified! Program will exit.2023-11-01 16:2

1
Cansu
Community Manager
27 days ago
S
sgillaRocket Pioneer
asked in Data Quality & Data Governance ⚙️

Splitter Transform on DQS for splitting string

I am trying to use Splitter to transform the stringExample : Input: BeWhoYouAreSplitter output - ‘e ho ou re’ - skipping the first letter as ‘Upper case’ is the separatorDesired output - ‘Be Who You Are’How do I achieve this? Also if the string is all lower how to split it?Example: Input: ‘bewhoyouare’desired output - ‘Be Who You Are’

3
L
Ataccamer
28 days ago
S
suryakanth.EmishaRocket Pioneer
asked in Data Quality & Data Governance ⚙️

How to check duplicate records in ONE Desktop, and using fuzzy logic?

Hello. i have a general material data csv file, and in that i have to check for duplicates based on material data. Currently i'm trying with the matching step by creating partitions and key rules but still i'm not sure about it. So is there anyone knows about solving this process?? Thanks!

4
Cansu
Community Manager
1 month ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Virtual Catalog Items (VCIs) - Part II 📇

Hi everyone,  We are here with Part II of our VCIs best practice series and today we will delve deeper into two significant aspects of working with VCIs - connecting to natively supported data sources and non-natively supported data sources.In the first post, we explored the creation and transformation of Virtual Catalog Items (VCIs) within Ataccama ONE. You can check it out here 👇🏻 Connecting to Natively Supported SourcesAtaccama ONE makes it simple to connect with data sources that are natively supported by following these steps: Create a VCI, as explained in the previous section. To read data from a natively supported source, add the "JDBC Reader" step to your plan. Depending on your transformation requirements, you can use the JDBC Reader step alone or combine it with existing catalog items. Configure the "JDBC Reader" step as follows: On the "General" tab, in "Data Source Name," provide the source identifier of the connected database. You can find the source identifier in

300
Cansu
Community Manager
1 month ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

Virtual Catalog Items (VCIs) - Part I📇

Hi everyone,In this post, we will talk about Virtual Catalog Items 🦋 You might have participated in our live training session that took place a few months back where we talked about how to create and work with VCIs. You can always watch back the session and check out the live demo here  👇🏻Virtual Catalog Items (VCIs) provide flexibility and efficiency when working with data in the ONE Desktop platform and in this article you can find best practices for creating and transforming Virtual Catalog Items to make the most of your data management efforts. Let’s start with the basics: Why Use Virtual Catalog Items?VCIs are particularly useful in several scenarios: Applying Transformation Logic: When you need to apply ONE Desktop and in-source (SQL) transformation logic before working with raw data. This includes tasks such as filtering data, joining tables, and standardizing values. Integrating Non-Native Sources: When you want to integrate data from sources that are not natively supporte

310
Cansu
Community Manager
1 month ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

How to join tables for cross-entity checks? ✅

Hi everyone!Data quality management often requires comparing data from various sources and performing cross-entity checks to ensure accuracy and consistency. In this post, I’ll walk you through the process of creating cross-entity checks. Let’s get started 👇🏻 An Overview of the General StepsTo perform cross-entity checks, you need to follow these five key steps: Create a Virtual Catalog Item (VCI) in IDE: The Virtual Catalog Item acts as a container to hold the tables you want to compare.   Join the Necessary Tables: Identify and join the relevant tables from your data sources that can be merged for comparison. Sync the VCI with the Web Application: Ensure that the Virtual Catalog Item you created is synchronized with the web application for further processing. Create a Rule to Compare Data: Develop a rule that specifies the comparison logic for data across different tables. Create a Monitoring Project: Set up a monitoring project to compare table values at a specific periodi

190
Cansu
Community Manager
1 month ago
L
Ataccamer
Lisa KovalskaiaAtaccamer
posted in Data Quality & Data Governance ⚙️

Building a notification-based integration with Ataccama: a cheat sheet

Hi there! Lisa from Ataccama here. I’m a Solution Consultant and I love exploring the different ways our platform can become part of the business and technology ecosystem for our customers.In this post, I will share a framework for integrating Ataccama with a work management or issue tracking software like Jira. Delivering data quality insights and alerts from Ataccama to an issue tracking tool can help to maintain transparency and accountability, and to facilitate cross-functional team work. Let’s explore an example! Use case Ataccama’s Data Quality Monitoring Projects are a powerful tool for regular data quality evaluation and reporting on the results. You can configure DQM Projects to run periodic evaluation of data against DQ Rules, and provide various insights to the business. Do you need to send a digest whenever fresh DQM results are ready? Or do you need to generate alerts when specific DQ rules fail, or when the data quality results fall under a certain threshold? Once you’ve

2
L
Ataccamer
1 month ago
R
Raj YalakaSpace Explorer II
asked in Data Quality & Data Governance ⚙️

How to Add Date Parameter to output file extension

Hi, I’m trying to add the data value to the file extension in the component but I’m not able to get the expected results. I have added a new string column using alter format step (tostring(today(), 'yyyy-MM-dd'). When I pass the column as $date or ${date} in the output filename, I’m not seeing the data on the output filename. 

2
R
1 month ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

How to Work with Data in ONE Desktop 🖥️

Hi everyone, happy Monday! In this post, we will go through a few best practices on how you can work with data in ONE Desktop. Before getting into it, don’t forget to check out some posts you might find useful: Browsing Data in ONE ExplorerFrom ONE Explorer, you have the ability to access various components of your data, much like you would in the Knowledge Catalog of the ONE Web Application. ONE Explorer in ONE Desktop includes:Catalog Items Components Lookup Items Monitoring Projects Rules Terms   This seamless access to your data provides a comprehensive overview of your information, enabling you to manage your data assets with precision.Working with Data in ONE DesktopThere are two primary methods for reading and working with your data within ONE Desktop:1. Catalog Item Reader Step:This step allows you to access data from different sources defined in the Metadata Management Module (MMM). When you drag and drop a Catalog Item from MMM into your plan, the Catalog Item Reader step is

400
Cansu
Community Manager
1 month ago
Cansu
Community Manager
CansuCommunity Manager
published in Data Quality & Data Governance ⚙️

How to import terms from ONE Desktop 🟣🔎

Hi everyone,  This week’s last best practice is on how to import terms from ONE Desktop 🖥️ In data governance, importing terms is a crucial step. Using ONE Desktop, you can streamline this process by batch-importing terms into your ONE Web Application instance. This guide walks you through the process of importing terms from a CSV file into your ONE instance using the Text File Reader and ONE Metadata Writer steps. Please note that while we use the Text File Reader in this example, you can choose alternative input steps like JDBC Reader or ONE Metadata Reader.Step 1: Prepare Your Term FileBegin by preparing your term file. In this example, we have a CSV file called party.csv, which contains several terms: Step 2: Create a New PlanCreate a new plan in ONE Desktop. Simply drag and drop your term file into the plan. This action will automatically generate a new Text File Reader step. Step 3: Edit Text File Reader PropertiesEdit the properties of the Text File Reader step:In the Columns s

510
Cansu
Community Manager
1 month ago

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Terms & ConditionsCookie settings