Data Quality & Catalog ⚙️

Eventually, you might need all your data quality results evaluated in One monitoring projects or within regular catalog items (via business terms and related DQ rules) exported to some reporting system or into excel. And it would be nice if it looked like this: TLDR; There are 2 components attached you can use to export DQ results, one for monitoring projects, another one for regular catalog items. Monitoring Projects The idea is to read all monitoring projects metadata, join it with their check results as well as rules names, catalog item names and voilà, we can save it to excel/csv or to some database table. It sounds like something very simple, and indeed there is no black magic, but nonetheless component looks a bit more elaborated: Catalog Items DQ (based on Term evaluations) The idea is to find all catalog items that have aggregated overall data quality and join them with the DQ rules results: This component has a bit simpler implementation: Of course, the output of the component

Hi, I am wondering if the following is possible. On the glossary's term screen not all properties/metadata will always be filled. For instance in most occasions there will not be an abbreviation for a term. Is there a way, on the view screen only, to hide this abbreviation field when it contains no value? Thanks for your time! Kind regards, Albert de Ruiter

Hi team! I'm trying to implement a simple maker-checker process for Business Glossary Management: there are different Data Domains, each with its own responsible: Data Steward and Data Owner. So, for each Data Domain, the corresponding Data Steward creates the term --> the corresponding Data Owner approves it. Therefore, it's essential to consider the separation of permissions by data domains. Please give me some ideas on how I can implement it in the current product version. As far as I could investigate, grouping terms by different folders/categories in the Glossary doesn't help since I can't assign any roles to a particular folder/category: I can't assign Steward1 and Owner1 to the BusinessTerms group and Steward2 and Owner2 to the SecurityTerms, for example. How should it be implemented, then? Many thanks for considering my request!

Hi, We've loaded 2000+ terms and assigned those with desktop as a migration from a data dictionary. When we now have new catalog items, Ataccama suggests 20+ terms for a CI Attribute. This is not usable. How we tweak it to: Limit to only top 5 Stop when a term is already assigned. And here's an idea: If the column header is the same or can be matched using a simple fuzzy match, that is a better indicator than most AI. Another: Please skip some of the datatypes, a boolean … probably not a good idea. Any idea's? We are also wondering if we can turn it off. Thank you in advance! Marnix

Hi I want only the valid rule names in valid_rules column for Monitoring project export. How to implement that?

Hi, I have created a SQL Catalog item in UI however I'm trying to retrieve only 10 or so rows from the database. I am using the SELECT TOP/LIMIT clause to choose how many rows I want to retrieve both queries are having errors, I'm not sure what the problem is, does anyone have any ideas? Thank you. SELECT TOP 10 FROM x SELECT * FROM x LIMIT 10

Hello,everyone. When you load data into the database using Jdbc Writer and execute the plan every time, it keeps adding the same data from above. How can I overwrite the data? For instance, I have a table like this. I saved it in the database, made some changes to the table, and when I re-executed the plan, it added the changes on top. Duplicates appeared, to put it bluntly.I will be very grateful for your help Thank you!

Hi community! We are almost halfway into our Monitoring Project Best Practice series, today we are going to talk about Lifecycles 🎢Let’s dive in! Introduction to Monitoring Project Lifecycles The introduction of Monitoring Project Lifecycles allows you to simulate a multi-environment process (Configure > Test > Send for approval > Make public, and repeat) within a single physical environment. While you can create two physical environments, the non-production environment should primarily be used for specific purposes, such as testing new sources, integration plans, model changes, and service re-configurations. It's important to note that non-production environments are not intended for developing data quality monitoring projects and migrating between environments. For the best results, we recommend having one production environment for all metadata and a non-production environment for model changes, API work, and testing. This setup differs from other modules like RDM and MDM, where tw

Hello Community! As part of our ongoing Data Quality Best Practice series, today we'll dive into the world of time series data analysis. Time series data analysis is a powerful tool that enables you to unlock valuable insights from your transaction data by identifying trends and detecting anomalies. To make the most of this feature, here are some best practices to follow: Understanding Transaction Data and Time Series Analysis Transaction data encompasses internal and external events or transactions within a business, often related to financial and logistical records. Common examples include sales orders, invoices, purchase orders, shipping documents, payments, and insurance claims. The benefits of time series analysis include: Visualizing time series data. AI-driven anomaly detection. Keep in mind that time series analysis supports columns with timestamp information, such as DATETIME, DATE, or DAY data types. Configure and Run Time Series Analysis To get started with time series analy

Can ‘SQL Select’ stage only be used to connect with SQL Server DB ?

Happy Wednesday community! The Power of Projects: A Primer Before you dive into the exciting world of data transformations and analysis, let's lay down the foundation: projects. Think of projects as your organized toolkit – a collection of plans, components, data files, workflows, and custom scripts, all neatly arranged within logical folders. These projects live within a workspace folder, ensuring everything is in its right place. Structuring Projects Imagine the harmony of a well-structured project. Here's how it can look: A Glance at a Well-Structured Project bat: Home to batch or shell scripts for triggering plans. bin: Houses DQC plans for data cleansing and deduplication. components: Stores the goldmine of component plans for main plans. data: Nurtures your data directories. ext: Contains external lookup files and replacement lists. bat: Serves batch or shell scripts to trigger plans for lookup files. build: Holds DQC plans for crafting lookups. lkp: The indexed version of lookup

Hi everyone! How can we avoid manually changing stewards for all business terms and catalog items in Ataccama One? I will be very grateful for your help Thank you!

Hi everyone! I hope you are all excited to take one more step into ONE Desktop today 👣Today on the menu we have…..Environments! Imagine effortlessly crafting and testing your solutions across different operational landscapes like DEV, TEST, or PROD. That's what Environments are all about! These versatile tools offer a seamless platform for: Running plans on diverse databases without building separate plans. Testing plans, workflows, and web services across testing and production servers. Crafting and exporting runtime configurations for various environments (perfect for those snazzy folder shortcuts). How Environments Work: A Guided Tour Environments are your game-changers. Here's a sneak peek at how they operate: Local Launch Environment When you're running your plans or workflows on your local machine through ONE Desktop, this environment kicks in. It's your go-to for efficient local testing and development. ONE Platform Launch Environment For those times when your plans need to be

👋 Welcome to our new article series, where we will go through the individual topics that are part of the Ataccama Data Quality & Governance Certified Professional: Foundation Level Ataccama ONE v13 certification exam. We will focus on practicing and pointing out the most common mistakes. Our first topic is Adding Data Sources. 🔎 LET’S TEST YOUR SKILLS Using ONE Desktop Application, create a database connection to your database. ​​Using the ONE Desktop Application, upload the 'bloodtypes.xlsx' into your database. Create the database table <your_name>_bloodtypes, and upload the contents of the 'bloodtypes.xlsx' file. 💡 How to do it? Open Ataccama ONE Desktop Application and create a new project In your newly created project, create a plan and name it loaddata.plan Add (drag and drop from the palette on the left side) steps Excel File Reader and Jdbc Writer Upload bloodtypes.xlsx (attached to this article) into your project Open Excel File Reader and fill in File Name, Sheet Name and S

Happy Wednesday everyone! Today’s topic is Tasks! We will cover how to optimize your task management on Ataccama to effectively organize and track your work. Let's dive in! A Glimpse into Task Management At the heart of the Ataccama ONE Platform💜 lies the Task management system. This tool brings together your tasks into a Kanban board, offering an intuitive visual representation of your workload and its progress. From prioritizing tasks to altering their status and engaging in collaborative discussions, this system is your gateway to productivity. Unlocking the Potential of Tasks Within the Task management system, you can accomplish a multitude of actions: Task Creation and Management: Create tasks and their subtasks, assign them to team members, and even solicit assistance from colleagues. Change Management: Request changes like entity creation or access adjustments seamlessly. Workflow Task Management: Efficiently handle tasks related to access requests, approval workflows, and more

Hi Team, I want to use the DQ monitoring project results component. From where do we get the Monitoring project Id for the component? Thanks in Advance Anagha

Hello, I’m setting a new connection for Snowflake datasource, and I need to use key pair authentication for this. In the credentials drop-down menu it doesn’t look like this option to be present: What do you advise?

At our site we have Ataccama on premise and several sources from different data systems. It happened a couple of times that profiling on one of the sources was no longer possible and that we found that out more than a week later. After which it was difficult to track back why this happened. We do have monitoring of all kinds of Ataccama components with Prometheus. But that didn't seem to detect these issues. I was looking for a way to monitor profiling functionality. So what I came up with was a monitoring project that profiles a small table from each source, every hour. Since you can schedule a monitoring project every hour at least, and since you can add notification in case of failure, this gives us warning at most near 1 hours after something breaks. Just adding a catalog item of a table to the MP turned out not to be sufficient though. Because when the monitoring project ran, it did only a structure check, not a profile. To get it profiling, it needed an anomaly check. When you do

We have a question. The basic viewer can view the global settings and with that also a list of users. We'd rather not share this info with the basic viewer. How can we change this?