Data Quality & Catalog ⚙️

How can I enable connection page to show glossary/business terms along with other properties. I tried editing the page template but not sure what details to add for fetching those details.

Hello Dears, Who went through migration from 13.9 to 14.5 or even to 15.2? How did it go? We were planning to upgrade soon to 14.5, but were surprised realizing how much manual activities it will take from our side. Such as recreating all monitoring projects, lookup files, post processing… My question: will migration to newer version always required so much manual intervention or it is only when it comes to migration from 13.9 to 14.5? Thank you in advance. Kind regards, Maria

Is it possible to customise the subject (screenshot below) and add things like DQ rule or Project name by any chance? Currently it just returns “Project run finished”. I have some groups with several monitoring projects and would like to know the project name in the subject line itself. And having the ability to add overall Data QUality results of that MP in the subject line will be a definite plus. For Ex: “Project run finished for Security Master Start of Day at 5:05AM PST on 4/2/2024 with overall Data Quality of 84% ” (the bold and underlined above would be variables available for that monitoring project)

We need the change datetime of rules, catalog items, attributes and monitor projects. How can we extract these using one Desktop. In One Web for these assets the history can viewed, but I don't see where I can get these change dates from. Where is the version information stored and how can we read that info?

Can we create a lineage in Ataccamma which shows the lineage from source till the target?There are many stages involved before reaching the target. This lineage should help me in doing root cause analysis when a column has a different value in the target(Delivery). can someone help me on this?

Hi everyone! This week we continue to explore different features of Monitoring Projects ✨Each day this week, we will share a best practice post on Monitoring Projects. We are kicking it off with a popular feature - notifications 🔔 General Notifications There are two types of notifications available: General and Specific. General notifications cover various scenarios such as monitoring failures, critical structure errors, new results availability, data quality issues, and anomalies. Specific notifications offer even greater granularity, allowing you to set custom data quality thresholds and filter conditions for triggering notifications. Adding General Notifications To set up notifications for predefined scenarios: Navigate to the Notifications tab of your project. Under General, select Add Notification to create a new notification or Edit an existing one. Provide a name for the notification. Choose the events that should trigger notifications (e.g., Monitoring failed, New version of r

Hi Community, Is it possible to customise the subject (screenshot below) and add things like DQ rule or Project name by any chance? Currently it just returns “Project run finished”.

Dear community, In the overview screen of an entity, like the business terms screen, each column has some predefined width. Some of the term names are long and are cut off because the column is too small. Is there a way to define the width of a column yourself? Thanks and kind regards! Albert

I have an Ataccama workflow (EWF file) developed in ONE Desktop that executes after an ETL job. The workflow kicks off multiple data quality monitoring projects, extracts and aggregates their results, generates a Tableau hyper file from those results and publishes it to Tableau server. I’m currently executing that workflow on a dedicated server, but would prefer to deploy it to Ataccama server and execute it via an API call. Is there a way to deploy and execute workflows on Ataccama server?

We are on 14.5.1. We have a custom property on our terms, that currently is mandatory. We want to find out if we can change it to non-mandatory. When I click on edit on the property, it only allows me to edit the name. Is it possible to turn off the mandatory flag?

If I create a component with a step that maps an expression as a parameter, and call that component from a workflow, I get a “Model is not valid” exception. The attached provides a minimal example of this behavior. The component, param_test.comp, uses a random record generator with a single column, the expression of which is mapped to a parameter named “param.” The parameter value is set to “param_value.” The workflow, param_test.ewf, calls the component, passing the parameter, “param,” with value “param_value.” If the component is executed in isolation, execution completes successfully. If the component is executed from the workflow, the below error message is printed and execution halts. Is this a supported scenario? If not, how can I map a parameter value to a column within a component that is called from a workflow? 25.03.2024 01:20:25 [INFO] Processor started25.03.2024 01:20:26 [INFO] Task 'Run DQC' : IN_QUEUE25.03.2024 01:20:26 [INFO] Task 'Run DQC' : RUNNING25.03.2024 01:20:26 [

Hey all! I am working through a use case that wants to check if data is being populated. The issue I am facing is that when there are no records being populated, the case statements I am using will not work (due to no records being in the table) and no rule seems to work as well due to the fact that when no records are passed through a monitoring project, it “passes” but shows that there are 0 records. Is there a default value that Ataccama will show on the back end that I can check for on a rule? I also want to add that they stated they do not want to use anomaly detection due to the fact that we cannot be 100% guaranteed that it will catch no records being populated. Has anyone else faced this issue previously? Looking for any assistance here. Thanks!

Hello Community: Iam on version 14.5.1. WE have created a custom field for Catalog items to flag (a boolean field) whether the data is certified for consumption. I would like to update from a batch file to the few hundred catalog items. Is there a template anyone has - to update catalog items ? I have previously used a component for updating catalog attributes (like fields in a table), but here i need to update the catalog item itself (and not the attribute). appreciate anyone sharing a template of a component that i can adopt and use.

Hello, I'd like to set up a rule for the 'transaction_date' column to verify if the latest date in the column equals to previous work day. How can I do this? Is there any function for that? If there isn’t , we also have an additional calendar table where previous work day indicated for each date. But how can I indicate another table inside a rule? Thanks in advance.

Short question here: I'm creating a debug step that writes to a database table. In that I would like to put the name (or path?) of the component that was running. Is there some kind of system variable (similar to $ewf_workflow_id in workflows)?

Hello, I have a monitoring project, and inside, there is a table (operations). This table is populated by ETL daily, and every day the ETL job finishes at a different time. That's why I can't assign a specific schedule to this monitoring project. There is another table (job_status) that shows the condition of the ETL job. Inside that table, there is a column named 'Status of ETL JOB'. If the value in that column is 'Completed,' then it means the job has completed, and now the monitoring project can be run. How can I create such a schedule for that MP? I would be grateful if you can help me. Thanks in advance.

Hi everyone! We hope that you’ve found our Getting Started with Data Governance best practice series useful! This week we are continuing the Getting Started series with Data Observability 🔭 In this post, we will introduce you to the key components of Data Observability and guide you through the process of configuring it for your data source. Understanding Data Observability Data Observability expands on the existing data quality monitoring capabilities in Ataccama. It allows you to gain insights into the state of your data by monitoring data quality, detecting anomalies, and tracking schema changes. The key components of data observability are: Data quality monitoring: Regularly checks data against predefined data quality rules, providing results and trends based on business terms or domains. AI-powered anomaly detection: Utilizes AI to identify unpredictable data quality issues, such as sudden drops in data volumes or average values in specific columns. Data discovery: Monitors and t

Hello Community: Iam on version 14.5.1 and I would like to create a custom term type - in addition to the three types that are available out of box such as Business Terms, Key Performance Indicator and Security Terms. If a new type is NOT possible, i would like to rename one of them to meet my needs. Can someone throw some details on how to do that?

This may have a very simple solution. I have a workflow that runs a component with a Json Call that does an API call. If the API call succeeds the data is stored in a database table. No problem. But what to do if it fails? In the Json Call I've created a Response Code Column called jsoncall_response. And if jsoncall_response=200 that means it was successful. But what to do if it is not successful? I can use a text writer to write that data to file. Actually I would like to see the component and the workflow fail. But there isn't a "Break” or “Fail” step as far as I can see. The closest that I can find is the Stop Reading step. BTW I've tried Stop Reading, but it seems to ignore my condition. In the Jscon Call I put in a misformed url and in the debug it clearly gave a HTTP 405 response. That didn't seem to stop it from going further. In the SNOW response output file it even wrote that jsoncall_response was indeed 405. Alternatively I've been thinking about maybe sending notifications f

Currently I created a Monitoring Project for a team that has a Catalog Item included called: CI A. They want that particular CI, with applied DQ checks (thus the MP), to be scheduled every month. Few weeks later the team requested to add another Catalog Item called CI B. They want this CI, with applied DQ checks, to be scheduled every week. Few weeks later the team requested to add another Catalog Item called CI C. They want this CI, with applied DQ checks, to be scheduled every day. At this point I'm running the MP every day, meaning it's also running CI A and CI B which don't need to be run every day. I'm told I need to create a seperate MP specific for the schedule in order to seperate this. Having three or more different MP for one team can be very dissatisfying and takes up a lot of space when looking at an overview. Whereas if it's one MP with a builtin feature to have more schedules specific for CIs is where my ideal vision is at (in my opinion). Q: Is it possible to have one, s

Is there any integration between Ataccama and DBT to read the DQ rules that are defined in DBT?