Discuss, learn, ask, suggest, share all things DQ&C 🔍
Recently active
Happy Wednesday community ☀️ Welcome back to our Data Quality series! In today's post, we'll delve deeper into configuring detection and validation rules in Ataccama ONE. If you missed our previous post, we recommend checking it out before continuing.Configure Detection LogicTo define your detection rule's logic, navigate to the "Implementation" tab when creating or editing a rule. If you're starting from scratch, select "Detection" as the rule logic. You can add multiple conditions using the Condition Builder, with each record needing to fulfill all conditions by default (using the logical operator AND). For less strict rules with the logical operator OR, try the Advanced Expression mode. Take advantage of Pattern Analysis results to create detection rules quickly. For instance, open the "Profile & DQ Insights" tab for the catalog item "customers," select the "email" attribute, and utilize the detected patterns by choosing "Use results in rule." Save your changes and don't forget
Hi,Situation: I wrote a(n) SQL catalog item joining multiple tables and in Ataccama I'm using rules to compare the values if they are the same. LONG = LONG, check if the nr_loan = the same a.nr_loan, etc.My question is if there's an easier way to see if two tables are identical. I'm doing this to check the integrity.I couldn't find anything relatable asked in the community.
Hello all, is there anyone who knows about creating a rule to detect the duplicate records of a particular attribute and mark it as duplicate using one web?
How do you extract the catalog item with their associated terms (final) in a report?
Hi everyone,In this post, we continue our discussion on Data Observability configurations today! Let's dive into some advanced settings, terms, and rules that will enhance your data observability experience. Observing Data with Selected TermsChoose your preferred method to select the data you want to observe. If you select Terms: Click on Add terms and select terms associated with important data from the available list. Once you've selected the terms, all associated catalog items will be chosen for monitoring. If necessary, you can exclude individual catalog items by selecting Edit selection. After choosing Edit selection, you will see all catalog items associated with the selected terms. Use the more options icon and select Exclude to exclude any of these items from monitoring. When you're ready, click Save. If you select AQL: Use Ataccama Query Language to define the data you want to observe. Once you've selected the terms, all associated catalog items will be chosen for
How can I set-up email in workflow? I am using Microsoft 365 Please help
Hello everyone, and happy Wednesday! Today, we are continuing our Getting Started with ONE DG series with Catalog Items 📒 Catalog items are essentially records within the data catalog. To access your data profiles, simply click on "Catalog Items" located on the left sidebar. By default, Catalog Items are organized into different groups within their respective tabs:Anomaly Overview: This tab displays a list of all catalog items that have anomalies. Domain Suggestions: Here, you'll find a list of catalog items with suggested terms that you can accept or reject. Master Data: This tab contains a list of catalog items sourced from MDM (Master Data Management) data sources integrated with the Ataccama platform. Reference Data: In this tab, you can find a list of catalog items sourced from RDM (Reference Data Management) data sources integrated with the Ataccama platform. Not Monitored: This tab displays a list of catalog items that are currently not included in any monitoring projects. Repo
Hi everyone! This week, we are continuing to explore the Ataccama ONE platform basics. Today, we are looking into Tasks and Collaboration. Collaborate with Your TeamCollaboration plays a vital role in any project, and ONE DG makes it simple for you. You can easily comment and initiate conversations with your team members by using the comment section. To notify someone specifically, just tag them using the "@" symbol. You can even add emojis and gifs to make your discussions more lively! Don't worry about missing any updates - you'll receive in-app notifications whenever someone tags you or responds to your comments. Streamline Your Workflow with TasksThe Task management system in the Ataccama ONE Platform helps you keep track of your work and ensures efficient progress management. The system organizes tasks into a Kanban Board, providing a visual representation of your workload. With tasks, you can perform the following actions:Create and manage tasks and their subtasks. Assign tasks t
Hi Everyone, My DQ and KeyCloak is running integrated for authentication but the password in the attribute name=”truststore-password” is using password without encryption. I tried to encrypt via command encrypt.bat on DQ but without success. I just have a success in the authentication using clear password. Is there someone that have success to encrypt the value for the property truststore-password with keycloak? <config class="com.ataccama.server.keycloak.KeycloakDeploymentContributor"> <keycloakConfigs> <keycloakConfig name="keycloak-admin-center.json"> <!-- Define common parameters for all clients. They can be overridden by client-specific settings.--> <url>https://srvkeycloak:8446/auth</url> <realm>ataccamaone</realm> <attributes> <!--<attribute name="ssl-required" value="external"/> --> <
Hi all! Hope you are enjoying our getting started series 😎 In this post, we will cover Business Glossary and learn more about Business Terms. Business terms play a crucial role in analyzing data, as they provide the basis for evaluation and quality checks. You can establish expected values and gain deeper insights by assigning terms to specific data elements. The Business Glossary automatically maps these terms to real data sources, ensuring that your catalog is always up to date and synchronized with the latest assigned terms. Plus, an unlimited hierarchy structure lets you visualize the relationships among different data elements, systems, and processes. Let's explore the different types of terms you'll encounter in the Business Glossary:Business Terms: These are predefined labels that help you understand and organize your data. They are assigned based on set conditions. Custom-Created Terms: This term group allows you to create your own terms according to your specific needs. For e
Hello all, I have a task of detecting duplicate records from particular tables. So is there anyone who know about creating rule for this, a rule to detect duplicate values and mark it using One Web and One Desktop?
I have a workflow(.ewf file) which in turn calls several components and workflows. How do i run the workflow and get to know the trace of which all components and workflows are executed?Is there any property that can be enabled which will give the components and workflows that are getting executed? If there is some kind of information like input to each step and output from the step that also would be useful
From the ataccama one url, click environment settings, create environment we have the below properties executor endpointadmin centerexecution mode then we have the option to add a datamart, with a jdbc string. What exactly is the executor endpoint, admin center and execution mode?is the datamart a place(database) to store something from the executor(server)?Please provide some links which explains this.
I need the jar files to create a neo4j database connection from ONE desktop.Can anyone please help?
Hello,I want to use “SFTP Download File” and use a wildcard for the filename and a date. Is that possible ? For example :/<mydirectory>/input/MYFILE_<TODAY_YYMMDD>_*.txt When i enter an asterisk symbol in the “File or Directory” section, I get an error. Thanks,Rene Requenes
Hi there! Shafeeq from Ataccama here. I’m a Software Support Engineer and occasionally I come across issues that are shared among different customers. One of those issues is how one can get Ataccama tools to preserve spaces before and/or after certain strings. This might be when trying to perform certain operations using the Column Assigner in the IDE, or in a monitoring project running on the Web App where a DQ rule checks that a field does not have trailing spaces, etc. The solution to this problem is simple, in the form of Java VM arguments. These are typically values that change the behaviour of the Java Virtual Machine (JVM) which is used by our products. Below, I’ll share the solution. When this issue arises for a plan, workflow, etc. in the IDE, you may follow the steps below: Click on the arrow next to the run button and then click on "Run Configuration". Click on the Runtimes tab and add the following parameter in the VM arguments text box: -Dengine.trimStringValues=f
Can anyone provide a document or link on how to effectively navigate and interpret the logs in the OperSearch Discovery portal? Our client is using Ataccama PAAS. And sometimes we go to OperSearch Discovery for logs. I find challenging to understand. For instance, to locate runtime errors, I need to input "Kubernetes.container_name = runtime-server" in the filter section, followed by "error" in the search section. This will display any errors related to the runtime server. Otherwise, the log page appears to be quite intricate. Any help or link is very much appreciated.
HiI’m Using the Windows DQ Analyzer - free version.I have used the business rules functionality, but I have been wondering what the idea behind the “null” value is? And what makes data fall into the “null” value?Br.Gregers
Which ONE Metadata Reader entity type will let me read the all suggested business terms against all items?
DPM and DPE - Ataccama ONE Gen2 Platform LatestOnce DPE connects to a data source, it creates metadata and optional sampling of records that do not satisfy DQ rules.Could you please answer the following questions or direct me to a link that can answer the below questions?1. Does DPE create other temporary files as a result of processing as well?2. What happens to the metadata, sample records that do not satisfy DQ rules and any temporary files generated during processing after the connection to the data source get closed?3. How does purging of data works in Ataccama?
In Ataccama ONE Desktop, we have steps like JSON Reader, or JSON Parser, to read JSON data and put them into a data flow.We also have a JSON Call step that allows you to construct a JSON template to send JSON Calls outwards.But what if you wanted to transform the data into a JSON string but not necessarily send it anywhere? If that’s what you want to do, read on…Our dataLet’s take an example of this data:I have some fruits:id name 1 Apple 2 Orange 3 Pear 4 Grape I have also some fruit variety names:id variety fruit_id 1 Gala 1 2 Golden Delicious 1 3 Jazz 1 4 Pink Lady 1 5 Navel 2 6 Blood 2 7 Seville 2 8 Conference 3 9 Comice 3 Join them together as our base data:(See json_1.comp in zip)Output looks like this: 💡 I want to now turn this into a JSON string. Variety should be an array within each fruit.To achieve this, we first need to aggregate the variety back into fruit, and then a final aggregation to get the whole set
What does the id mention in invalid_rules column in Monitoring Project Export refer to?the alphanumeric term within() after rule name as in the image belowIs this the ruleId of the rule? If not, what does this refer to?Is there a way I can configure Metadata Export step to give rule id?
The idea is to divide the invalid_rules column as obtained in export for Monitoring Project into two columns- one containing the rule name and the other containing the explanation; and implement this for all the rules mentioned in a cell.For example, The below cell would be divided into invalid_rule_name column containing two records-validation [USA] State and calidation[Noth America] Phone Numbers, and invalid_rule_explanation column containing explanation for the respective rules.The challenge I am facing here is that the rule ID mentioned in Monitoring Project report export is not the same as the rule ID mentioned in the list of rules I export; which I am trying to use to identify the rule in invalid_rules column from Monitoring Project report.Please help me find a solution for this
Hello everyone! Foreword and problem statementThere are several subcategories of terms. I need to set up access for manager and steward of each subcategory so that:The steward of certain subcategory is able to create new terms, edit old ones in his subcategory only. However, he can not publish any changes The manager, in addition to above, can publish changes, but also in his own subcategory only.I have the list of questions concerning access management in v.14.2:Is it possible to grant ownership to node? And if it’s yes, how can I implement it? Do you know any way to solve my manager-steward task? (there are my understanding of the solution below) In States of approval workflowwhat are the “User roles” : governance or identity provider ones? (looks like the second ones, but doesn’t work) what is the “OWNER” for this version of workflow?_______________________There is a long prelude to where the questions came from. Version: 13.9There is the ability to create a special capability to b
METADATA MODEL The metadata model is used to configure the application entities and their properties. The Metadata Model describes the entities and their properties in the application. Web Application) use the Metadata Model to create the frontend user interface. The metadata model is also responsible for the configuration of the left navigation menu of the application. ONE METADATA PROPERTY REFERENCESEntities consist of properties of Simple types, such as strings and integers, and Object types, which link the entity to other entities. ADDING NEW PROPERTY TO THE MMM MODELGo to left panel of the Webapp and choose Global settings Choose metadata model under Maintenance CenterChoose the Custom filed where you need to add the new property to CatalogItem, Glossary and sources. In our case we adding to CatalogItem Click on the Custom field CatalogItem. We can see General Information and Property. Click add property and type in the property to be added. Fill in the name and choose the
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.