Skip to main content

Data Quality & Catalog ⚙️

Discuss, learn, ask, suggest, share all things DQ&C 🔍

  • 788 Topics
  • 2,012 Replies
788 Topics
A
Ataccama Community AdminIntergalactic Expert
published in Data Quality & Catalog ⚙️
Getting Started with Data Observability: Advanced Configurations
Getting Started with Data Observability: Advanced ConfigurationsBest Practice

Hi everyone,In this post, we continue our discussion on Data Observability configurations today! Let's dive into some advanced settings, terms, and rules that will enhance your data observability experience. Observing Data with Selected TermsChoose your preferred method to select the data you want to observe. If you select Terms: Click on Add terms and select terms associated with important data from the available list. Once you've selected the terms, all associated catalog items will be chosen for monitoring. If necessary, you can exclude individual catalog items by selecting Edit selection. After choosing Edit selection, you will see all catalog items associated with the selected terms. Use the more options icon and select Exclude to exclude any of these items from monitoring. When you're ready, click Save. If you select AQL: Use Ataccama Query Language to define the data you want to observe. Once you've selected the terms, all associated catalog items will be chosen for

1990
A
A
Universe Traveller
Ajeesh_GUniverse Traveller
asked in Data Quality & Catalog ⚙️
send Outlook email from desktop application

How can I set-up email in workflow? I am using Microsoft 365 Please help

2901
shafeeqzaman
Ataccamer
A
Ataccama Community AdminIntergalactic Expert
published in Data Quality & Catalog ⚙️
Getting Started with ONE DG: Catalog Items 📒
Getting Started with ONE DG: Catalog Items 📒Best Practice

Hello everyone, and happy Wednesday! Today, we are continuing our Getting Started with ONE DG series with Catalog Items 📒 Catalog items are essentially records within the data catalog. To access your data profiles, simply click on "Catalog Items" located on the left sidebar. By default, Catalog Items are organized into different groups within their respective tabs:Anomaly Overview: This tab displays a list of all catalog items that have anomalies. Domain Suggestions: Here, you'll find a list of catalog items with suggested terms that you can accept or reject. Master Data: This tab contains a list of catalog items sourced from MDM (Master Data Management) data sources integrated with the Ataccama platform. Reference Data: In this tab, you can find a list of catalog items sourced from RDM (Reference Data Management) data sources integrated with the Ataccama platform. Not Monitored: This tab displays a list of catalog items that are currently not included in any monitoring projects. Repo

2642
A
A
Ataccama Community AdminIntergalactic Expert
published in Data Quality & Catalog ⚙️
Getting Started with ONE DG: Collaboration & Tasks 🤝Best Practice

Hi everyone! This week, we are continuing to explore the Ataccama ONE platform basics. Today, we are looking into Tasks and Collaboration. Collaborate with Your TeamCollaboration plays a vital role in any project, and ONE DG makes it simple for you. You can easily comment and initiate conversations with your team members by using the comment section. To notify someone specifically, just tag them using the "@" symbol. You can even add emojis and gifs to make your discussions more lively! Don't worry about missing any updates - you'll receive in-app notifications whenever someone tags you or responds to your comments. Streamline Your Workflow with TasksThe Task management system in the Ataccama ONE Platform helps you keep track of your work and ensures efficient progress management. The system organizes tasks into a Kanban Board, providing a visual representation of your workload. With tasks, you can perform the following actions:Create and manage tasks and their subtasks. Assign tasks t

1430
A
M
Data Voyager
msilvaData Voyager
asked in Data Quality & Catalog ⚙️
Configure runtimeConfig on DQ to auth via KeyCloak using the property truststore-password encrypter

Hi Everyone,      My DQ and KeyCloak is running integrated for authentication but the password in the attribute name=”truststore-password” is using password without encryption.    I tried to encrypt via command encrypt.bat on DQ  but  without success. I just have a success in the authentication using clear password.    Is there someone that have success to encrypt the value for the property truststore-password with keycloak? <config class="com.ataccama.server.keycloak.KeycloakDeploymentContributor">        <keycloakConfigs>            <keycloakConfig name="keycloak-admin-center.json">                <!-- Define common parameters for all clients. They can be overridden by client-specific settings.-->                <url>https://srvkeycloak:8446/auth</url>                <realm>ataccamaone</realm>                <attributes>                    <!--<attribute name="ssl-required" value="external"/> -->                    &lt

1172
A
Ataccamer
A
Ataccama Community AdminIntergalactic Expert
published in Data Quality & Catalog ⚙️
Getting Started with ONE DG: Business Terms 📑Best Practice

Hi all! Hope you are enjoying our getting started series 😎 In this post, we will cover Business Glossary and learn more about Business Terms. Business terms play a crucial role in analyzing data, as they provide the basis for evaluation and quality checks. You can establish expected values and gain deeper insights by assigning terms to specific data elements. The Business Glossary automatically maps these terms to real data sources, ensuring that your catalog is always up to date and synchronized with the latest assigned terms. Plus, an unlimited hierarchy structure lets you visualize the relationships among different data elements, systems, and processes. Let's explore the different types of terms you'll encounter in the Business Glossary:Business Terms: These are predefined labels that help you understand and organize your data. They are assigned based on set conditions. Custom-Created Terms: This term group allows you to create your own terms according to your specific needs. For e

3760
A
S
Data Pioneer
suryakanth.EmishaData Pioneer
asked in Data Quality & Catalog ⚙️
how to detect duplicate records in One Web?

Hello all, I have a task of detecting duplicate records from particular tables. So is there anyone who know about creating rule for this, a rule to detect duplicate values and mark it using One Web and One Desktop?

3506
S
Data Pioneer
M
Data Voyager
maneelloydsData Voyager
asked in Data Quality & Catalog ⚙️
Debugging and trace information

I have a workflow(.ewf file) which in turn calls several components and workflows. How do i run the workflow and get to know the trace of which all components and workflows are executed?Is there any property that can be enabled which will give the components and workflows that are getting executed? If there is some kind of information like input to each step and output from the step that also would be useful 

361
A
M
Data Voyager
maneelloydsData Voyager
asked in Data Quality & Catalog ⚙️
Remote executor and datamart

From the ataccama one url, click environment settings, create environment we have the below properties executor endpointadmin centerexecution mode then we have the option to add a datamart, with a jdbc string.  What exactly is the executor endpoint, admin center and execution mode?is the datamart a place(database) to store something from the executor(server)?Please provide some links which explains this.

463
A
A
Data Pioneer
aiwaliaData Pioneer
asked in Data Quality & Catalog ⚙️
Trying to connect a neo4j database to ONE Desktop

I need the jar files to create a neo4j database connection from ONE desktop.Can anyone please help?

461
M
Ataccamer
R
Data Voyager
rene.requenesData Voyager
asked in Data Quality & Catalog ⚙️
Ataccama ONE Desktop - SFTP wildcard

Hello,I want to use “SFTP Download File” and use a wildcard for the filename and a date.  Is that possible ?  For example :/<mydirectory>/input/MYFILE_<TODAY_YYMMDD>_*.txt When i enter an asterisk symbol in the  “File or Directory” section, I get an error. Thanks,Rene Requenes 

551
A
Ataccamer
shafeeqzaman
Ataccamer
shafeeqzamanAtaccamer
published in Data Quality & Catalog ⚙️
How to preserve spaces in DQC plans and DPE monitoring projects
How to preserve spaces in DQC plans and DPE monitoring projectsBest Practice

Hi there! Shafeeq from Ataccama here. I’m a Software Support Engineer and occasionally I come across issues that are shared among different customers. One of those issues is how one can get Ataccama tools to preserve spaces before and/or after certain strings. This might be when trying to perform certain operations using the Column Assigner in the IDE, or in a monitoring project running on the Web App where a DQ rule checks that a field does not have trailing spaces, etc.  The solution to this problem is simple, in the form of Java VM arguments. These  are typically values that change the behaviour of the Java Virtual Machine (JVM) which is used by our products. Below, I’ll share the solution.  When this issue arises for a plan, workflow, etc. in the IDE, you may follow the steps below:   Click on the arrow next to the run button and then click on "Run Configuration".   Click on the Runtimes tab and add the following parameter in the VM arguments text box:  -Dengine.trimStringValues=f

1350
shafeeqzaman
Ataccamer
O
Star Blazer L1
OjaswiniStar Blazer L1
posted in Data Quality & Catalog ⚙️
Does anyone have any document for How to read/ use OperSearch Discovery portal.

Can anyone provide a document or link on how to effectively navigate and interpret the logs in the OperSearch Discovery portal? Our client is using Ataccama PAAS. And sometimes we go to OperSearch Discovery for logs. I find challenging to understand. For instance, to locate runtime errors, I need to input "Kubernetes.container_name = runtime-server" in the filter section, followed by "error" in the search section. This will display any errors related to the runtime server. Otherwise, the log page appears to be quite intricate. Any help or link is very much appreciated.

460
O
Star Blazer L1
G
Data Voyager
GregersData Voyager
asked in Data Quality & Catalog ⚙️
"Null" value in Business rules

HiI’m Using the Windows DQ Analyzer - free version.I have used the business rules functionality, but I have been wondering what the idea behind the “null” value is? And what makes data fall into the “null” value?Br.Gregers

1061
anna.spakova
Ataccamer
P
Data Voyager
Pattern GroupsData Voyager
asked in Data Quality & Catalog ⚙️
CatalogItem Term Suggestion Metareader

Which ONE Metadata Reader entity type will let me read the all suggested business terms against all items?   

1688
anna.spakova
Ataccamer
C
Data Voyager
ckaushikData Voyager
asked in Data Quality & Catalog ⚙️
Temp Data Files Purging in Ataccama

DPM and DPE - Ataccama ONE Gen2 Platform LatestOnce DPE connects to a data source, it creates metadata and optional sampling of records that do not satisfy DQ rules.Could you please answer the following questions or direct me to a link that can answer the below questions?1. Does DPE create other temporary files as a result of processing as well?2. What happens to the metadata, sample records that do not satisfy DQ rules and any temporary files generated during processing after the connection to the data source get closed?3. How does purging of data works in Ataccama?

1241
anna.spakova
Ataccamer
M
Data Pioneer L2
MayKwokData Pioneer L2
published in Data Quality & Catalog ⚙️
Creating JSON String From Data
Creating JSON String From DataBest Practice

In Ataccama ONE Desktop, we have steps like JSON Reader, or JSON Parser, to read JSON data and put them into a data flow.We also have a JSON Call step that allows you to construct a JSON template to send JSON Calls outwards.But what if you wanted to transform the data into a JSON string but not necessarily send it anywhere? If that’s what you want to do, read on…Our dataLet’s take an example of this data:I have some fruits:id name 1 Apple 2 Orange 3 Pear 4 Grape  I have also some fruit variety names:id variety fruit_id 1 Gala 1 2 Golden Delicious 1 3 Jazz 1 4 Pink Lady 1 5 Navel 2 6 Blood 2 7 Seville 2 8 Conference 3 9 Comice 3  Join them together as our base data:(See json_1.comp in zip)Output looks like this: 💡 I want to now turn this into a JSON string. Variety should be an array within each fruit.To achieve this, we first need to aggregate the variety back into fruit, and then a final aggregation to get the whole set

4630
M
Data Pioneer L2
A
Data Pioneer
aiwaliaData Pioneer
asked in Data Quality & Catalog ⚙️
Monitoring Project Export Rule Names

What does the id mention in invalid_rules column in Monitoring Project Export refer to?the alphanumeric term within() after rule name as in the image belowIs this the ruleId of the rule? If not, what does this refer to?Is there a way I can configure Metadata Export step to give rule id?

27010
anna.spakova
Ataccamer
A
Data Pioneer
aiwaliaData Pioneer
asked in Data Quality & Catalog ⚙️
Divide results from Invalid Rule column in Monitoring Project Report Export into multiple columns

The idea is to divide the invalid_rules column as obtained in export for Monitoring Project into two columns- one containing the rule name and the other containing the explanation; and implement this for all the rules mentioned in a cell.For example, The below cell would be divided into invalid_rule_name column containing two records-validation [USA] State and calidation[Noth America] Phone Numbers, and invalid_rule_explanation column containing explanation for the respective rules.The challenge I am facing here is that the rule ID mentioned in Monitoring Project report export is not the same as the rule ID mentioned in the list of rules I export; which I am trying to use to identify the rule in invalid_rules column from Monitoring Project report.Please help me find a solution for this

1592
A
Data Pioneer
M
Data Voyager
manzuraData Voyager
posted in Data Quality & Catalog ⚙️
Manage Nodes Access, Access Level in v.14.2 (VS v.13.9)

​Hello everyone! Foreword and problem statementThere are several subcategories of terms. I need to set up access for manager and steward of each subcategory so that:The steward of certain subcategory is able to create new terms, edit old ones in his subcategory only. However, he can not publish any changes The manager, in addition to above, can publish changes, but also in his own subcategory only.I have the list of questions concerning access management in v.14.2:Is it possible to grant ownership to node? And if it’s yes, how can I implement it? Do you know any way to solve my manager-steward task? (there are my understanding of the solution below) In States of approval workflowwhat are the “User roles” : governance or identity provider ones? (looks like the second ones, but doesn’t work) what is the “OWNER” for this version of workflow?_______________________There is a long prelude to where the questions came from. Version: 13.9There is the ability to create a special capability to b

1042
M
Data Voyager
S
Data Voyager
SathyaData Voyager
published in Data Quality & Catalog ⚙️
Adding New Property to Catalog Item in MMM ModelBest Practice

METADATA MODEL The metadata model is used to configure the application entities and their properties. The Metadata Model describes the entities and their properties in the application.  Web Application) use the Metadata Model to create the frontend user interface. The metadata model is also responsible for the configuration of the left navigation menu of the application. ONE METADATA PROPERTY REFERENCESEntities consist of properties of Simple types, such as strings and integers, and Object types, which link the entity to other entities. ADDING NEW PROPERTY TO THE MMM MODELGo to left panel of the Webapp and choose Global settings   Choose metadata model under Maintenance CenterChoose the Custom filed where you need to add the new property to CatalogItem, Glossary and sources. In our case we adding to CatalogItem   Click on the Custom field CatalogItem. We can see General Information and Property. Click add property and type in the property to be added.   Fill in the name and choose the

1440
S
Data Voyager
E
Data Voyager
Evgeny ChernoskutovData Voyager
asked in Data Quality & Catalog ⚙️
How to increase the speed of profiling data for using Oracle DB?

Hello, I have a speed of profiling data for using Oracle DB of about 4000-7000 rows per minute.I use Ataccama DQ Analyzer (Version: 11.1.1.ga-2017-06-14).  Please, can you take me to advise me on how to increase the speed of profiling data for using Oracle DB?I hope to take the profiling speed to about 100 000 rows per minute...  Connation config:   Log:17.03.2023 12:56:04 [INFO] Switching to logging setup from C:\Users\ECHERN~1\AppData\Local\Temp\loggingConfig3307204922660014311.tmp17.03.2023 12:56:04 [INFO] Using following licenses:17.03.2023 12:56:04 [INFO] C:\Ataccama DQ Analyzer 11\runtime\license_keys\license.plf17.03.2023 12:56:04 [INFO] Using DQ Analyzer configuration file C:\Ataccama DQ Analyzer 11\workspace\sqb_prof_pilot\data\out\sqb_all_table.plan17.03.2023 12:56:04 [INFO] Using runtime configuration file C:\Users\echernoskutov\AppData\Local\Temp\runtimeCfg1065525328424309714.tmp17.03.2023 12:56:04 [INFO] Changing CPU affinity to 4 CPU(s).17.03.202

1422
E
Data Voyager
M
Star Blazer L2
Marnix WisselaarStar Blazer L2
published in Data Quality & Catalog ⚙️
Catch OneWeb GraphQL in Chrome/Edge test and useBest Practice

We are using some GraphQL to ‘do stuff’. Turns out the easiest way to find the GraphQL is using the Developer-tools in Chrome or Edge. See enclosed. 

1403
M
Star Blazer L2
anna.spakova
Ataccamer
anna.spakovaAtaccamer
published in Data Quality & Catalog ⚙️
Export metadata model into Excel
Export metadata model into ExcelBest Practice

Hello everyone, Anna here! I work as a projects consultant in Ataccama and today I put together a guide for you on how to export a metadata model definition from Ataccama ONE.Ataccama ONE allows to export metadata model definition from the platform into a JSON file. However, the JSON file is not an ideal format for business users. For that reason, parsing of the JSON into more user-friendly format, for instance Excel, is desired. The current JSON parser step in Ataccama DQC, however, is not able to parse this JSON structure.This article introduces a Python script that was developed to parse the JSON structure of the metadata export and provide the metadata model in Excel and CSV formats. It is supported for v13 of Ataccama ONE.As a first step, the metadata model has to be exported from Ataccama. Next step is to run the Python script on top of the exported metadata model to obtain the CSV and Excel format. Chapter JSON parser in Python describes in detail the Python script as well as th

3420
anna.spakova
Ataccamer
S
Data Voyager
smunotData Voyager
asked in Data Quality & Catalog ⚙️
MCQs in Certification

What kind of MCQs are asked in Certification exam?

1111
DannyRyan
Ataccamer

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings