Skip to main content

Product Updates
Events
Training Resources
Groups

Home
Learn & Share
Data Quality & Catalog ⚙️

Data Quality & Catalog ⚙️

Discuss, learn, ask, suggest, share all things DQ&C 🔍

555 Topics
1,356 Replies

When you subscribe we will email you when there is a new topic in this category

555 Topics

olayinkadaramolaData Voyager

asked in Data Quality & Catalog ⚙️

Hello, I am trying to create a rule for validity on Names. The logic and condition is that the rule matches names that are Title Case --first letter upper case follow by all lower case Accented Accepts special characters Mandatory Could have Hyphen - I have this regex ^[a-zA-Z '-]+$ ^[A-Z][a-z]*(?:[ '-][A-Z][a-z]*)*$(?<=^|\s)[a-z]*[A-Z][a-z]*[A-Z]*[a-z]* above regex capture and validate names like John DoeMary Anne which are okay. but also validate name in all lower case for example john doe which is not okay. i want to know what i need to do to allow the regex not validate names in all lower case .

srija piratlaAtaccamer

published in Data Quality & Catalog ⚙️

Profiling Triggered While Running Monitoring Project

Hello everyone ! I’m Srija ,Solutions Consultant here at Ataccama. Today, I’d like to talk about how to trigger profiling while running a monitoring project.While running the monitoring project, the profiling process is getting triggered without any schedule or event created on the catalog item to initiate profiling. This unexpected profiling of over 1000+ records significantly slows down the system. The likely cause for this issue is the anomaly detection feature being enabled on the catalog item.To address this issue, you need to check if anomaly detection is enabled on the catalog item. Anomaly Detection in Ataccama DQWhat Anomaly Detection Does:Anomaly detection in Ataccama Data Quality (DQ) automatically identifies outliers and unusual patterns in your data. This feature is designed to ensure data quality by detecting anomalies such as unexpected values, missing data, or patterns that do not conform to the expected behavior.Implications of Enabling Anomaly Detection:Automatic

SathyaAtaccamer

published in Data Quality & Catalog ⚙️

Adding Data Domain Entity to the MMM Model

Hi everyone, I'm Sathya, Solutions Consultant at Ataccama. In this post I am going to talk about creating Data Domain , relationship using AQL expression. Step 1: On the organization tab, go to Global settings Step 2: Click Navigation Icon Step 3: List of entities will be listed, under Business glossary we will have Data Domains Step 4: Click on the 3 dots and click Add Child option to add a child to the list. Step 5: Pop up window appears, fill in the details as follows:Navigation item type: Listing Listing Node: Term Icon: Folders Text Medium Name: Fill the name of the Data Domain Save and publish. Step 6: All the terms will be added to the Data domain. To filter out we have to update the AQL expression. Step 7:Click on the 3 dots on the right corner of the terms page where the data domain is displayed and choose Edit page. Step 8: In the Filter part we must fill in the AQL Expression Step 9: Syntax for AQL expression:name is "(name of the dataDomain)" OR @termRelationship(target)

C_RUniverse Traveller

asked in Data Quality & Catalog ⚙️

How to rename Post Processing Plans

Hi all, I have a group of PPP that need to be renamed. Is it possible to update name once published? How? Thanks

NazrinDData Voyager

asked in Data Quality & Catalog ⚙️

combine email with ataccama MP results

I want to configure one general email notification that will be used for all of my monitoring programs. I would prefer to receive a single email notification that compiles the findings from various monitoring projects rather than the numerous emails I usually receive for each monitoring project. I'm aware that I should use One Desktop, but I'm not sure how to proceed precisely. Well, I know that I have to call monitoringProjectProcessing which allows me to retreive the finish time with processing ID. And then I have to complie to get the ID link of notification but the problem is here that I dont know the step exactly. It would be great if you could help.Thanks in advance.

sumeyraData Pioneer

asked in Data Quality & Catalog ⚙️

Writing data to One Data from One Desktop

Hello all 🤓I would like to write my post processing data (export file of my DQ results) to One-Data (database) from One Desktop. Since one-data is not jdbc, I couldn’t use JDBC writer. It shows red icon when I want to try to use this step. Can anyone recommend what can I do here? Thanks !

Kelvin FongData Voyager

asked in Data Quality & Catalog ⚙️

(How) can we display the properties of the table attributes in the catalog item tabs?

Let’s say we have some properties placed at the table attribute level. Is it possible to display those table attribute properties in the data structure page of the catalog item as shown below? Thanks in advance.

PrithikaData Voyager

asked in Data Quality & Catalog ⚙️

Export plan Shows No Records

Hello, I’m trying to figure out why the export plan section in Ataccama ONE desktop keeps showing “No Records Found” although I have created a list of export plans. Once I create an export plan, I save and publish it, but it does not show up as part of the export plan section instead it says, “JSON error”. Does anyone have any idea on how to fix this issue ?Thank you,Prithika

asked in Data Quality & Catalog ⚙️

ONE Expression - Trim

Hi,There are certain records that contain “white spaces” at the beginning and/or at the end. Ex: “ 100137069 “, “ 100017129 “, etc. We don't allow the use of white spaces and want to build a DQ rule to check the invalid records. Hence the advanced expression: length(String_ATTRIBUTE_1) <> length(trim(String_ATTRIBUTE_1)). When I test the DQ rule it works as intended. I copy the above results and paste them in the test environment and it shows me the record is invalid. When I remove the white spaces it will considered valid. However, when I run the Monitoring Project based on the same records and DQ rule it gives me 100% validity. How is this possible? And maybe more importantly, how can I create a rule that will show the invalid records shown earlier?

Albert de RuiterStar Blazer L2

asked in Data Quality & Catalog ⚙️

Custom relationship

Hi Community,I have used this community tutorial (thanks @anna.spakova !) and this part of the documentation portal to create a custom relationship (and an existing custom relationship as example). So that should be enough info, but still I encounter an issue. As follows.The relationship is between an extension of term (entity term) and custom entity application. The new relationship has been defined in entity entityTermSourceOfRecordRel as followsThis new entity has also been defined in entity metadata as array.A new relationship type has been defined for this purpose:In the entity term screen I have defined the realtionship as follows:{"_type": "relationships","relationshipPropertyPath": "/entityTermSourceOfRecordRel"}On the entity term screen I would now expect to be able a new relationship like this (from the example we already had implemented):Instead I only have the option to define a new relationship type, like:What am I overlooking? We are on version 13.9.Thanks and kind regard

thomas.prociukData Voyager

asked in Data Quality & Catalog ⚙️

Datastage jobs and flows Connectivity

Hey Everyone!We have a use case involving datastage and curious if anyone has worked with this in the past. We have a team who has datastage jobs that are triggered by a table being pushed information downstream. We are wondering if there is a way either in the web or desktop, if we can use the trigger files that are created by datastage, somehow plug into the datastage flow, or another way that would trigger a monitoring project simultaneously to run data quality checks against the load? if certain data is not loaded in correctly, it will break the connection for the two tables, and in turn break the job, and they want to see the population of what is breaking the join. Hope this makes sense! Thanks,Thomas

apejkoUniverse Traveller

asked in Data Quality & Catalog ⚙️

Triggering Review Workflow From One Desktop

Hi everyone! I’m wondering if its possible to trigger or use the Review workflow from one desktop? For instance, im currently updating a large amount of attribute descriptions and am publishing them with one desktop. Whenever I use the One Metadata writer it will apply the descriptions automatically without further approval.I would like to know if its possible to write the descriptions and then have them sent to the tasks board for the data owner to review and approve or deny.Example of what im trying to do:Use one desktop to write descriptions to attributes Review workflow is triggered, and new descriptions are assigned to data owner for review (preferably in batches of 15-20 if possible) Owner either approves changes, causing descriptions to be updated or denies them causing no change to descriptionsThank you for any help!

srija piratlaAtaccamer

published in Data Quality & Catalog ⚙️

How to extract the description/comments from all the Ataccama tables?

Hello everyone!In this post, I’ll walk you through how to extract the description of all the columns and tables using the one desktop plan. Providing detailed descriptions and comments is a fundamental best practice in data management and data standards. This approach significantly enhances theunderstanding and usability of data.The summary of all the tables, attributes and their descriptions can be extracted using the below plan.1. Open a plan and add Attribute and Catalog Item metadata reader steps.2. The Attribute meta data reader helps to extract the attribute names, comments, description. Open the meta data reader and provide the column names as below and provide the Parent Id Column name as ‘CatalogItemID’.3. The Catalog Items meta data reader helps to extract the catalog Item name based on the catalog Item Identifier. Open the meta data reader and provide the column names as below and provide the Id column name as ‘catalogItemID’.4. Use the Join Step and join the meta data reade

srija piratlaAtaccamer

published in Data Quality & Catalog ⚙️

How to ensure post processing plan streamlining 📚

Hi everyone! In this post, I’m going to walk you through how to ensure that changes made in the post-processing plan are visible across different user accounts.Issue DescriptionIn version 14.5 of ONE Desktop, there is an issue where if you create a post-processing plan for a monitoring project from one account, the changes are not reflected when another user logs into the same environment.SolutionTo fix this issue, you need to download the remote file every time before making any changes to the Post Processing Component to ensure you are editing the latest file.Steps to Fix the Issue1. Login to ONE Desktop:Connect to your server by navigating to Server -> New -> ONE Platform2. Create a Post-Processing Plan:Go to the Monitoring Project -> New -> Post-Processing Plan. 3. Download the Latest Changes:To get the latest changes from one account:Right-click on the Component. Select Download. Choose Download Remote File By downloading the remote file, you will get the latest c

KarthikeyanData Pioneer

asked in Data Quality & Catalog ⚙️

row count in a column

Hi Team, I am working on task to check the number of rows in a column. I am using count of ATTRIBUTE_1 but not able to achieve it . can you please suggest a better approach. let me know if i miss anything.

clyormzUniverse Traveller

asked in Data Quality & Catalog ⚙️

ataccama java heap memory configuration

Hello Ataccama team Checking ataccama admin center on resource section there is mentioned the JVM Heap Memory,I would like to know more about the Ataccama JVM Heap Memory configuration , as I understand Ataccama manage Java APIs (mmm comment, dpe,...) , I am curious to know if Ataccama has a configuration file to define the limits of JAVA memory to use on each ataccama api, or if Ataccama has a configuration file to manage the whole java resources.

srija piratlaAtaccamer

published in Data Quality & Catalog ⚙️

Working with API to transform business terms and rules stewardship

Hi I'm Srija, Solutions Consultant here at Ataccama.In this post, we can see a workaround for changing the stewardship for out-of-the-box rules and terms along with the project related business terms and rules. This workaround helps to transfer the stewardship of business rules and terms to ‘Organization’ without causing the API error.How to perform the transfer of stewardship?ONE web application By creating a custom plan to change stewardship for multiple business terms and rules. How to create a custom plan to bulk change stewardship for business terms and rules?In ONE desktop, you can use metadata reader steps from term or rule, and you can join with group metadata reader using stewardship name and name. In the JSON step, use the below GraphQL query to change the stewardship for terms or rules.{"query":"mutation TransferStewardship_rule($gid: GID!, $newStewardId: GID, $accessLevelForPreviousSteward: String) { ruleTransferStewardship( gid: $gid newStewardId: $newStewardId

KarthikeyanData Pioneer

asked in Data Quality & Catalog ⚙️

Compare two different columns from two different files

I am working on the usecase of comparing two different columns in two different csv files. I need to compare these column, remove any duplicates and merge them into single column. I am trying to use join component but not sure about this approach will work or any better solution is available to achieve this use case - If anyone can suggest a better apprach it will be helpful. - Thanks.

anna.spakovaAtaccamer

published in Data Quality & Catalog ⚙️

Integrating git with orchestration server

Hi everyone, me again 😊 I work as a professional services consultant in Ataccama and I prepared another article for you, this time about some recommendations for integration with git, mainly for self-managed deployments. Specifically, when you have the DQG suite and you want to integrate your orchestration server. You can probably reuse some of these recommendations for other modules as well.Before you start reading, please note that I reused some of the already existing articles:https://community.ataccama.com/master-data-management-reference-data-management-92/one-desktop-git-recommendations-187?tid=187&fid=92I also recommend checking out this article in our documentation for working with git in our IDE:https://docs.ataccama.com/one-desktop/latest/projects/git.html#introduction-to-git-and-egitAlso please note, that this article doesn’t cover how to work with git in general, but more on how to work with the Ataccama projects, what files to version, what to look out for etc.Recomme

Prasad RaniData Pioneer

asked in Data Quality & Catalog ⚙️

Can i get notification on term assignments

I have certain Business terms that i want to track on which fields are getting mapped from which datasets on a given day.Is there a way, i can send a notification to myself (and any others interested) on the occurance details of a term - and i schedule this to run periodically like every monday? To be specific, Iam want to track Social Security Number. I want to get a report every monday morning - with list of Catalog items and attributes that are mapped to this term. (from the occurances details). Is it possible?? How??

Vanja GilbertUniverse Traveller

asked in Data Quality & Catalog ⚙️

Possible to create filters to rules?

Hello! I am looking for assistance with applying filters to rules. Currently, I understand that the "where" clause can be used in rules, but it seems to apply to all records uniformly. What I need is a way to create a rule that only affects a subset of the data, so that the total count of records affected by the rule is different from the total count of records in the Catalog Item. For example, if I have a dataset with 1000 records, I want to create a rule that only applies to, say, 200 of those records based on specific criteria, without affecting the other 800 records. So if there is 100 records failing the result should be 100/200 instead of 100/1000.Is there a way to achieve this selective application of rules? Any guidance or examples on how to set this up would be greatly appreciated. Thank you!

olayinkadaramolaData Voyager

asked in Data Quality & Catalog ⚙️

is there a way to change the date format in ataccama one. i want the format to be dd mm yyyy, what i have currently is mm dd yyyy

Ajeesh_GUniverse Traveller

asked in Data Quality & Catalog ⚙️

How to run .vbs file in workflow

How can I run a .vbs file workflow. May be either ‘Run Shell Script’ or ‘Run Windows Command’. Or is there any other options available? I tied to run this through ‘Run Shell Script’ but it showing an issue with Interpreter(/bin/sh)

Yasharth MisraData Voyager

asked in Data Quality & Catalog ⚙️

how to find duplicates in ONE Desktop

Hello can you please assist in finding out how to find the duplicates in a column using one desktop.

Prasad RaniData Pioneer

posted in Data Quality & Catalog ⚙️

AI enabled term detection is giving significant false positives. How can i tune it

For terms like USA Zipcode, USA Social Security Number etc., - the term is enabled with AI detection and it is getting tagged to hundreds of fields - most of them are NOT zipcode or SSN. If it was giving a score and recommendation, I would have discarded the recommendation, but it is assigning the term automatically and I have to go in and remove that - which became painful. I have now disabled the AI detection but is there a better way to tune it to make it work better?

1
...
3
4
5
6
7
8
9
...
23

Page 6 / 23

Badge winners

clyormzhas earned the badge First idea 💡
Srimanhas earned the badge First idea 💡
Edwin van Aschhas earned the badge First idea 💡
ChrisBowmanhas earned the badge First idea 💡
NicolasDrolethas earned the badge First idea 💡

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more

Powered by Gainsight

Community Terms of Use & Conditions Cookie settings

Sign up

Already have an account? Login

For Ataccama Customers and Partners

or

Username *

E-mail address *

Company *

Job Title *

A bit about yourself

How did you hear about the Data People Community? *

Ataccama version

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the Ataccama Community

No account yet? Create an account

For Ataccama Customers and Partners

or

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK