Data Quality &amp; Catalog ⚙️

PrithikaData Voyager

Masking Sensitive Attributes

Hello, Is it possible to mask attributes that are classified as restricted/confidential using Ataccama One Web? I was able to assign rules and terms to attributes, but I was not able to create/apply any masking methods. Please let me know if anyone has tried this before. Thanks in advance!

2494

SoniyaData Voyager

Regarding Completeness DQ control

Hi Team,I am trying to find how we can achieve completeness DQ control by comparing data in the file and the table irrespective of the position of primary key in either of them. Also, need to know how we can achieve the same by using the composite keys.

1472

Vanja GilbertUniverse Traveller

Invalid samples for sections

Hi,I have a question about the display of invalid samples when a specific section is selected. It's helpful that the invalid samples are based on that section's checks, but I'm wondering why all attributes are shown by default.Wouldn't it make sense to show only the attributes relevant to the failed rules, with an option to add more if needed? The current setup is a bit confusing.Thank you!

2004

Albert de RuiterRocket Pioneer L1

Relationship type visualization / Graph visualization

Hi community,Does anyone know about documentation about visualizing relationship types? On the documantation portal I found this, but that doesn't seem to cover the setup in the Graph Visualization section in the Application Settings. The documentation mentions “For more information, see Configure Graph Visualization.”, but that is a link to the page you are already on.Kind regards,Albert

2924

Maria GolikovaData Pioneer

Renaming Monitoring projects and DQ rules

Hello Dears! Would someone advise if there any watch outs once I want to rename Monitoring projects and DQ rules on ONE Web?I have running projects and DQ rules naming convention of which I would like to improve. If I just simply change names, won’t it have negative impact anywhere?Thank you. Kind regards,Maria

1823

Antonio DeChausayAtaccamer

Creating Rule Folders in ONE Web

Hi Community I’m Antonio, a Professional Services Consultant here at Ataccama. Assets created in ONE do not come with the option of being sorted into folders out of the box. This guide will demonstrate how to set up folders for rules, but can be used for other asset types and isn’t specific to rules. In this guide we will describe how to set up a custom attribute ‘ruleFolderValueList’ on rules which will hold data from a list of values, then create new navigation sections in the tool where we can apply AQL expressions to filter rules based on our set criteria. This will allow us to split rules into folders based on the value that has been entered into the custom attribute. The process and results outlined have only been tested in v13.9.3 and v14.5.0 of the ONE Web application. Create Value ListFirst we will create the list of values. To do this follow the below stepsGo to Global Settings > Metadata Model. Select Create entity.Provide the following information:Name of the value that

4203

AdilhussainData Voyager

How to Parameterize input to DQ rule?

I would like to setup a generic data quality rule to check string length. Requirements for length checks differs per attribute. Say column1 needs to have max length of 10 column2 needs to have max length of 15 and so onIs it possible to create a generic rule to check string length, passing value of length as parameter?I am aware of attribute length insights available as part of DQ profiling results but I would like to make the check as part of monitoring project. Thanks!

2782

RadziahData Voyager

Summary of DQ Checks and Results in Ataccama ONE Desktop

HiI would like to get a summary of DQ checks and results like the one we have in ONE web like how many in % and count of broken rules, how many are passing and how many records do we have in total etc. in Ataccama ONE desktop. I tried looking out any related topic in the community but I could not find one. Is it possible to achieve this via Ataccama ONE desktop as we have an existing plan comparing data fields in two different sources. Do you mind sharing the suggested steps to achieve this? Thanks in advance!

37610

KaterinaAtaccamer

Customizing DQ Monitoring Projects Results Export

When it comes to data quality (DQ) management, monitoring projects provide us with ability to export DQ results and maintain a close eye on data quality. However, the initial export contains attributes organized in columns, alongside additional columns housing information regarding the DQ results for these attributes. Analysing this data can be challenging because DQ rule results and explanations are presented as a single string.Despite the fact, that extracting DQ results in a customized format might require a more detailed approach, it might be better aligning with the client's specific needs. In this guide, you will find information how to export DQ results tailored to specific requirements.The Task at Hand:Our objective is to ensure a separate line for each attribute in an export file, with the following columns related to DQ results:the name of the DQ rule; an explanation for the check; a list of terms assigned to that attribute; the attribute value.We have three main steps:Attrib

7211

SathyaData Voyager

Creating New Entity Data Classification And Adding New Relationship Types To MMM Model.

Hi everyone, I'm Sathya, Solutions Consultant at Ataccama. In this post I am going to discuss about adding Data Classification and relationship types. Step 1: Go to Global setting and you will see Metadata Model, click on Create Entity Step 2: Once Create Entity page opens, fill the Name of the Entity: Data Classification Step 3: Click Add Property and fill in the property needed for the Entity: Name and Description with its data type and hit save Step 4: Goto Metadata Model under Global settings and search for metadata Step 5: Click Add property: Fill in name, type: Embedded Object Array and Object type:dataClassification Step 6: Goto Metadata Model under Global settings and search for term Step 7: Click add property and Fill in data Classification type as Referenced Object, PII and CDE as Boolean Step 8: Hit Apply changes and the System will be Switching to Maintenance mode Step 9: Click DB backup and the application restarts. Step 10: Click on Run changes again and all the chan

1830

olayinkadaramolaData Voyager

Rules- with many conditions

Hi, is it possible to have a single rule that checks many condition. example is below i want a single rule named--Name Validationi want this rule to check and validate name that are in -Title Case --first letter upper case follow by all lower case Accented--- Accepts accented character Accepts special characters Could have Hyphen - the rule should should invalidate any other names that does not meet these conditions . or is there a better way to do this ? preferred to break it down so its will be easy to know the reason why the names are not valid whenever its does not matches

1652

michael.willisData Voyager

Call Runtime-Server OnlineService using Azure Synapse

Hello,I have a runtime-server online service that I am trying to call using Azure Synapse using OAuth authentication. I created a REST Linked Service in Azure that connects successfully to the runtime-server online service. I am using a copy activity in Azure to do a POST call on the online service. When running the copy activity I get the following error:{ "errorCode": "2200","message": "Failure happened on 'Source' side. ErrorCode=UserErrorCharSetInvalidOrNotSupported,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The CharSet '\"UTF-8\"' is invalid or not supported from Response.Headers.Content-Type.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.ArgumentException,Message='\"UTF-8\"' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.\r\nParameter name: name,Source=mscorlib,'", "failureType": "UserError", "target": "CopyDataCatalogToRaw","details": [] }T

1492

Albert de RuiterRocket Pioneer L1

Filtering in metadatareader on mutation datetime

Hi Community,Would there be a way in Ataccama Desktop to select records from for instance entity term that have been inserted/updated/deleted from a certain datetime, like the Published on datetime in the History tab? My intention is to use the metadatareader to read the details from term and then use this mutation datetime in the filter in the metadatareader.Kind regards,Albert

35915

PrithikaData Voyager

Masking Attributes and Exporting

Hello, I’m trying to mask certain attributes in a table and be able to export them while exporting the table as a file. Is it possible to mask them as part of the JSON code while creating an export plan or create masking rules and apply it to the attributes and then export them. (using ataccama ONE Web). Has anyone tried doing this before or has any idea about this? Please let me know!Thank you,Prithika

2602

boopeshmData Voyager

Preventive Data Controls using Ataccama

Hi Everyone,We are in the process of transitioning to Ataccama as our DQ solution. In our current setup, we perform various data controls such as:Empty File Check Duplicate File Check Header or Trailer Validation (e.g., validating trailer record count or checksum)We are keen to understand if these types of data controls can also be implemented using Ataccama. Our goal is to consolidate all data quality processes into a single tool for better efficiency and management.Thank you!

1842

SathyaData Voyager

How to add relationship types in Ataccama

Hi everyone, I am Sathya, Solutions Consultant at Ataccama. In this post I am going to discuss about how to create Relationship types(Source and Target reference). Step 1: To add new relationship type, go to Global settings on the left navigation bar. Step 2: Under Application Settings, choose Relationship types Step 3: Available Relationship Types will be listed. Step 4: Click on create button on the top right corner of the screen layout. Step 5: Fill in the name, Source rel name and Target rel name of the Relationship Type Step 6: DomainOf relationship type is created, with its source rel name and Target rel name with its ID. Step 7: All the relationship Type is listed along with the newly created DomainOf. I hope you found this article useful, please let us know if you have any questions or thoughts in the comments below.

1250

olayinkadaramolaData Voyager

Rules on Names

Hello, I am trying to create a rule for validity on Names. The logic and condition is that the rule matches names that are Title Case --first letter upper case follow by all lower case Accented Accepts special characters Mandatory Could have Hyphen - I have this regex ^[a-zA-Z '-]+$ ^[A-Z][a-z]*(?:[ '-][A-Z][a-z]*)*$(?<=^|\s)[a-z]*[A-Z][a-z]*[A-Z]*[a-z]* above regex capture and validate names like John DoeMary Anne which are okay. but also validate name in all lower case for example john doe which is not okay. i want to know what i need to do to allow the regex not validate names in all lower case .

29413

srija piratlaAtaccamer

Profiling Triggered While Running Monitoring Project

Hello everyone ! I’m Srija ,Solutions Consultant here at Ataccama. Today, I’d like to talk about how to trigger profiling while running a monitoring project.While running the monitoring project, the profiling process is getting triggered without any schedule or event created on the catalog item to initiate profiling. This unexpected profiling of over 1000+ records significantly slows down the system. The likely cause for this issue is the anomaly detection feature being enabled on the catalog item.To address this issue, you need to check if anomaly detection is enabled on the catalog item. Anomaly Detection in Ataccama DQWhat Anomaly Detection Does:Anomaly detection in Ataccama Data Quality (DQ) automatically identifies outliers and unusual patterns in your data. This feature is designed to ensure data quality by detecting anomalies such as unexpected values, missing data, or patterns that do not conform to the expected behavior.Implications of Enabling Anomaly Detection:Automatic

1970

SathyaData Voyager

Adding Data Domain Entity to the MMM Model

Hi everyone, I'm Sathya, Solutions Consultant at Ataccama. In this post I am going to talk about creating Data Domain , relationship using AQL expression. Step 1: On the organization tab, go to Global settings Step 2: Click Navigation Icon Step 3: List of entities will be listed, under Business glossary we will have Data Domains Step 4: Click on the 3 dots and click Add Child option to add a child to the list. Step 5: Pop up window appears, fill in the details as follows:Navigation item type: Listing Listing Node: Term Icon: Folders Text Medium Name: Fill the name of the Data Domain Save and publish. Step 6: All the terms will be added to the Data domain. To filter out we have to update the AQL expression. Step 7:Click on the 3 dots on the right corner of the terms page where the data domain is displayed and choose Edit page. Step 8: In the Filter part we must fill in the AQL Expression Step 9: Syntax for AQL expression:name is "(name of the dataDomain)" OR @termRelationship(target)

3000

C_RData Pioneer

How to rename Post Processing Plans

Hi all, I have a group of PPP that need to be renamed. Is it possible to update name once published? How? Thanks

1622

NazrinDData Voyager

combine email with ataccama MP results

I want to configure one general email notification that will be used for all of my monitoring programs. I would prefer to receive a single email notification that compiles the findings from various monitoring projects rather than the numerous emails I usually receive for each monitoring project. I'm aware that I should use One Desktop, but I'm not sure how to proceed precisely. Well, I know that I have to call monitoringProjectProcessing which allows me to retreive the finish time with processing ID. And then I have to complie to get the ID link of notification but the problem is here that I dont know the step exactly. It would be great if you could help.Thanks in advance.

2472

Kelvin FongData Voyager

(How) can we display the properties of the table attributes in the catalog item tabs?

Let’s say we have some properties placed at the table attribute level. Is it possible to display those table attribute properties in the data structure page of the catalog item as shown below? Thanks in advance.

1662

PrithikaData Voyager

Export plan Shows No Records

Hello, I’m trying to figure out why the export plan section in Ataccama ONE desktop keeps showing “No Records Found” although I have created a list of export plans. Once I create an export plan, I save and publish it, but it does not show up as part of the export plan section instead it says, “JSON error”. Does anyone have any idea on how to fix this issue ?Thank you,Prithika

1904

JTHStar Blazer L3

ONE Expression - Trim

Hi,There are certain records that contain “white spaces” at the beginning and/or at the end. Ex: “ 100137069 “, “ 100017129 “, etc. We don't allow the use of white spaces and want to build a DQ rule to check the invalid records. Hence the advanced expression: length(String_ATTRIBUTE_1) <> length(trim(String_ATTRIBUTE_1)). When I test the DQ rule it works as intended. I copy the above results and paste them in the test environment and it shows me the record is invalid. When I remove the white spaces it will considered valid. However, when I run the Monitoring Project based on the same records and DQ rule it gives me 100% validity. How is this possible? And maybe more importantly, how can I create a rule that will show the invalid records shown earlier?

2466