Discuss, learn, ask, suggest, share all things DQ&C π
Recently active
Hi community,Does anyone know about documentation about visualizing relationship types? On the documantation portal I found this, but that doesn't seem to cover the setup in the Graph Visualization section in the Application Settings. The documentation mentions βFor more information, seeΒ Configure Graph Visualization.β, but that is a link to the page you are already on.Kind regards,Albert
Hello Dears!Β Would someone advise if there any watch outs once I want to rename Monitoring projects and DQ rules on ONE Web?I have running projects and DQ rules naming convention of which I would like to improve. If I just simply change names, wonβt it have negative impact anywhere?Thank you.Β Kind regards,Maria
Hi CommunityΒ Iβm Antonio, a Professional Services Consultant here at Ataccama.Β Assets created in ONE do not come with the option of being sorted into folders out of the box. This guide will demonstrate how to set up folders for rules, butΒ can be used for other asset types and isnβt specific to rules.Β In this guide we will describe how to set up a custom attribute βruleFolderValueListβ on rules which will hold data from a list of values, then create new navigation sections in the tool where we can apply AQL expressions to filter rules based on our set criteria. This will allow us to split rules into folders based on the value that has been entered into the custom attribute. The process and results outlined have only been tested in v13.9.3 and v14.5.0 of the ONE Web application.Β Create Value ListFirst we will create the list of values. To do this follow the below stepsGo to Global Settings > Metadata Model.Β Select Create entity.Provide the following information:Name of the value that
I would like to setup a generic data quality rule to check string length. Requirements for length checks differs per attribute. SayΒ column1 needs to have max length of 10Β column2 needs to have max length of 15 and so onIs it possible to create a generic rule to check string length, passing value of length as parameter?I am aware of attribute length insights available as part of DQ profiling results but I would like to make the check as part of monitoring project. Thanks!
HiI would like to get a summary of DQ checks and results like the one we have in ONE web like how many in % and count of broken rules, how many are passing and how many records do we have in total etc. in Ataccama ONE desktop. I tried looking out any related topic in the community but I could not find one. Is it possible to achieve this via Ataccama ONE desktop as we have an existing plan comparing data fields in two different sources. Do you mind sharing the suggested steps to achieve this? Thanks in advance!
When it comes to data quality (DQ) management, monitoring projects provide us with ability to export DQ results and maintain a close eye on data quality. However, the initial export contains attributes organized in columns, alongside additional columns housing information regarding the DQ results for these attributes. Analysing this data can be challenging because DQ rule results and explanations are presented as a single string.Despite the fact, that extracting DQ results in a customized format might require a more detailed approach, it might be better aligning with the client's specific needs. In this guide, you will find information how to export DQ results tailored to specific requirements.The Task at Hand:Our objective is to ensure a separate line for each attribute in an export file, with the following columns related to DQ results:the name of the DQ rule; an explanation for the check; a list of terms assigned to that attribute; the attribute value.We have three main steps:Attrib
HiΒ everyone, I'm Sathya, Solutions Consultant at Ataccama. In this post I am going to discuss about adding Data Classification and relationship types.Β Step 1:Β Go to Global setting and you will see Metadata Model, click on Create EntityΒ Β Β Step 2:Β Once Create Entity page opens, fill the Name of the Entity: Data ClassificationΒ Step 3:Β Click Add Property and fill in the property needed for the Entity: Name and Description with its data typeΒ and hit saveΒ Step 4:Β Goto Metadata Model under Global settings and search for metadataΒ Step 5:Β Click Add property: Fill in name, type: Embedded Object Array and Object type:dataClassificationΒ Step 6:Β Goto Metadata Model under Global settings and search for termΒ Step 7:Β Click add property and Fill in data Classification type as Referenced Object, PII and CDE as BooleanΒ Β Β Step 8:Β Hit Apply changes and the System will be Switching to Maintenance modeΒ Step 9:Β Click DB backup and the application restarts.Β Β Step 10:Β Click on Run changes again and all the chan
Hi, is it possible to have aΒ single rule that checks many condition. example is belowΒ Β Β i want a single rule named--Name Validationi want this rule to checkΒ and validate name that are in -Title Case --first letter upper case follow by all lower caseΒ Accented--- Accepts accented characterΒ Accepts special charactersΒ Could have Hyphen -Β the rule should should invalidate any other names that does not meet these conditions . or is there a better way to do this ? preferred to break it down so its will be easy to know the reason why the names are not valid whenever its does not matchesΒ
Hello,I have a runtime-serverΒ online service that I am trying to call using Azure Synapse using OAuth authentication. I created a REST Linked Service in Azure that connects successfully to the runtime-server online service. IΒ am usingΒ a copy activity in Azure to do a POST call on the online service. When running the copy activity I get the following error:{ "errorCode": "2200","message": "Failure happened on 'Source' side. ErrorCode=UserErrorCharSetInvalidOrNotSupported,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The CharSet '\"UTF-8\"' is invalid or not supported from Response.Headers.Content-Type.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.ArgumentException,Message='\"UTF-8\"' is not a supported encoding name. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.\r\nParameter name: name,Source=mscorlib,'", "failureType": "UserError", "target": "CopyDataCatalogToRaw","details": [] }T
Hi Community,Would there be a way in Ataccama Desktop to select records from for instance entity term that have been inserted/updated/deleted from a certain datetime, like the Published on datetime in the History tab? My intention is to use the metadatareader to read the details from term and then use this mutation datetime in the filter in the metadatareader.Kind regards,Albert
Hello, Iβm trying to mask certain attributes in a table and be able to export them while exporting the table as a file. Is it possible to mask them as part of the JSON code while creating an export plan or create masking rules and apply it toΒ the attributes and then export them. (using ataccama ONE Web). Has anyone tried doing this before or has any idea about this? Please let me know!Thank you,PrithikaΒ Β
Hi Everyone,We are in the process of transitioning to Ataccama as our DQ solution. In our current setup, we perform various data controls such as:Empty File Check Duplicate File Check Header or Trailer Validation (e.g., validating trailer record count or checksum)We are keen to understand if these types of data controls can also be implemented using Ataccama. Our goal is to consolidate all data quality processes into a single tool for better efficiency and management.Thank you!
Hi everyone, I am Sathya, Solutions Consultant at Ataccama. In this post I am going to discuss about how to create Relationship types(Source and Target reference).Β Step 1: To add new relationship type, go to Global settings on the left navigation bar.Β Step 2:Β Under Application Settings, choose Relationship typesΒ Β Step 3:Β Available Relationship Types will be listed.Β Β Step 4:Β Click on create button on the top right corner of the screen layout.Β Β Step 5:Β Fill in the name, Source rel name and Target rel name of the Relationship TypeΒ Β Step 6:Β DomainOf relationship type is created, with its source rel name and Target rel name with its ID.Β Β Step 7:Β All the relationship Type is listed along with the newly created DomainOf.Β Β I hope you found this article useful, please let us know if you have any questions or thoughts in the comments below.Β Β
Hello, I am trying to create a rule for validity on Names. The logic and condition is that the rule matches names that areΒ Title Case --first letter upper case follow by all lower caseΒ Accented Accepts special charactersΒ Mandatory Could have Hyphen -Β I have this regexΒ ^[a-zA-Z '-]+$Β ^[A-Z][a-z]*(?:[ '-][A-Z][a-z]*)*$(?<=^|\s)[a-z]*[A-Z][a-z]*[A-Z]*[a-z]*Β above regex capture and validate names likeΒ John DoeMary Anne which are okay.Β but also validate name in all lower case for example john doe which is not okay. i want to know what i need to do to allow the regex not validate names in all lower case .Β
Hello everyone ! Β Iβm Srija ,Solutions Consultant here at Ataccama. Today, Iβd like to talk about how to trigger profilingΒ while running a monitoring project.While running the monitoring project, the profiling process is getting triggered without any schedule or event created on the catalog item to initiate profiling. This unexpected profiling of over 1000+ records significantly slows down the system. The likely cause for this issue is the anomaly detection feature being enabled on the catalog item.To address this issue, you need to check if anomaly detection is enabled on the catalog item.Β Β Β Β Anomaly Detection in Ataccama DQWhat Anomaly Detection Does:Anomaly detection in Ataccama Data Quality (DQ) automatically identifies outliers and unusual patterns in your data. This feature is designed to ensure data quality by detecting anomalies such as unexpected values, missing data, or patterns that do not conform to the expected behavior.Implications of Enabling Anomaly Detection:Automatic
Hi everyone, I'm Sathya, Solutions Consultant at Ataccama. In this post I am going to talk about creating Data Domain , relationship usingΒ AQL expression.Β Step 1:Β On the organization tab, go to Global settingsΒ Step 2:Β Click Navigation IconΒ Step 3:Β List of entities will be listed, under Business glossary we will have Data DomainsΒ Step 4:Β Click on the 3 dots and click Add Child option to add a child to the list.Β Step 5:Β Pop up window appears, fill in the details as follows:Navigation item type: Listing Listing Node: Term Icon: Folders Text Medium Name: Fill the name of the Data Domain Save and publish.Β Β Β Β Step 6:Β All the terms will be added to the Data domain. To filter out we have to update the AQL expression.Β Step 7:Click on the 3 dots on the right corner of the terms page where the data domain is displayed and choose Edit page.Β Step 8: In the Filter part we must fill in the AQL ExpressionΒ Step 9:Β Syntax for AQL expression:name is "(name of the dataDomain)" OR @termRelationship(target)
Hi all, I have a group of PPP that need to be renamed.Β Is it possible to update name once published? How? Thanks
I want to configure one general email notification that will be used for all of my monitoring programs. I would prefer to receive a single email notification that compiles the findings from various monitoring projects rather than the numerous emails I usually receive for each monitoring project. I'm aware that I should use One Desktop, but I'm not sure how to proceed precisely. Well, I know that I have to call monitoringProjectProcessing which allows me to Β retreive the finish time with processing ID. And then I have to complie to get the ID link of notification but the problem is here that I dont know the step exactly. It would be great if you could help.Thanks in advance.
Letβs say we have some properties placed at the table attribute level. Is it possible to display those table attribute properties in the data structure page of the catalog itemΒ as shown below? Thanks in advance.Β Β
Hello, Iβm trying to figure out why the export plan section in Ataccama ONE desktop keeps showing βNo Records Foundβ although I have created a list of export plans. Once I create an export plan, I save and publish it, but it does not show up as part of the export plan section instead it says, βJSON errorβ. Does anyone have any idea on how to fix this issue ?Thank you,PrithikaΒ Β
Hi,There are certain records that contain βwhite spacesβ at the beginning and/or at the end. Ex: βΒ Β Β 100137069Β Β Β Β β, βΒ Β Β 100017129Β Β Β Β β, etc. We don't allow the use of white spaces and want to build a DQ rule to check the invalid records. Hence the advanced expression:Β Β length(String_ATTRIBUTE_1) <> length(trim(String_ATTRIBUTE_1)). When I test the DQ rule it works as intended. I copy the above results and paste them in the test environment and it shows me the record is invalid. When I remove the white spaces it will considered valid. However, when I run the Monitoring Project based on the same records and DQ rule it gives me 100% validity. How is this possible? And maybe more importantly, how can I create a rule that will show the invalid records shown earlier?Β Β
Hi Community,I have usedΒ this community tutorialΒ (thanks @anna.spakovaΒ !) andΒ this part of the documentation portalΒ to create a custom relationship (and an existing custom relationship as example). So that should be enough info, but still I encounter an issue. As follows.The relationship is between an extension of term (entity term) and custom entity application. The new relationship has been defined in entityΒ entityTermSourceOfRecordRel as followsThis new entity has also been defined in entity metadata as array.A new relationship type has been defined for this purpose:In the entity term screen I have defined the realtionship as follows:{"_type":Β "relationships","relationshipPropertyPath":Β "/entityTermSourceOfRecordRel"}On the entity term screen I would now expect to be able a new relationship like this (from the example we already had implemented):Instead I only have the option to define a new relationship type, like:What am I overlooking? We are on version 13.9.Thanks and kind regard
Hey Everyone!We have a use case involving datastage and curious if anyone has worked with this in the past. We have a team who has datastage jobs that are triggered by a table being pushed information downstream. We are wondering if there is a way either in the web or desktop, if we can use the trigger files that are created by datastage,Β somehow plug into the datastage flow, or another wayΒ that would trigger a monitoring project simultaneously to run data quality checks against the load? if certain data is not loaded in correctly, it will break the connection for the two tables, and in turn break the job,Β and they want to see the population of what is breaking the join. Hope this makes sense!Β Thanks,Thomas
Hi everyone! Iβm wondering if its possible to trigger or use theΒ Review workflow from one desktop? For instance, im currently updating a large amount of attribute descriptions and am publishing them with one desktop. Whenever I use the One Metadata writer it will apply the descriptions automatically without further approval.I would like to know if its possible to write the descriptions and then have them sent to the tasks board for the data owner to review and approve or deny.Example of what im trying to do:Use one desktop to write descriptions to attributes Review workflow is triggered, andΒ new descriptions are assigned to data owner for review (preferably in batches of 15-20 if possible) Owner either approves changes, causing descriptions to be updated or denies them causing no change to descriptionsThank you for any help!
Hello everyone!In this post, Iβll walk you through how to extract the description of all the columns and tables using the one desktop plan. Providing detailed descriptions and comments is a fundamental best practice in data management and data standards. This approach significantly enhances theunderstanding and usability of data.The summary of all the tables, attributes and their descriptions can be extracted using the below plan.1. Open a plan and add Attribute and Catalog Item metadata reader steps.2. The Attribute meta data reader helps to extract the attribute names, comments, description. Open the meta data reader and provide the column names as below and provide the Parent Id Column name as βCatalogItemIDβ.3. The Catalog Items meta data reader helps to extract the catalog Item name based on the catalog Item Identifier. Open the meta data reader and provide the column names as below and provide the Id column name as βcatalogItemIDβ.4. Use the Join Step and join the meta data reade
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.