Discuss, learn, ask, suggest, share all things DQ&C π
Recently active
Hi Team,Is there any way to add rule level notification using desktop tool or graphql and assign the recipients?Β
Is there any global setting at Monitoring Project level to get DQ issue detected warning ifΒ DQ percentage is less than 95%.?14.5.3 version,Β warning seems to be given after below 60%.Β It means that ifΒ for catalog item in Monitoring project shows overall data quality less than 95%, it should be shown amber warning. instead of green ok.Β
HiI have a use case to track changes that have been made on ONE Data in ONE Web. Please find the expected process flow below:Run DQ evaluation on the catalog itemΒ β‘ pushed failed records to ONE DataΒ β‘ user make changes on ONE DataΒ β‘ ONE Data able to track the changesΒ β‘ push final changes to Snowflake tableMy questions would be can we track who make the changes, when and what changes have been made on ONE Web ONE DQ&C? I saw that this may be achieved in ONE RDM but unfortunately we donβt have that. What is an alternative to achieve this one the ONE Web DQ&C? Thanks in advance!
I have two Ataccama sandbox instances both of versionΒ 16.3.0In one instance, I can see catalog items while creating a visualization, but after selecting them, no records are displayed even though the records are present in the ONE data table.For example, consider the Turkey City table shown in the image. The left side of the image shows the visualization screen, while the right side shows the corresponding ONE data table containing the records.Β In case of another sandbox, I canβt even see any catalog items while creating visualization as shown in below screenshot.Β Is this issue occurring because I donβt have the appropriate access roles or permission levels, or could it be caused by something else?
Is it possible to read/extract Data Quality results using metadata reader? has anyone tried that. Can some one share a template if you have one?I dont want it to be a Post Processing Component. I rather want it as an independent Plan that can be scheduled to run daily at a specific time that extracts all the DQ results of all projects of that day. And then using JDBC write i want to add it to my database. And then run some powerBI reports.thanks.
How to Show a trend of data quality score ?Β Example : Trend of data quality scores is going up or low. 9/10 - Quality Score is 50%9/11 - Quality Score is 60%9/12 - Quality Score is 70 %9/13 - Quality Score is 80 %
Hi β@anna.spakovaΒ β@Lisa KovalskaiaΒ I have a requirement to duplicate existing rules for multiple use cases. I donβt want to do it manually by using Ataccama One UI but would like to achieve the same by parameterizing rule names (per use case). Can we make use of any Ataccama desktop features/APIs? Waiting for your inputs. Thanks!
Hi,Iβm trying to get some details/links regardingΒ APIs to read information from Ataccamaβs data observability feature.Looking forward to getting some inputs here.Thanks in advance!
Hi β@anna.spakovaΒ Is it possible to get rule count (number of rules ) for each βRule Definition Sourceβ in Data Stories?Β Thanks.Β
Hi, I am trying to configure landing page of ataccama one web to show results of selected monitoring projects only. I see there is option to add a test_page with admin access but it allows selection of only limited widgets. Is it not possible to show MP results on landing page of the user?Thanks in advance!
Under which entity type are the Passed count and Failed count stored in a Monitoring Project?In a situation where I want to export the result counts for each Monitoring Project, which entity type should be used in the ONE Metadata Reader?I have already checked Monitoring Project, but it only provides a result status (Success / Failed) and does not include the actual Passed and Failed record counts.Β
Export Monitoring Project Metedata + Results For Multiple Runs And VersionsΒ IntroductionΒ For DQ monitoring and reporting, it may be useful to extract the results of monitoring project reports into a spreadsheet, or to an external database. It may also be useful to extract multiple or all reports/runs for a monitoring project at once.The following topic covers a component created to export results, how it can be used, and how it was created (in case modifications need to be made).Β Β Scope of the ComponentΒ The component extracts DQ results from a monitoring project. The output is on the rule instance level, where each rule included in the monitoring project across all tables is output with its dq validity percentage, and the number of valid and invalid records for said rule. Each record is also output with a date field, to organize results by the report run date (as multiple reports may be included in one export)Β Using the ComponentΒ To use the component, move the component files to a fold
Hi,I would like to know if itβs possible to run python program from Ataccama one βΒ Transformation plans feature? If yes, how do we do it, which steps/components from desktop could be used?Β Really appreciate your suggestions here! β@Lisa KovalskaiaΒ β@anna.spakovaΒ Thanks!Β
Hi there!!I am looking for some documentation related toΒ capabilities of Ataccama v15.4 or v16.4 in terms of DQ of semi structured (json/parquet)Β or unstructured data altogether, if itβs supported what are the pre-requisites we need to set up.Please do share.Thanks in advance!
Hello everyone,Iβm trying to export specific details from my monitoring projects and would appreciate some guidance.For each monitoring project, I want to retrieve only the most recent run, including: Monitoring project name Monitoring project ID Stewardship information Result name Result status Result count -(PassedΒ Count/ Failed Count ) Date the run was executed.Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Iβm currently using a metadata reader to retrieve the monitoring project name, ID, and stewardship, which works as expected. However, when I add fields from the embedded Processing entity and join in the relevant columns, the export returns multiple rows per monitoring project, as it includes historical runs instead of only the latest one.Has anyone encountered this before? What is the recommended approach to: Filter or ident
hi Team,Β I just worked on a project with Purview and the way we set up DQ there was different. Creating data products and then creating DQ connection separately than the one used to scan the physical metadata.Β But here in Ataccama its different like here we can ceate DQ rules directly on to the Terms, and here I saw we create the source directly.Β Want to understand similarities and differences between Atacama and Purview from any experts who has spent time in both tools :)Β Β Thanks in advance.Β Regards,Vikas
What is the exam fee forΒ Ataccama ONE v13 Data Quality & Governance Foundation Level certification for attendees who exchange a minimum of 12 months hands-on experience for the must have training? How can we apply for the certification exam? Is the application process online or offline? Is the exam proctored?Β
what is the graphql mutation for creating a monitoring project in AtaccamaΒ Unable to open playground so need help with a direct graphql mutation query for creating Monitoring project
Hello Ataccama Community,Β I have a question regarding the possibility of using nested /consecutive backreferencing into an AQL filter .Β The actual question would be whether if we can create a filter on the βtermβ entity to show only the terms which have 0 termInstances on rules . This was the initial issue because we have used this AQL Filter into a Term Report Page : @termInstance(target).count(Β )==0 . However, this was not enough because it did not filter only the term instances from the rules, but also the ones from the catalog item , attributes , etc. I have initially tried to place something like this : @rule.inputGroups.inputs.termInstance(target).count() =Β 0 , but I got an error and I am not sure if nested backreferencing is possible .After that, we changed the approach and we decided to add another entity βtermsUsedOnRulesβ and I configured it exactly the same as βtermInstances' .Then I have added this entity as a property on the rule as an Embedded Object Array and filtering b
I am trying to create a transformation plan in DQ One Web that merges existing data in a One Data table to the most recent run from a Monitoring Project.The situation is difficult and I will try to be as concise as possible:Β We have a One Data table that is created from a Monitoring Project Data Remediation plan.Β We input notes on the One Data table that explains what needs to be done to that invalid record Every time the monitoring project is ran, the notes in that One Data table are deleted.The fix in my mind is that we could union what is currently in the One Data table (with notes) with the new remediation export in order to retain the notes on the βolderβ version of the One Data table, while allowing any new invalid records to be written into the One Data table.Essentially, I need to somehow create (within the transformation plans on One Web) a left outer join where everything with the current run coming in, but joining with those on the right (the older run) that has notes.Β Β Any
Hello,How to write the following condition in Ataccama Desktop (Join step):left join anl.productsjur p on (asd.creditpurpose = p.cp or coalesce(asd.creditpurpose, p.cp) is null)Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β and (asd.creditpurpose2 = p.cp2 or coalesce(asd.creditpurpose2, p.cp2) is null)Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β and (asd.creditobject = p.co or coalesce(asd.creditobject, p.co) is null)The difficulty is that in the join step only one field for left key and right key is available for selection..
Hi everyone!Β Welcome to the third (and the last) part of our introduction series! The continuing posts will be focused on diving deep into the functionalities of DQ. If you have missed the first two posts you can check them out here:In this post, we'll explore the best practices for monitoring data quality in Ataccama ONE. Monitoring your data quality allows you to track important metrics in real-time and identify potential issues.Β Creating a Monitoring ProjectTo create a new monitoring project, navigate to "Data Quality" and select "Monitoring Project." Click on "Create" and provide a name and description for your project. On the "Configuration & Results" tab, choose the catalog items you want to monitor. You can monitor multiple catalog items within a single project as long as it makes sense for your organization's data quality requirements.Β Customizing DQ DimensionsIn each monitoring project, you can customize the DQ dimensions that contribute to the overall data quality. Look f
Welcome to the first blogΒ in a series exploring the concept and implementation of a Metadata Warehouse, hereafter referred to as MDWH.You are most likely already familiar with concepts that alignΒ with an MDWH,Β such as a data warehouse or a data lake. An MDWH is quite similar to a data warehouse, but focuses on exported metadata stored in another databaseβspecifically, Ataccamaβs metadata. In this scenario, there is no need to integrate data from multiple systems, because there is only one Ataccama implementation. (After all, why use more than one metadata application?π)Β What are the drivers for having an MDWH?At Stater, our MDWH serves the following purposes:Reuse of metadataAtaccama is the golden source for metadata, so when details are required in different applications, they should not differ from those in Ataccama. Examples where reusing metadata is essential include Data modelsMany data models contain common entities/tables and attributes/columns, both across models and within a
Hi,I would like to know if itβs possible to automate rule creation in Ataccama without actually using UI maybe by running scripts or API calls from backend.
Hello All,I need your help on Ataccama One DQ issue.I created an accuracy rule for Zipcode using lkp table.In the lkp table, Zipcode is stored as an integer, but in the Catalog,src attribute as nvarchar(20).When building the rule, Ataccama forces me to choose integer as the data type because the lkp table uses integers.But since the Catalog attribute is nvarchar, the rule is not working. and tried with lkp as string and i/p attribute in lkp also as string which thrown an error while evaluation.I also cannot change the src attribute datatype from nvarchar(20) to integer because:If I convert it to integer using cast, NULL values will move to completeness checks,And I will not get the incorrect rows under accuracy.So I need a solution that allows the accuracy rule to work even though the lookup table uses an integer Zipcode and the Catalog column uses nvarchar.Thanks in advance.DeepaliΒ
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.