Skip to main content

Data Quality & Catalog ⚙️

Discuss, learn, ask, suggest, share all things DQ&C 🔍

  • 786 Topics
  • 2,012 Replies
786 Topics
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
How to monitor the current execution of a plan?Tutorial

DQA offers a way to monitor the processing visually, providing counts of processed records, percentages on completion of the different operations (steps) and visual cues to the flow of data to identify bottlenecks. This can be achieved by opening a special view of any plan being executed, which will mirror the original plan, showing the processing of records and other aforementioned visual features.To open the monitoring, click the Show progress button, which appears in the Console when the plan is being executed.You can also configure this monitoring window to be opened automatically whenever a plan is executed. The default configuration for the monitoring can be changed by:Open Window > Preferences Navigate to Ataccama DQ Analyzer > Launching > Progress Viewer. Change Automatically open progress viewer for launched plans to Always. You can also change how the Progress Viewer should behave once the execution finishes. 

380
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
Is it possible to automate table profile creation?F.A.Q.

Table profile creation can be automated after buying the more advanced Data Quality Center (DQC) software.Schedule a call | Ataccama On the other hand, with Data Quality Analyzer (DQA, free software), the automation is not possible.  

370
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
How to configure the temporary folder used by DQA (permissions error)?F.A.Q.

You might experience problems when installing DQA on a computer with limited permissions or high restrictions. Even if the installation is done in a folder where you have rights for installation, DQA might fail to execute plans. The reason is that DQA makes use of a temporary folder (usually in the path to the user cache folder) for placing instances of executed plans. This folder is the default Java temporary folder. The execution might throw an exception due to being unable to create a temp plan file.You can override the path to this folder by passing its chosen value as part of the special environment variable JAVA_OPTS , which serves as the Java Virtual Machine arguments. This should be created as follows:Open for editing the dqa.ini file placed in the root folder of the DQA installation, using an editor sensitive to End Of Lines (EOLs) in Windows, WordPad is an option, Notepad++ is another option. Add the variable java.io.tempdir . The variable must be passed using the -D notation

420
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
How to reserve more memory for the DQA IDE?Best Practice

If you experience out of memory errors when processing a large dataset. You can increase the amount of system memory that is available to the Ataccama DQA application.Open dqa.ini located in the root folder of the DQA installation. This file contains the JAVA_OPTS that will be used by the virtual machine on start.Edit Xmx and substitute the number (256 by default) by a higher number for reserving more memory, e.g. -Xmx1024 

480
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
How to reserve more memory for executing plans?Best Practice

DQA plans store data while processing it. This means the more input data there and the more the plan logic multiplies this data to produce desired outputs, the more memory needs to be reserved it.Some of the most memory-intensive steps are Profiling and JDBC Reader.There steps to change the default settings are:Open Window > Preferences > Ataccama DQ Analyzer > Launching. Change the value of the Default memory for launch configuration (MB) to 1024, which is recommended for modern for modern machines. 

550
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
DQ Analyzer for Windows can help you gain valuable insights into your data.Tutorial

Did you know our DQ Analyser for Windows (32 or 64bit) can help you to gain valuable insights into your data, and the application is free to use.Visit https://www.ataccama.com/download/dq-analyzer Some of the features include;Duplicate count Pattern & mask analysis Business rule analysis Numeric statistics Ability to drill down into the data on a per record basis Foreign key analysis Dependency analysis Primary key analysis Concerned about duplicate values in your datasets?Frequency Analysis can tell us that we have a value ‘SYSTEM MIGRATION’ in the customer name column 4551 times. It is clear that these customer names were not transferred into the current system correct - supposedly during previous migration activity.One of my personal favorites for the identification of duplicate records is Group Frequency analysis.Here we can see the same value of 4551 as a group size, with a group count of 1. We can interpret the group analysis as there is 1 value that appears 4551 times.Or usi

1460
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
Data Quality Analyzer Tutorial - 4. Email AnalysisTutorial

Learn how to standardize and extract email addresses from a customer data set. Used functions are regex matching, conditions to separate data flows, and merging data flows. 

480
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
Data Quality Analyzer Tutorial - 3. Advanced ProfilingTutorial

Learn how to configure custom mask analysis, enable and configure drill-through, check your data against business rules, and configure primary key analysis. 

410
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
Data Quality Analyzer Tutorial - 2. Understanding Profiling ResultsTutorial

Understand and interpret the information generated in profiling results with the free Ataccama DQ Analyzer (DQA). 

350
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
Data Quality Analyzer Tutorial - 1. Creating a profileTutorial

Get started with Ataccama DQ Analyzer (DQA) and learn how to create a data profile. 

530
DannyRyan
Ataccamer
DannyRyan
Ataccamer
DannyRyanHead of Technical Training
published in Data Quality & Catalog ⚙️
How to setup email notifications for DQ monitoring projects?Tutorial

Did you know that you can setup email notifications when certain events occur in your DQ Monitoring Projects? There are 4 types of notifications that can be assigned to events; Notification + Email sends both an in-app notification and an email to the recipients in case of the event occurrence. Email sends an email to the recipients in case of the event occurrence. Notification sends an in-app notification to the recipients in case of the event occurrence. Leave the events notification options blank if you do not want to be notified of a particular event occurrence. Process Steps Configure the MMM Module for generating links in emails.Ensure the MMM Module has been configured for SMTP (There is a link to the documentation at the end of this post)Underneath your existing configuration for plugin.email.ataccama.one.addressAdd a new configuration line for plugin.email.ataccama.one.base.url which is the base URL for generating links in emails. The default value is http://localhost:80

6000
DannyRyan
Ataccamer

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings