Data Quality & Catalog ⚙️

We continue with our series - today’s topic is Detection rules.👉 Get Certified with Ataccama article series#1 Adding Data Sources#2 Add a Data Source to the Catalog#3 Business Terms🔎 LET’S TEST YOUR SKILLSUsing the ONE Web App, in the ONE Data Quality module, create a Detection Rule for 'Blood Type', if the value in the input column is one of the following 4 values, then Detect Term A B AB O  💡 How to do it? Navigate to Data Quality - Detection Rules and click on CreateFill in Name and other information if needed and Save itGo to tab Implementation and fill in Input Attributes - always make sure you follow best practices for naming conventions In the section Rule Logic, select DetectionSwitch to Advanced Expression, write a condition and Test RulePublish the rule Do you have any question or tips&tricks? Let us know in comments section! 

I am doing POC to adopt ataccama, please help me out on this requirementRequirement:Previous and current run data validation.For example:what is the threshold of yesterday sales or one week back sales and todays sales 

Is there a best approach to have LKPs updated via a webservice so that other web services can consume the updated lkp without restarting DQS? I have Web Service ‘A’ which builds a dictionary file. I also have Web Service ‘B’ which uses the dictionary built by Web Service ‘A’.I would like to run Web Service ‘A’ so that Web Service ‘B’ is able to immediately use the updated dictionary without restarting the DQS service.   Currently when i run Web Service ‘ A’ I get a locked file issue:26.07.2022 12:43:56 [FATAL]   [OnlineServicesComponent][buidTestDictionary.online][buidTestDictionary][buidTestDictionary.comp] [40] Internal error occurred during run of the plan: java.nio.channels.OverlappingFileLockException[STEP Lookup Builder[Lookup Builder]]java.nio.channels.OverlappingFileLockException    at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source)    at sun.nio.ch.SharedFileLockTable.add(Unknown Source)    at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source)    at com.ataccama.dqc.i

Hi all-- Please find below an introductory primer for the Ataccama Expressions language, for those familiar with Excel formulae (which are deliberately very similar in structure and use to the way we’ve designed the expression language)… IntroductionOverviewOne of Ataccama’s core values when developing Ataccama ONE is to create a user experience which simplifies as far as possible the technical complexities of data manipulation and data quality, and to put as many capabilities into the hands of business and non-technical users as possible.With this philosophy in mind we have created a powerful expression language, which moves away from the complexity of development languages such as scripting/python/SQL. Despite containing powerful and extensible functionality for complex data challenges, the design of the expressions have been made as simple as possible, and can be easily learned by anyone with a familiarity in MS-Excel formulas and features.This expression language is used throughout

I am working on 12.9 at my company, where I can see the data classification ?I do not see that, please help me out if this version support. Thanks & regards,Mukesh

How to Show a trend of data quality score ? Example : Trend of data quality scores is going up or low. 9/10 - Quality Score is 50%9/11 - Quality Score is 60%9/12 - Quality Score is 70 %9/13 - Quality Score is 80 %

How to change  environment in ataccama web interface?

I did full profiling for data catalog and it finished successfully, but “Error” occured after when we want to see data. Where there may not be access for source/table?

How to Check whether the target column has only DATE values or not.?

In the second article of Get certified with Ataccama series, we will focus on Adding a Data Source to the Catalog. 👉 First article of the series - #1 Adding Data Sources 🔎 LET’S TEST YOUR SKILLSUsing the ONE Web App, in the ONE Knowledge Catalog module, add your database as a data source. Using the ONE Web App, in the ONE Knowledge Catalog module, add the table you created in ONE Desktop application to the Catalog Items. 💡 How to do it? Go to ONE Web App - Data Catalog - Sources and click on CreateFill in Name and Description (if needed)Fill in Connection details - Select connection type, Name and JDBCAdd Credentials, select Credential type, fill in required information and Test ConnectionSave and publishNavigate to Connections in and click on BrowseSelect your tableFrom Profile menu select ProfileAfter the profiling is done, check that your table is in the Catalog items list Do you have any question or tips&tricks? Let us know in comments section! 

Has anyone successfully connected to Salesforce environments (prod/test/dev etc.) thru One Desktop (13.7.1).  Thanks

What makes manual data quality so hard to scale in large organization? Is there a step by step approach to automating data quality? Why is an automated DQM system the future of enterprise data quality?You can find an answer to all these questions (and many more!) in our latest whitepaper.👉 Download it here

Can we connect Ataccama to Sharepoint xl file? if yes, how can I achieve this? I am new this forum and new to ataccama.

My organization runs McAfee antivirus.  I’ve just tried out the DQ Analyzer 11 tool and really like it, but it takes forever to start up or perform any actions.  Just starting up, the antivirus pegs 1 CPU for about 30 minutes before the application starts.  And then various actions like starting a new profile or clicking a menu item will also be delayed by several minutes at least.We’ve tried excluding the install directory, but it hasn’t helped.  Any ideas?  Has anyone else run into this issue before, and how did you solve it? 

Can we create custom check on data as per our need and get stats out of it ??for example.. I want to check whether colA has values between list [‘A’,’AB’,’O’,’B’] and stats about the values that are present in data.

I was thinking that this should be possible with a hack to the postgreSQL jdbc configuration using the redshift jdbc driver. Does anyone know if this is possible or if there is another avenue?Edit: I found the connection after the install and properly configured the Redshift driver and connection settings.

Hi,We have a problem in a workflow in DQC, with task “Send Mail”, because the generated message is incompatible with the internet message format RFC 5322.When there are at least 2 recipients, in the generated message, the header “to” is duplicate as many times as there are recipients, but the RFC 5322 standard does not allow this.How to correct this problem ?Thanks for your help.

I have Ataccama 12.x version and I do not see their Aggregate functionPlease help, how to get it enabled or patch it Thanks,Mukesh

Full support for team based permissions in Ataccama ONE is coming in v14. However, you may need to implement team based permissions sooner than v14. This document is a guide to how you could achieve this in v13.8. BackgroundPermissions in ONE can be configured as role based or user based. In most cases, configuration by role is preferred as it will usually be a one-time set up, where the role only needs to be configured once in the beginning, then any users allocated to that role will inherit all permissions on role.The out of the box roles in ONE have some capabilities already allocated. These are auto-sharing permissions, which allows the role to have access to all items of that entity. E.g. Your role could have the capability of Rule Owner. This means the user will be automatically be owner of all rules that would ever be created on the platform.But what if your user should only be owner to rules (or any other entities) within their team / division / domain?As it stands in v13.8, th

Ataccama is known and appreciated by our customers for complex, big deployments where massive amounts of data are processed. We were one of the first to support MapReduce and Spark as native pushdown and enable distributed processing across different cluster nodes.Ataccama’s engine works with the major of enterprise data sources and processes them to provide valuable insight, such as statistics of the data, detection of domains, and data quality.To provide the best experience for our customers that use Snowflake, we developed a tighter integration between our technologies. Specifically, we use the Snowflake data processing capabilities in a pushdown manner which means that data is processed directly on Snowflake.The key advantages of this integration are:Lower data transfer costs for the client. Snowflake’s computational capabilities deliver results faster because of reduced IO operations. Customers don’t incur additional costs to maintain external systems for Big Data. Improved securi

We created a number of data quality business rules that will fail records from the source file we are reading.  1.5 Million records.How do you port the failed records to a flat file?  There is no drill down in One Desktop so we are having issues getting business users approval on code without seeing the failed records?

I have a rule binded to 2 columns in a catalog item.By mistake i have changed the data type of those columns without unbinding the rule.Now i am unable to unbind thise rules from that column.i am getting error while saving or publishing the project : server refused sent data as invalid. Please guide me,what i will do here

​How to tune/optimize a DQC load plan?In day-to-day life, the consultants may come across several performance issues of various magnitude in their load plans at some point in the projects. There are several optimization tips mentioned below for tuning the long running plans.Check the memory allocationDQC plans store the data while processing, which means that the more input data there is, the more memory DQC will require. Therefore the intensive executions can result in a memory overflow error if the default memory allocated for DQC is consumed. In such cases, an increase of memory could present a solution.Related Errors: OutOfMemoryError: Java Heap Space or GC Overhead Limit Exceeded... java.lang.OutOfMemoryError: unable to create new native thread... The memory allocation can be done in several ways,Increasing the memory of the DQC IDE(Desktop) Increasing the memory allocation to specific plan in the IDE Increasing the memory allocation for the entire project in IDE Increasing the me

If you would like to filter e.g. business terms, use:$type == 'businessTerm'Switch to Advanced filtering with AQL (Ataccama Query Language)Terms list before applying a filterFiltering resultsLink to documentation:Searching Using AQL Expressions