Data Quality & Catalog ⚙️

My organization runs McAfee antivirus.  I’ve just tried out the DQ Analyzer 11 tool and really like it, but it takes forever to start up or perform any actions.  Just starting up, the antivirus pegs 1 CPU for about 30 minutes before the application starts.  And then various actions like starting a new profile or clicking a menu item will also be delayed by several minutes at least.We’ve tried excluding the install directory, but it hasn’t helped.  Any ideas?  Has anyone else run into this issue before, and how did you solve it? 

Can we create custom check on data as per our need and get stats out of it ??for example.. I want to check whether colA has values between list [‘A’,’AB’,’O’,’B’] and stats about the values that are present in data.

I was thinking that this should be possible with a hack to the postgreSQL jdbc configuration using the redshift jdbc driver. Does anyone know if this is possible or if there is another avenue?Edit: I found the connection after the install and properly configured the Redshift driver and connection settings.

Hi,We have a problem in a workflow in DQC, with task “Send Mail”, because the generated message is incompatible with the internet message format RFC 5322.When there are at least 2 recipients, in the generated message, the header “to” is duplicate as many times as there are recipients, but the RFC 5322 standard does not allow this.How to correct this problem ?Thanks for your help.

I have Ataccama 12.x version and I do not see their Aggregate functionPlease help, how to get it enabled or patch it Thanks,Mukesh

Full support for team based permissions in Ataccama ONE is coming in v14. However, you may need to implement team based permissions sooner than v14. This document is a guide to how you could achieve this in v13.8. BackgroundPermissions in ONE can be configured as role based or user based. In most cases, configuration by role is preferred as it will usually be a one-time set up, where the role only needs to be configured once in the beginning, then any users allocated to that role will inherit all permissions on role.The out of the box roles in ONE have some capabilities already allocated. These are auto-sharing permissions, which allows the role to have access to all items of that entity. E.g. Your role could have the capability of Rule Owner. This means the user will be automatically be owner of all rules that would ever be created on the platform.But what if your user should only be owner to rules (or any other entities) within their team / division / domain?As it stands in v13.8, th

Ataccama is known and appreciated by our customers for complex, big deployments where massive amounts of data are processed. We were one of the first to support MapReduce and Spark as native pushdown and enable distributed processing across different cluster nodes.Ataccama’s engine works with the major of enterprise data sources and processes them to provide valuable insight, such as statistics of the data, detection of domains, and data quality.To provide the best experience for our customers that use Snowflake, we developed a tighter integration between our technologies. Specifically, we use the Snowflake data processing capabilities in a pushdown manner which means that data is processed directly on Snowflake.The key advantages of this integration are:Lower data transfer costs for the client. Snowflake’s computational capabilities deliver results faster because of reduced IO operations. Customers don’t incur additional costs to maintain external systems for Big Data. Improved securi

We created a number of data quality business rules that will fail records from the source file we are reading.  1.5 Million records.How do you port the failed records to a flat file?  There is no drill down in One Desktop so we are having issues getting business users approval on code without seeing the failed records?

I have a rule binded to 2 columns in a catalog item.By mistake i have changed the data type of those columns without unbinding the rule.Now i am unable to unbind thise rules from that column.i am getting error while saving or publishing the project : server refused sent data as invalid. Please guide me,what i will do here

​How to tune/optimize a DQC load plan?In day-to-day life, the consultants may come across several performance issues of various magnitude in their load plans at some point in the projects. There are several optimization tips mentioned below for tuning the long running plans.Check the memory allocationDQC plans store the data while processing, which means that the more input data there is, the more memory DQC will require. Therefore the intensive executions can result in a memory overflow error if the default memory allocated for DQC is consumed. In such cases, an increase of memory could present a solution.Related Errors: OutOfMemoryError: Java Heap Space or GC Overhead Limit Exceeded... java.lang.OutOfMemoryError: unable to create new native thread... The memory allocation can be done in several ways,Increasing the memory of the DQC IDE(Desktop) Increasing the memory allocation to specific plan in the IDE Increasing the memory allocation for the entire project in IDE Increasing the me

If you would like to filter e.g. business terms, use:$type == 'businessTerm'Switch to Advanced filtering with AQL (Ataccama Query Language)Terms list before applying a filterFiltering resultsLink to documentation:Searching Using AQL Expressions

Every step that allows for the input of an expression shows a button called Template.... This button can be used to pre-fill the Expression field with one from an out-of-the-box list.You can create your own expressions for common or complex repeatable operations, follow these steps:Go to Window in the top navigation menu and select Preferences Navigate the menu to Ataccama DQC and go to Expression Editor and select TemplatesChoose between Expression Templates and RegEx TemplatesSelect FolderClick on Add Template..Fill in Name, Expression and DescriptionConfirm with OK

Any one can help to create this template using version 13.6?  It show the items clearly.  Our users can know the data presentation without explanation.  However, I can find this template in our current version.

Any best practices for using the Purpose function in Ataccama One (webapp)? I see that it is recommended for monitoring projects but when I create a monitoring project I want to be able to search for that tag. Any tips for searching just based on the purpose tag? I’m very new to Ataccama so any and all tips would be appreciated!

I’m trying to read the file name of a file so I can write it in an email within a Workflow. I’m using the “Operate On File” task. The Operation I selected is “INFO”. I then point to the Source File. Next I’m trying to determine which variable I use to supply to the Send Email task. The Operate On File Documentation says you can use task variables and the example they give is “name”, “localName”, “size”, and “timestamp”. When I try using these variables in the Send Email task they just come across as blank in the email (variables appear to be null). Here is the body of the email I’m using: Name = ${name}localName = ${localName} Two questions:Does any one happen to know what I’m doing wrong in above?Additionally is there a way to pull out a file name within a Plan. I’m using the “Excel File Reader” step. I want to pass along the file name in the flow so I can use it later on. Thanks!

Hi, I’ve just started data profiling with Ataccama One so please bear with me.I have data with technical IDs (e.g. “000123456”, “009876543”,...) in a csv that I imported. The IDs are however interpreted as a number.How exactly can I change the attribute type to text? I have done so when importing the file: but it seems to go to numbers nonetheless.The original file has comma-separated values that are all between double quotes. Thanks for any help! edit: typo

I opened the on-prem AtaccamaONE environment this morning and was looking at recent (auto) profiled catalog items. A majority of them are not found - the error is “Catalog item not found” and the provided explanation is “This catalog item doesn’t exist anymore or someone has revoked your access”. Neither of these cases should be true as I am the only one who currently works in the environment and I’m the one who launched the data discovery on 3/09. This is a recent error - I have not seen this before. Any help you could provide would be appreciated.Thanks.

How can we use functionality like ‘for’ and ‘while’ loop in detection rule field while writing/creating rules for capturing data/records.

Hello, I’m new with Ataccama and I have a simple problem. I want call a Rest service:https://api.bulkemailverifier.com/api/a/v1?key=xxxxxxxxxx&email=myemail.comI’m using the JSON Call step and If I put the full URL with my email it’s working, but now I want read the emails forma txt file and I don’t know how to make the url dynamic.In the step I have 3 parameters: URL: Resource  I set it as https://api.bulkemailverifier.com/api/a/v1?key=xxxxxxxxxx&email=Method: GETURL: HERE IS WHERE i DONT KNOW WHAT TYPE. I have tried a lot of combinations without successThe input file has one column with the mail: src_mailThanks 

How can we download files from remote path ( connected through vpn) on to local?I tried with Operate on File component but it is not working.Is there any work around for it?

Enterprise Data Governance as the management of data across its pipeline - from production to consumption (and retirement). How to enable our “data fellows” to leverage on data to save money, save time and reduce risk?What are the key elements to success? What are your thoughts?