Learn & Share

Hi Team, I have few data loads that read data using API’s and writing to Snowflake tables. Often the data is incomplete & inconsistent. Im checking if I can use Attacama to compare the data between different snowflake schema’s : eg) Raw vs Curated? I understand we can write SQL Execute (select col1 from Tab A minus select col1 from Tab b and create a plan. But i have 500+ jobs and needs a lot of plans to be created. Looking for correct way of doing this?  

We created a number of data quality business rules that will fail records from the source file we are reading.  1.5 Million records.How do you port the failed records to a flat file?  There is no drill down in One Desktop so we are having issues getting business users approval on code without seeing the failed records?

Tested on v13.3Steps overview:Download all components Install ONE Desktop Launch ONE Desktop and create the MDM Banking example project Place the files in the right structure Modify a bunch of files so they are working on MacOS Configure the Mac permissions to run things Launch ONE Desktop and create the MDM Banking example project Launch it all (keycloak, postrgres), one by oneDownload:Builds (Latest versions here): MDM: MDM-Assembly-13.3.2-final-linux.zip ONE Desktop: Ataccama-one-desktop-13.2.1.final-macos.dmg ONE Runtime: Server-assembly-13.3.2-final.tar Plugins (Latest versions here, the official Postgres version for Linux missing for now): Postgres: pgsql.zip Keycloak: https://ataccama.s3.amazonaws.com/Frances/Ataccama-one-desktop-13.0.0-GA_keycloak_12_0_4_plugin.zip Your license (.plf file)For latest versions of the data (optional):ataccama.json permissions file, found hereInstall ONE Desktop:Install One Desktop (Ataccama-one-desktop-13.2.1.final-macos.dmg) Rename

Hi Team, I have bunch of queries sitting in my Oracle tables(queries in one of the column). I am trying to see if I can read the oracle table as source and use the column values(which has sql queries ) to be sent to database & get return value. Is there a component that can help me achieve that? jdbc reader, sql execute components expect you to hard code the query. But in my case the query will come as value from one the table.  eg) Col1 Col2 1 select count(*) from tab A 2 select count(*) from tab B In the above example I wanted to use col2 as input and pass the values from Col2 to DB and assign the output to next transformation. Hope its clear.

Copy Columns (Instance Layer)Copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, there must exist a relationship definition between entities. After it is defined, it is possible to define the direction of copy columns: parent to child or child to parent.In the following example we will create a matching rule (firstn+lastn+bd+con) for Party entity (Parent entity) with same First Name, Last Name, Birthday, Contact (Child entity). Notice that the contact information is stored in the Contact entity which is a child to Party entity. In that case we would need to transfer the information (contact) into the party entity in order to use it in the matching on that particular entity. In that case we will use Copy Columns functionality to move and store associated contacts of the party record directly in the party entity.The relationship of Party (Parent) to Contact (Child) is 1:N, which means that a particular party can have 1 or m

I have a rule binded to 2 columns in a catalog item.By mistake i have changed the data type of those columns without unbinding the rule.Now i am unable to unbind thise rules from that column.i am getting error while saving or publishing the project : server refused sent data as invalid. Please guide me,what i will do here

​How to tune/optimize a DQC load plan?In day-to-day life, the consultants may come across several performance issues of various magnitude in their load plans at some point in the projects. There are several optimization tips mentioned below for tuning the long running plans.Check the memory allocationDQC plans store the data while processing, which means that the more input data there is, the more memory DQC will require. Therefore the intensive executions can result in a memory overflow error if the default memory allocated for DQC is consumed. In such cases, an increase of memory could present a solution.Related Errors: OutOfMemoryError: Java Heap Space or GC Overhead Limit Exceeded... java.lang.OutOfMemoryError: unable to create new native thread... The memory allocation can be done in several ways,Increasing the memory of the DQC IDE(Desktop) Increasing the memory allocation to specific plan in the IDE Increasing the memory allocation for the entire project in IDE Increasing the me

First part of the tutorial - Ataccama Admin Centre: ServerConfig and RuntimeConfig for beginners (Self-managed installations) 1/2Use cases Admin Centre Security 1. ServerConfig 2. RuntimeConfig Online service endpoint 1. ServerConfig for OnlineServicesComponent 2. RuntimeConfig for pathvar and rdmapp 3. ServerConfig for Http dispatcher Workflow and Scheduler ServerConfig for Scheduler component configuration ServerConfig for Workflow component configuration RuntimeConfig for workflow and components Security for workflow by role Deploy your files and test them locally Check the UI and Server Health Status Test User Permissions Deploy to serverUse casesNow let’s consider some use cases and see how we can set up the Admin Centre to meet our needs.Security on the Admin Centre UI Online service endpoint Schedulers, workflows and plans / componentsAdmin Centre SecuritySecurity on the Admin Centre UI itself is taken care of by the Http dispatcher Component. The one-executor.serv

If you would like to filter e.g. business terms, use:$type == 'businessTerm'Switch to Advanced filtering with AQL (Ataccama Query Language)Terms list before applying a filterFiltering resultsLink to documentation:Searching Using AQL Expressions

We are trying to connect Ataccama with Python code. Could you please help us with what are the possible ways to connect with python, it would be really great if you can share us with any reference links or docs. Thanks in advance.

Hi  Folks, I’m super-new to Ataccama - in fact I haven’t even had a hands-on yet. Curious  to know whether we can embed content from Ataccama onto Confluence pages via an API or  something? Anyone done this before? Mick

Every step that allows for the input of an expression shows a button called Template.... This button can be used to pre-fill the Expression field with one from an out-of-the-box list.You can create your own expressions for common or complex repeatable operations, follow these steps:Go to Window in the top navigation menu and select Preferences Navigate the menu to Ataccama DQC and go to Expression Editor and select TemplatesChoose between Expression Templates and RegEx TemplatesSelect FolderClick on Add Template..Fill in Name, Expression and DescriptionConfirm with OK

As part of the continues integration development, I would like to implement Unit testing automation, this topic is to discuss some ideas of how to do that in Atacama, I have some specific doubts like:Architecture suggested for unit testing automation  Database approach for unit testingThanks in advance

When using Run Shell Script task in a Workflow, be careful that this task uses a different expression markup: %expression()% Link to documentation: Run Shell Script

Hello there,Please, when sending data about a person to MDM, can we differentiate this data between VERIFIED and UNVERIFIED?I mean, unverified DATA should be stored, but it should never overwrite the existing verified data in MDM.Example:User ELON MUSK, 60 years old, elon@tesla.com, California, etc is created as verified data in MDM and master-ID ABC is assigned to his record. Months later someone tries to create a profile and the data is not yet verified, that data would be ELON MUSKK, 58 years old, elon@tesla.com, Texas.   The end result coming from from a “GET” should be ELON MUSK, 60 years old, elon@tesla.com, California, master-ID ID ABCIt would be OK to MDM store that UNVERIFIED Data somewhere, but it should NEVER overwrite VERIFIED data nor consider UNVERIFIED information as correct data to be returned to MDM callers/consumers.I know that other MDMs keep unverified data separated of VERIFIED data, I would like to know what is the recommended way to do this in Ataccama.Any help w

Any one can help to create this template using version 13.6?  It show the items clearly.  Our users can know the data presentation without explanation.  However, I can find this template in our current version.

Any best practices for using the Purpose function in Ataccama One (webapp)? I see that it is recommended for monitoring projects but when I create a monitoring project I want to be able to search for that tag. Any tips for searching just based on the purpose tag? I’m very new to Ataccama so any and all tips would be appreciated!

I’m trying to read the file name of a file so I can write it in an email within a Workflow. I’m using the “Operate On File” task. The Operation I selected is “INFO”. I then point to the Source File. Next I’m trying to determine which variable I use to supply to the Send Email task. The Operate On File Documentation says you can use task variables and the example they give is “name”, “localName”, “size”, and “timestamp”. When I try using these variables in the Send Email task they just come across as blank in the email (variables appear to be null). Here is the body of the email I’m using: Name = ${name}localName = ${localName} Two questions:Does any one happen to know what I’m doing wrong in above?Additionally is there a way to pull out a file name within a Plan. I’m using the “Excel File Reader” step. I want to pass along the file name in the flow so I can use it later on. Thanks!

Hi, I’ve just started data profiling with Ataccama One so please bear with me.I have data with technical IDs (e.g. “000123456”, “009876543”,...) in a csv that I imported. The IDs are however interpreted as a number.How exactly can I change the attribute type to text? I have done so when importing the file: but it seems to go to numbers nonetheless.The original file has comma-separated values that are all between double quotes. Thanks for any help! edit: typo

I am getting the error “Column Name cannot be empty” under StringValue. When I click on the drop down, there’s no options available. 

What is LDAP?LDAP is directory that stores companies usernames, passwords, email addresses, connection information and much more. It is usually used due to it's ability to allow hundreds of active connections at any given point.In terms of configuration and connectivity, in Ataccama we authenticate/authorize through a 3rd party tool called Keycloak. Items to consider prior to connecting to LDAP - Best PracticeYou first need to understand how your organizations Active Directory is setup. Some pre-requirements needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, LDAP groups is a way where we can map Ataccama Roles to a single group. This will heavily assist in ease of maintenance going forward and not leave your devops team stuck doing manual work such as adding/removing new users each time someone is added/removed from LDAP. Once you have dec

Hi, is there a method to make a “wait n seconds” between 2 steps?Thanks a lot