Learn & Share

I need the party entity (MDC -ataccama one) to send a notification to kafka when insert or update

When I am publish message in kafka, it throw exception error I am using ataccama 12.7 com.ataccama.nme.core.NmeException: java.lang.NullPointerException at com.ataccama.nme.engine.event.handler.SimpleEventProcessor$NmeSimpleEventProcRuntime.publish(SimpleEventProcessor.java:66) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.mainLoop(NmeEventPublisherBase.java:210) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.run(NmeEventPublisherBase.java:172) at java.base/java.lang.Thread.run(Unknown Source)Caused by: java.lang.NullPointerException at com.ataccama.dqc.streaming.io.writer.KafkaWriter.validate(KafkaWriter.java:130) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursiveInternal(ValidationUtils.java:158) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursive(ValidationUtils.java:127) at com.ataccama.dqc.tasks.common.config.ValidatingStepConfigBase.validateElement(ValidatingStepConfigBase.java:39) at com.atacca

Hi,We have a problem in a workflow in DQC, with task “Send Mail”, because the generated message is incompatible with the internet message format RFC 5322.When there are at least 2 recipients, in the generated message, the header “to” is duplicate as many times as there are recipients, but the RFC 5322 standard does not allow this.How to correct this problem ?Thanks for your help.

I have Ataccama 12.x version and I do not see their Aggregate functionPlease help, how to get it enabled or patch it Thanks,Mukesh

Full support for team based permissions in Ataccama ONE is coming in v14. However, you may need to implement team based permissions sooner than v14. This document is a guide to how you could achieve this in v13.8. BackgroundPermissions in ONE can be configured as role based or user based. In most cases, configuration by role is preferred as it will usually be a one-time set up, where the role only needs to be configured once in the beginning, then any users allocated to that role will inherit all permissions on role.The out of the box roles in ONE have some capabilities already allocated. These are auto-sharing permissions, which allows the role to have access to all items of that entity. E.g. Your role could have the capability of Rule Owner. This means the user will be automatically be owner of all rules that would ever be created on the platform.But what if your user should only be owner to rules (or any other entities) within their team / division / domain?As it stands in v13.8, th

 This article will explain how you can build a relationship between different entities in Gen 2. Out-of-the-box you can have relationship between the same entities - e.g. Catalog Items. But what if you would like to define a relation between different entities? E.g. Source and Report (custom entity)? Following are steps that will explain how to do it. AssumptionsYou have your two entities between which you would like to have a relationship. For this article we will use source (default entitiy) and report (custom). Create the relationship entityFirst, go to the Global Settings (or Organization) → Metadata Model and create a new entity that will store the relationships. Name it e.g. sourceReportRelationship. It will have following properties:order (Integer) source (Referenced Object: source) target (Referenced Object: report) type (Referenced Object: relationshipType). Put the new relationship entity in metadata entityUnder Metadata model, navigate to the entity metadata. Add a new prope

Ataccama is known and appreciated by our customers for complex, big deployments where massive amounts of data are processed. We were one of the first to support MapReduce and Spark as native pushdown and enable distributed processing across different cluster nodes.Ataccama’s engine works with the major of enterprise data sources and processes them to provide valuable insight, such as statistics of the data, detection of domains, and data quality.To provide the best experience for our customers that use Snowflake, we developed a tighter integration between our technologies. Specifically, we use the Snowflake data processing capabilities in a pushdown manner which means that data is processed directly on Snowflake.The key advantages of this integration are:Lower data transfer costs for the client. Snowflake’s computational capabilities deliver results faster because of reduced IO operations. Customers don’t incur additional costs to maintain external systems for Big Data. Improved securi

Hi Team, I have few data loads that read data using API’s and writing to Snowflake tables. Often the data is incomplete & inconsistent. Im checking if I can use Attacama to compare the data between different snowflake schema’s : eg) Raw vs Curated? I understand we can write SQL Execute (select col1 from Tab A minus select col1 from Tab b and create a plan. But i have 500+ jobs and needs a lot of plans to be created. Looking for correct way of doing this?  

We created a number of data quality business rules that will fail records from the source file we are reading.  1.5 Million records.How do you port the failed records to a flat file?  There is no drill down in One Desktop so we are having issues getting business users approval on code without seeing the failed records?

Tested on v13.3Steps overview:Download all components Install ONE Desktop Launch ONE Desktop and create the MDM Banking example project Place the files in the right structure Modify a bunch of files so they are working on MacOS Configure the Mac permissions to run things Launch ONE Desktop and create the MDM Banking example project Launch it all (keycloak, postrgres), one by oneDownload:Builds (Latest versions here): MDM: MDM-Assembly-13.3.2-final-linux.zip ONE Desktop: Ataccama-one-desktop-13.2.1.final-macos.dmg ONE Runtime: Server-assembly-13.3.2-final.tar Plugins (Latest versions here, the official Postgres version for Linux missing for now): Postgres: pgsql.zip Keycloak: https://ataccama.s3.amazonaws.com/Frances/Ataccama-one-desktop-13.0.0-GA_keycloak_12_0_4_plugin.zip Your license (.plf file)For latest versions of the data (optional):ataccama.json permissions file, found hereInstall ONE Desktop:Install One Desktop (Ataccama-one-desktop-13.2.1.final-macos.dmg) Rename

Hi Team, I have bunch of queries sitting in my Oracle tables(queries in one of the column). I am trying to see if I can read the oracle table as source and use the column values(which has sql queries ) to be sent to database & get return value. Is there a component that can help me achieve that? jdbc reader, sql execute components expect you to hard code the query. But in my case the query will come as value from one the table.  eg) Col1 Col2 1 select count(*) from tab A 2 select count(*) from tab B In the above example I wanted to use col2 as input and pass the values from Col2 to DB and assign the output to next transformation. Hope its clear.

Copy Columns (Instance Layer)Copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, there must exist a relationship definition between entities. After it is defined, it is possible to define the direction of copy columns: parent to child or child to parent.In the following example we will create a matching rule (firstn+lastn+bd+con) for Party entity (Parent entity) with same First Name, Last Name, Birthday, Contact (Child entity). Notice that the contact information is stored in the Contact entity which is a child to Party entity. In that case we would need to transfer the information (contact) into the party entity in order to use it in the matching on that particular entity. In that case we will use Copy Columns functionality to move and store associated contacts of the party record directly in the party entity.The relationship of Party (Parent) to Contact (Child) is 1:N, which means that a particular party can have 1 or m

I have a rule binded to 2 columns in a catalog item.By mistake i have changed the data type of those columns without unbinding the rule.Now i am unable to unbind thise rules from that column.i am getting error while saving or publishing the project : server refused sent data as invalid. Please guide me,what i will do here

​How to tune/optimize a DQC load plan?In day-to-day life, the consultants may come across several performance issues of various magnitude in their load plans at some point in the projects. There are several optimization tips mentioned below for tuning the long running plans.Check the memory allocationDQC plans store the data while processing, which means that the more input data there is, the more memory DQC will require. Therefore the intensive executions can result in a memory overflow error if the default memory allocated for DQC is consumed. In such cases, an increase of memory could present a solution.Related Errors: OutOfMemoryError: Java Heap Space or GC Overhead Limit Exceeded... java.lang.OutOfMemoryError: unable to create new native thread... The memory allocation can be done in several ways,Increasing the memory of the DQC IDE(Desktop) Increasing the memory allocation to specific plan in the IDE Increasing the memory allocation for the entire project in IDE Increasing the me

First part of the tutorial - Ataccama Admin Centre: ServerConfig and RuntimeConfig for beginners (Self-managed installations) 1/2Use cases Admin Centre Security 1. ServerConfig 2. RuntimeConfig Online service endpoint 1. ServerConfig for OnlineServicesComponent 2. RuntimeConfig for pathvar and rdmapp 3. ServerConfig for Http dispatcher Workflow and Scheduler ServerConfig for Scheduler component configuration ServerConfig for Workflow component configuration RuntimeConfig for workflow and components Security for workflow by role Deploy your files and test them locally Check the UI and Server Health Status Test User Permissions Deploy to serverUse casesNow let’s consider some use cases and see how we can set up the Admin Centre to meet our needs.Security on the Admin Centre UI Online service endpoint Schedulers, workflows and plans / componentsAdmin Centre SecuritySecurity on the Admin Centre UI itself is taken care of by the Http dispatcher Component. The one-executor.serv

If you would like to filter e.g. business terms, use:$type == 'businessTerm'Switch to Advanced filtering with AQL (Ataccama Query Language)Terms list before applying a filterFiltering resultsLink to documentation:Searching Using AQL Expressions

We are trying to connect Ataccama with Python code. Could you please help us with what are the possible ways to connect with python, it would be really great if you can share us with any reference links or docs. Thanks in advance.

Hi  Folks, I’m super-new to Ataccama - in fact I haven’t even had a hands-on yet. Curious  to know whether we can embed content from Ataccama onto Confluence pages via an API or  something? Anyone done this before? Mick

Every step that allows for the input of an expression shows a button called Template.... This button can be used to pre-fill the Expression field with one from an out-of-the-box list.You can create your own expressions for common or complex repeatable operations, follow these steps:Go to Window in the top navigation menu and select Preferences Navigate the menu to Ataccama DQC and go to Expression Editor and select TemplatesChoose between Expression Templates and RegEx TemplatesSelect FolderClick on Add Template..Fill in Name, Expression and DescriptionConfirm with OK

As part of the continues integration development, I would like to implement Unit testing automation, this topic is to discuss some ideas of how to do that in Atacama, I have some specific doubts like:Architecture suggested for unit testing automation  Database approach for unit testingThanks in advance

When using Run Shell Script task in a Workflow, be careful that this task uses a different expression markup: %expression()% Link to documentation: Run Shell Script

Hello there,Please, when sending data about a person to MDM, can we differentiate this data between VERIFIED and UNVERIFIED?I mean, unverified DATA should be stored, but it should never overwrite the existing verified data in MDM.Example:User ELON MUSK, 60 years old, elon@tesla.com, California, etc is created as verified data in MDM and master-ID ABC is assigned to his record. Months later someone tries to create a profile and the data is not yet verified, that data would be ELON MUSKK, 58 years old, elon@tesla.com, Texas.   The end result coming from from a “GET” should be ELON MUSK, 60 years old, elon@tesla.com, California, master-ID ID ABCIt would be OK to MDM store that UNVERIFIED Data somewhere, but it should NEVER overwrite VERIFIED data nor consider UNVERIFIED information as correct data to be returned to MDM callers/consumers.I know that other MDMs keep unverified data separated of VERIFIED data, I would like to know what is the recommended way to do this in Ataccama.Any help w

Any one can help to create this template using version 13.6?  It show the items clearly.  Our users can know the data presentation without explanation.  However, I can find this template in our current version.