Skip to main content

Learn & Share

Discuss data, share best practices, learn, and grow with Ataccama's Community.

834 Topics
A
Anonymous
published in Data Quality & Catalog ⚙️
#2 Add a Data Source to the CatalogGet certified with Ataccama

In the second article of Get certified with Ataccama series, we will focus on Adding a Data Source to the Catalog. 👉 First article of the series - #1 Adding Data Sources 🔎 LET’S TEST YOUR SKILLSUsing the ONE Web App, in the ONE Knowledge Catalog module, add your database as a data source. Using the ONE Web App, in the ONE Knowledge Catalog module, add the table you created in ONE Desktop application to the Catalog Items. 💡 How to do it? Go to ONE Web App - Data Catalog - Sources and click on CreateFill in Name and Description (if needed)Fill in Connection details - Select connection type, Name and JDBCAdd Credentials, select Credential type, fill in required information and Test ConnectionSave and publishNavigate to Connections in and click on BrowseSelect your tableFrom Profile menu select ProfileAfter the profiling is done, check that your table is in the Catalog items list Do you have any question or tips&tricks? Let us know in comments section! 

1240
A
S
Data Voyager
Sunil AroraData Voyager
asked in Data Quality & Catalog ⚙️
Salesforce Connection

Has anyone successfully connected to Salesforce environments (prod/test/dev etc.) thru One Desktop (13.7.1).  Thanks

882
Catherine
Data Pioneer
P
Data Voyager
PCappData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Problems connecting to a MS SQL Server

We are having a difficult time connecting a MS SQL database to Ataccama

1733
Catherine
Data Pioneer
A
Anonymous
published in Data Quality & Catalog ⚙️
Automate the Delivery of High Quality DataWhitepaper

What makes manual data quality so hard to scale in large organization? Is there a step by step approach to automating data quality? Why is an automated DQM system the future of enterprise data quality?You can find an answer to all these questions (and many more!) in our latest whitepaper.👉 Download it here

430
A
A
Universe Traveller
Ajeesh_GUniverse Traveller
asked in Data Quality & Catalog ⚙️
Connect with Share point

Can we connect Ataccama to Sharepoint xl file? if yes, how can I achieve this? I am new this forum and new to ataccama.

1291
A
Universe Traveller
J
Data Voyager
justinpData Voyager
asked in Data Quality & Catalog ⚙️
Has anyone else experienced antivirus interference with DQ Analyzer 11?

My organization runs McAfee antivirus.  I’ve just tried out the DQ Analyzer 11 tool and really like it, but it takes forever to start up or perform any actions.  Just starting up, the antivirus pegs 1 CPU for about 30 minutes before the application starts.  And then various actions like starting a new profile or clicking a menu item will also be delayed by several minutes at least.We’ve tried excluding the install directory, but it hasn’t helped.  Any ideas?  Has anyone else run into this issue before, and how did you solve it? 

442
J
Data Voyager
D
Ataccamer
David SuenAtaccamer
published in Master Data Management & Reference Data Management 🛰️
How to Set up Auto Login for Users in KeycloakTutorial

This is needed when a user would like to go directly to the platform and skip the “login with Azure AD SSO or OKTA” page. This can be turned on (or off) by following the below directions: Login to the Keycloak Administration Console Ensure the Ataccamaone Realm is selected In the left-hand-menu, Click on Authentication Under Flows select Browser  On the Identity Provider Redirector line, click on Actions and select Config a. If you would like to disable (turn off) the auto login - we can select the ‘disabled’ radio button on the Identity provider redirector line Enter the name of the Identity provider, e.g. saml, in both the Alias and Default Identity Provider boxes Select Save

14810
D
Ataccamer
alexAguilarMx
Data Pioneer
alexAguilarMxData Pioneer
asked in Master Data Management & Reference Data Management 🛰️
Does it good idea use MDC read inside export operations?

Hi,I am wondering to know if is good idea:use MDC Reader step for delta export operations where we need to complement data from entities of other layer   oradd another Data Sources with full mode of this other entity, then use a Join step to get the information required.  Both are feasible, not sure what is better talking about batch processing, performance, etc.

1495
alexAguilarMx
Data Pioneer
S
Data Voyager
sd_2000Data Voyager
asked in Data Quality & Catalog ⚙️
Custom Data quality check

Can we create custom check on data as per our need and get stats out of it ??for example.. I want to check whether colA has values between list [‘A’,’AB’,’O’,’B’] and stats about the values that are present in data.

811
S
Data Voyager
K
Data Voyager
KostroskData Voyager
asked in Data Quality & Catalog ⚙️
Can DQ Analyzer connect to Redshift?

I was thinking that this should be possible with a hack to the postgreSQL jdbc configuration using the redshift jdbc driver. Does anyone know if this is possible or if there is another avenue?Edit: I found the connection after the install and properly configured the Redshift driver and connection settings.

381
K
Data Voyager
J
Data Voyager
joedysonData Voyager
asked in Master Data Management & Reference Data Management 🛰️
How emit notification to kafka?

I need the party entity (MDC -ataccama one) to send a notification to kafka when insert or update

733
J
Data Voyager
J
Data Voyager
joedysonData Voyager
posted in Master Data Management & Reference Data Management 🛰️
Problem with publish message in kafka

When I am publish message in kafka, it throw exception error I am using ataccama 12.7 com.ataccama.nme.core.NmeException: java.lang.NullPointerException at com.ataccama.nme.engine.event.handler.SimpleEventProcessor$NmeSimpleEventProcRuntime.publish(SimpleEventProcessor.java:66) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.mainLoop(NmeEventPublisherBase.java:210) at com.ataccama.nme.engine.event.handler.NmeEventPublisherBase$Runner.run(NmeEventPublisherBase.java:172) at java.base/java.lang.Thread.run(Unknown Source)Caused by: java.lang.NullPointerException at com.ataccama.dqc.streaming.io.writer.KafkaWriter.validate(KafkaWriter.java:130) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursiveInternal(ValidationUtils.java:158) at com.ataccama.dqc.model.validation.ValidationUtils.validateRecursive(ValidationUtils.java:127) at com.ataccama.dqc.tasks.common.config.ValidatingStepConfigBase.validateElement(ValidatingStepConfigBase.java:39) at com.atacca

681
J
Data Voyager
A
Data Voyager
ADB59Data Voyager
asked in Data Quality & Catalog ⚙️
RFC 5322 Internet Message Format incompatible with DQC

Hi,We have a problem in a workflow in DQC, with task “Send Mail”, because the generated message is incompatible with the internet message format RFC 5322.When there are at least 2 recipients, in the generated message, the header “to” is duplicate as many times as there are recipients, but the RFC 5322 standard does not allow this.How to correct this problem ?Thanks for your help.

1013
A
Data Voyager
M
Data Voyager
mukeshData Voyager
asked in Data Quality & Catalog ⚙️
How to get Aggregate functions in DQ Analyzer

I have Ataccama 12.x version and I do not see their Aggregate functionPlease help, how to get it enabled or patch it Thanks,Mukesh

431
M
Data Voyager
M
Data Pioneer L2
MayKwokData Pioneer L2
published in Data Quality & Catalog ⚙️
How to configure team based permissions in v13.8Tutorial

Full support for team based permissions in Ataccama ONE is coming in v14. However, you may need to implement team based permissions sooner than v14. This document is a guide to how you could achieve this in v13.8. BackgroundPermissions in ONE can be configured as role based or user based. In most cases, configuration by role is preferred as it will usually be a one-time set up, where the role only needs to be configured once in the beginning, then any users allocated to that role will inherit all permissions on role.The out of the box roles in ONE have some capabilities already allocated. These are auto-sharing permissions, which allows the role to have access to all items of that entity. E.g. Your role could have the capability of Rule Owner. This means the user will be automatically be owner of all rules that would ever be created on the platform.But what if your user should only be owner to rules (or any other entities) within their team / division / domain?As it stands in v13.8, th

2300
M
Data Pioneer L2
anna.spakova
Ataccamer
anna.spakovaAtaccamer
published in Master Data Management & Reference Data Management 🛰️
How to create relationships between different entities in Gen 2Tutorial

 This article will explain how you can build a relationship between different entities in Gen 2. Out-of-the-box you can have relationship between the same entities - e.g. Catalog Items. But what if you would like to define a relation between different entities? E.g. Source and Report (custom entity)? Following are steps that will explain how to do it. AssumptionsYou have your two entities between which you would like to have a relationship. For this article we will use source (default entitiy) and report (custom). Create the relationship entityFirst, go to the Global Settings (or Organization) → Metadata Model and create a new entity that will store the relationships. Name it e.g. sourceReportRelationship. It will have following properties:order (Integer) source (Referenced Object: source) target (Referenced Object: report) type (Referenced Object: relationshipType). Put the new relationship entity in metadata entityUnder Metadata model, navigate to the entity metadata. Add a new prope

2080
anna.spakova
Ataccamer
A
Anonymous
published in Data Quality & Catalog ⚙️
State of Data Quality report is here!Report

Our new State of Data Quality report is HERE!  We've partnered with TDWI to survey 1,000+ executives and business users and understand key factors that help today's companies succeed with data quality.Access key findings from this year’s survey in the e-book, including: The current state of enterprise data quality management in 2022 Common challenges organizations encounter when managing their data quality What’s required to establish a mature enterprise DQ management practice Recommendations from TDWI analysts for data management professionals... and more! Download the free report here: https://www.ataccama.com/ebook/2022-state-of-data-quality

310
A
A
Anonymous
published in Data Quality & Catalog ⚙️
How Ataccama ONE uses Snowpark data frames to automate profiling

Ataccama is known and appreciated by our customers for complex, big deployments where massive amounts of data are processed. We were one of the first to support MapReduce and Spark as native pushdown and enable distributed processing across different cluster nodes.Ataccama’s engine works with the major of enterprise data sources and processes them to provide valuable insight, such as statistics of the data, detection of domains, and data quality.To provide the best experience for our customers that use Snowflake, we developed a tighter integration between our technologies. Specifically, we use the Snowflake data processing capabilities in a pushdown manner which means that data is processed directly on Snowflake.The key advantages of this integration are:Lower data transfer costs for the client. Snowflake’s computational capabilities deliver results faster because of reduced IO operations. Customers don’t incur additional costs to maintain external systems for Big Data. Improved securi

3852
A
S
Data Pioneer L2
Siva_MadhavanData Pioneer L2
asked in Master Data Management & Reference Data Management 🛰️
Compare values between data in different schemas

Hi Team, I have few data loads that read data using API’s and writing to Snowflake tables. Often the data is incomplete & inconsistent. Im checking if I can use Attacama to compare the data between different snowflake schema’s : eg) Raw vs Curated? I understand we can write SQL Execute (select col1 from Tab A minus select col1 from Tab b and create a plan. But i have 500+ jobs and needs a lot of plans to be created. Looking for correct way of doing this?  

1372
S
Data Pioneer L2
R
Data Voyager
RoberTallitschData Voyager
asked in Data Quality & Catalog ⚙️
Creating flat file using Ataccama One Desktop where records failed business rule

We created a number of data quality business rules that will fail records from the source file we are reading.  1.5 Million records.How do you port the failed records to a flat file?  There is no drill down in One Desktop so we are having issues getting business users approval on code without seeing the failed records?

1332
S
Data Pioneer L2
john.efechaobor
Ataccamer
john.efechaoborAtaccamer
published in Master Data Management & Reference Data Management 🛰️
Running MDM v13.x Locally on MacOSTutorial

Tested on v13.3Steps overview:Download all components Install ONE Desktop Launch ONE Desktop and create the MDM Banking example project Place the files in the right structure Modify a bunch of files so they are working on MacOS Configure the Mac permissions to run things Launch ONE Desktop and create the MDM Banking example project Launch it all (keycloak, postrgres), one by oneDownload:Builds (Latest versions here): MDM: MDM-Assembly-13.3.2-final-linux.zip ONE Desktop: Ataccama-one-desktop-13.2.1.final-macos.dmg ONE Runtime: Server-assembly-13.3.2-final.tar Plugins (Latest versions here, the official Postgres version for Linux missing for now): Postgres: pgsql.zip Keycloak: https://ataccama.s3.amazonaws.com/Frances/Ataccama-one-desktop-13.0.0-GA_keycloak_12_0_4_plugin.zip Your license (.plf file)For latest versions of the data (optional):ataccama.json permissions file, found hereInstall ONE Desktop:Install One Desktop (Ataccama-one-desktop-13.2.1.final-macos.dmg) Rename

3100
john.efechaobor
Ataccamer
S
Data Pioneer L2
Siva_MadhavanData Pioneer L2
asked in Master Data Management & Reference Data Management 🛰️
How to Execute SQL Query using Components

Hi Team, I have bunch of queries sitting in my Oracle tables(queries in one of the column). I am trying to see if I can read the oracle table as source and use the column values(which has sql queries ) to be sent to database & get return value. Is there a component that can help me achieve that? jdbc reader, sql execute components expect you to hard code the query. But in my case the query will come as value from one the table.  eg) Col1 Col2 1 select count(*) from tab A 2 select count(*) from tab B In the above example I wanted to use col2 as input and pass the values from Col2 to DB and assign the output to next transformation. Hope its clear.

2775
S
Data Pioneer L2
konstantinos.patikas
Ataccamer
konstantinos.patikasAtaccamer
posted in Master Data Management & Reference Data Management 🛰️
Copy Columns - how to use data of related entities in the parent entity Matching

Copy Columns (Instance Layer)Copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, there must exist a relationship definition between entities. After it is defined, it is possible to define the direction of copy columns: parent to child or child to parent.In the following example we will create a matching rule (firstn+lastn+bd+con) for Party entity (Parent entity) with same First Name, Last Name, Birthday, Contact (Child entity). Notice that the contact information is stored in the Contact entity which is a child to Party entity. In that case we would need to transfer the information (contact) into the party entity in order to use it in the matching on that particular entity. In that case we will use Copy Columns functionality to move and store associated contacts of the party record directly in the party entity.The relationship of Party (Parent) to Contact (Child) is 1:N, which means that a particular party can have 1 or m

820
konstantinos.patikas
Ataccamer
S
Data Voyager
SmrutiData Voyager
asked in Data Quality & Catalog ⚙️
error while publishing the project

I have a rule binded to 2 columns in a catalog item.By mistake i have changed the data type of those columns without unbinding the rule.Now i am unable to unbind thise rules from that column.i am getting error while saving or publishing the project : server refused sent data as invalid. Please guide me,what i will do here

361
anna.spakova
Ataccamer
V
Ataccamer
VaniAtaccamer
posted in Data Quality & Catalog ⚙️
How to tune/optimize a DQC load plan?

​How to tune/optimize a DQC load plan?In day-to-day life, the consultants may come across several performance issues of various magnitude in their load plans at some point in the projects. There are several optimization tips mentioned below for tuning the long running plans.Check the memory allocationDQC plans store the data while processing, which means that the more input data there is, the more memory DQC will require. Therefore the intensive executions can result in a memory overflow error if the default memory allocated for DQC is consumed. In such cases, an increase of memory could present a solution.Related Errors: OutOfMemoryError: Java Heap Space or GC Overhead Limit Exceeded... java.lang.OutOfMemoryError: unable to create new native thread... The memory allocation can be done in several ways,Increasing the memory of the DQC IDE(Desktop) Increasing the memory allocation to specific plan in the IDE Increasing the memory allocation for the entire project in IDE Increasing the me

5350
V
Ataccamer

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings