Skip to main content

Product Updates
Events
Training Resources
Groups

Home
Learn & Share

Learn & Share

Discuss data, share best practices, learn, and grow with Ataccama's Community.

803 Topics

Recently active

Most replies Most views Newest first

cbre_jonathanData Voyager

asked in Data Quality & Governance ⚙️

Customize Business Glossary Term View

Hi,I’m trying to modify our view of our business terms - currently we have a lot of information showing in the default view that our end users may not care about. What I really want to do is modify the view, so that it only shows Name, Abbreviation, Description and Synonym. I have figured out how to add the columns, but I can’t figure out how to remove unwanted columns. For reference here is the default screen layout in Ataccama One via the Debugging Tools. {"_type": "entity.page.listing","children": {"_type": "relationships.hierarchyListing","enableSelection": true,"columns": {"type": {"dataPath": [],"name": "Type","after": "name","renderer": {"_type": "glossary.termType.tableCell"}}}}}

CatherineData Pioneer

asked in Data Quality & Governance ⚙️

Ataccama Web - User Activity Logging Reporting

Use Case: As part of the Data Governance Program KPIs reporting, we need to report on unique user usage count and details of who within the Ataccama Web on a monthly, month-to-date, and year-to-date. Can this be easily captured using OpenSearch Dashboards historically or elsewhere?

MayKwokData Pioneer L2

published in Master Data Management & Reference Data Management 🛰️

How to generate a Basic Auth headerTutorial

Basic Auth headers are frequently used as part of communication between applications, e.g. as part of API calls.Common use cases in the Ataccama context include (and not limited to):Graphql call to ONE application API call to an online service set up on a DQC / runtime / orchestration serverOption 1: Do it in a ONE Desktop plan Create a new plan Create a step that evaluates expressions, e.g. Alter Format Go into the expression window and put this expression in. The username and password are separated with a colon: coding.toBase64('username:password') Click Evaluate Copy the result string Your header is going to be Basic dXNlcm5hbWU6cGFzc3dvcmQ=Option 2: Use an online base64 encode websiteIf you are building something to integrate with Ataccama but you don’t actually have Ataccama ONE Desktop installed…. Go to a website that offers this conversion (e.g. link. Warning: this site is not affiliated with Ataccama) Put in your username and password in this format: username:password, se

MayKwokData Pioneer L2

published in Data Quality & Governance ⚙️

Parse JSON value in (Databricks) SQLBest Practice

I’ve been working on a use case recently which I think may be of interest to the community here.Some reference data is stored in a single table DATA_KEY_ID column tells us what reference data set it is DATA_VAL column stores the actual reference data in the format of a json array Goal is to extract the values from the json array, and use it as a lookup item in Ataccama ONE, which in turn is used in Data Quality Validation rulesTo start with, you need to have some preview access to the data and understand the schema of the json. For the example in this post, I have very simple “id” and “name” properties for my json. You will need to analyse that data before starting this, so you know what to put into the Json parser, or which values you want to get out from the json object. Solution 1: Virtual Catalog Item (VCI)Bring the reference data table into Ataccama ONE Build a VCI, with a Catalog Item Reader of the reference data table Use a filter step to filter the exact set I need (it will al

kriangsaksumthongData Voyager

posted in Data Quality & Governance ⚙️

importing DQ rules fail.

I’m having this import error note, how to fix this? thank you.

bazData Voyager

asked in Master Data Management & Reference Data Management 🛰️

RDM - Permissions to stop a role editing published rows

Hi - I need to set the permissions for a role in RDM that allows users within this role to create new rows, edit newly created rows and move them to publish. Once a row has been either Published or ‘Awaiting Confirmation’ the user should not be able to edit or delete the row.Is there a way of doing this in RDM? baz

KarthikaSPillaiData Pioneer

asked in Data Quality & Governance ⚙️

How to stop monitoring project?

I tried to run monitoring project at morning, but it is forever running. Please let me a know a way to stop it. Thanks!@anna.spakova @Catherine

sxk020501Data Voyager

posted in Master Data Management & Reference Data Management 🛰️

Dynamically provisioning Ataccama environment for supporting automated unit testing capability

I’d like to implement a dynamically provisioning Ataccama environment for supporting automated unit testing capability. I.eBuilding a Container image of Ataccama software stored in a private cloud and. Developers can rapidly iterate code, commit in a repo, CI triggers application build, container image build and Automated unit testing of new features via Azure pipelines.And It seems Ataccama is working on a Containerized deployments soon according to their web site,Does anybody know the timeline of when this feature can be available to do the above goals? Also can we use it as a preview for unit testing environment?

matthewtaylorukAtaccamer

posted in Data Quality & Governance ⚙️

Data Quality and Integration Scenarios

IntroductionThis post was created to answer the following questions:When moving data through data pipelines, how can the Ataccama ONE DQ platform support Data Integration platforms other than its own. How do other Integration Platforms access the DQ rules? How do other Integration Platforms flag records for Data Stewards to review?DQ platform in context of integration processesThere are three key patterns that are typically used for integration of the ETL process with Data Quality:Option 1The integration platform calls Ataccama DQ rules exposed as real-time API, sending in batches or micro-batches of data for evaluation.Option 2The Integration platform loads data into a staging/landing layer and triggers (via API call) a batch process in Ataccama, Ataccama evaluates/improves the data and populates the cleansed layer in the data lake, including DQ scores and flags. Option 3Ataccama can be connected to a data stream, such as Kafka, and on the fly evaluating data in the stream - every mes

pdanpoonkijData Pioneer L2

asked in Data Quality & Governance ⚙️

Trigger a workflow after monitoring project finished

Hello, communityI’m currently working on a project which requires a workflow on the runtime server to be triggered after the monitoring project on Ataccama ONE is finished. Can anyone recommend a solution to achieve this? Thank you in advance

Maxim KimAtaccamer

posted in Master Data Management & Reference Data Management 🛰️

What to do if Approval Requests page throws error with NodePath()

One of the customers had experienced the error upon entering Approval Requests page:And it happened after metadata changes that looked reasonably safe.So what happened?It turns out that if you have an entity that is a part of workflow (draft, approve, …, publish) and you already have published some of them -- deleting attribute which is an embedded array of other entities would lead to this kind of error.Luckily, there is a way to fix it if you have access to MMM database: delete from "mdArbitraryReference_r" where id in ( select id_i from "mdArbitraryReference_q" where "referencedNodePath_s" not in (select id from "_MmdDictionary" where type = 'PATH')); delete from "mdArbitraryReference_q" where "referencedNodePath_s" not in (select id from "_MmdDictionary" where type = 'PATH');These SQL queries fix DB inconsistency removing invalid records from arbitrary references tables.I would still suggest to consult with support if you encounter such an error first, though. :)

KarthikaSPillaiData Pioneer

asked in Data Quality & Governance ⚙️

do we have row level quality score?

I see column level quality score in monitoring project. Do we have row level quality score?

OjaswiniStar Blazer L1

asked in Data Quality & Governance ⚙️

DQ Monitoring project UAT/Test deployment

Hello,I am new to DQ and Ataccama. Our code is in development environment and soon to be deployed in Test/UAT for UAT testing and later to Prod once we get UAT signoff.My question is what is the way to export DQ monitoring project and its related components (like rules, etc) from Dev environment to UAT/Test environment? Is there any export capability? I have also heard that we may need to recreate it again in UAT environment. Really appreciate any direction in this.Thanks in advance.

pdanpoonkijData Pioneer L2

asked in Data Quality & Governance ⚙️

Configuring export plan for exporting evaluation rules

I want to export DQ Evaluation Rules from Ataccama ONE and import them onto another Ataccama environment. To my knowledge, this will require an export plan. However, I’m unsure how to configure the JSON file for the plan.Could anyone guide me on this, please? Thank you in advance

Marnix WisselaarStar Blazer L2

asked in Data Quality & Governance ⚙️

Approximate string matching - fuzzy matching / lookup - How do I create a fuzzy match?

Hi,We'd like to compare two string-fields with several words. Is there a fuzzy matching or approximate string matching step for a plan?In this case it is not reference of lookup data. They are name fields. One is in the DB the other in a file. Kind regards, Marnix

lumir.studnicka

published in Master Data Management & Reference Data Management 🛰️

Git: how to deal with an unwanted merge of branches

IntroductionSometimes you realize you just performed an unwanted merge & push to origin/remote Git repository. E.g. you made a mistake in selecting "target" and "source" branch and merged the "env" branch to some "feature" instead the other way round. And most probably you want to remedy this as soon as possible and before other team members use such "broken" branch any further.One of the options can be a revert commit but that would undo a previous (or selected) commit, i.e. undo a single commit. But you might have merged a series of commits which you would like to undo at once and set the git branch to a certain state in the past. On top of it, you can even add (cherry-pick) any individual commits that happened after (contributions of your colleagues you want to keep). ProcedureDisclaimer and preparationFollowing procedure uses a “hard” reset; it is recommended not to perform this in the project phase when rapid development is happenning and/or to communicate your intention with

lsmall028Data Enthusiast

asked in Data Quality & Governance ⚙️

What does strange number of lookup header sections mean for Monitoring Projects?

I’m running a monitoring project and it is failing. Here is what I am receiving as a message: Caused by: java.io.IOException: strange number of lookup header sections: 0What does this mean?

AlesAtaccamer

published in Data Quality & Governance ⚙️

Efficient Plan Design - number of fields matters

You can optimize your plans by simple things. In this case I would like to show how the number of columns/fields influences the performance of the processing. Test conditionslocal environment, stable conditions 14M records generated by Random Record Generator 78 fields containing fixed strings or dates No parallelism level set Test Cases One stream – simple reading - BASELINEOverall progress: 117k/s Duration: 3:08min Split into 4 streams, same number of fields – you really need ALL fields in ALL streamsOverall progress: 91k/s Duration: 3:42min Split into 4 streams, reduced number of field to 1 – maybe you just need 1 field in each streamOverall progress: 109k/s Duration: 3:19min Split into 4 streams, 3 of them with reduced number of fields to 1 – you might not need all fields in other streamsOverall progress: 101k/s Duration: 3:33min In this example it’s just about simple transformation flow however in case you use JOINs, RepresentativeCreators, Aggregators etc. and you reduce the

CatherineData Pioneer

asked in Data Quality & Governance ⚙️

How to schedule a profiling job for specific schema vs all schemas within the data source?

Use Case; to configure a scheduled profiling job daily for a specific schema of a data source within Ataccama Web application and not via Desktop?

claireData Voyager

asked in Data Quality & Governance ⚙️

How to connect with an ODBC connection ?

I need to connect to an AS400 database. I have an odbc connection working for that but I can use ODBC with DQA. No OBDC driver is available. it is not like what is displayed on the FAQ support page when they explain how to connect to MS access using ODBC.it seems the driver has disapeared :-(

john.efechaoborAtaccamer

published in Master Data Management & Reference Data Management 🛰️

Connecting Azure AD to Keycloak (SAML)Best practice

Security Assertion Markup Language (SAML) allows users to use one set of credentials to log into many different websites. It is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers(SP). SAML transactions use Extensible Markup Language (XML) for standardized communications between the identity provider and service providers. SAML is the link between the authentication of a user’s identity and the authorization to use a service. Items to consider prior to connecting to SAML- Best PracticeIt is important to understand how your organization’s Active Directory is setup. Some pre-requisites needed can include things such as:Do you have a list of groups you need to import? If the answer is no, you may want to speak to your engagement manager from Ataccama to discuss in detail. In short, with AD Groups we can map Ataccama Roles to a single group. This benefit of doing this is, your internal dev-ops team won’t have to manually add/remove u

Adrian AndersonAtaccamer

published in Data Quality & Governance ⚙️

Adding Check-digit Validated Tax File Number Detection and DQ Evaluation Rules to ONE

Australian Tax Files Numbers, commonly known as TFNs, are 9-digit numbers where the last digit is a validation check-digit.How would we go about setting up Detection and DQ Evaluation Rules for TFNs?The following article applies to ONE version 13.x. The screenshots are from v13.9.For the Detection Rule, we might want to ignore any extraneous characters in the data, for example, the data that is profiled might contain, TFN:123456782, that could be easily transformed into a valid TFN by removing the prefix.For the DQ Evaluation Rule, we might want to consider values that contain the TFN: prefix like in previous example as invalid, to indicate that further processing/cleansing of the data is required.The good news is that we can use almost the same condition for detecting and validating the numeric part of the TFN. For the DQ Evaluation Rule, we’ll use an extra condition to check that only numbers are present.Detection RuleTo configure the Detection Rule in ONE, head to the Rules section

srikanth.gurramData Voyager

asked in Master Data Management & Reference Data Management 🛰️

Looping record by record in Ataccama One Desktop

We have one use case where we are looking for a feature of looping the records and reading it record by record .. Can you please share any template plan for the same..

KundanData Pioneer L2

asked in Data Quality & Governance ⚙️

Appending data to file

I have a plan file in which there are multiple flows which generate multiple result csv files having different number/size of column .I want to store the output of each flow into a single csv file by appending the rows.I want to this happen every time I run the plan file Example - On 1st run of the plan file each result csv file have 10 rows and there are 5 flows . This should lead to a total of 50 rows in the appended file . Now on the 2nd run of the plan file the 10 rows are generated again for each flow , this should lead to a total of 100 rows( 50(1st run) + 50 (2nd run) ) in appended file .

KarthikaSPillaiData Pioneer

asked in Data Quality & Governance ⚙️

How to validate the scenario in description?

"When one column have countries, another column have city" This city should be in that country else it is invalid. Please help.

1
...
23
24
25
26
27
28
29
...
33

Page 26 / 33

Badge winners

ChrisBowmanhas earned the badge First idea 💡
NicolasDrolethas earned the badge First idea 💡
Catherinehas earned the badge Community Champion
Albert de Ruiterhas earned the badge Community Champion
ThomasProhas earned the badge Community Champion

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more

Powered by Gainsight

Community Terms of Use & Conditions Cookie settings

Sign up

Already have an account? Login

For Ataccama Customers and Partners

or

Username *

E-mail address *

Company *

Job Title *

A bit about yourself

How did you hear about the Data People Community? *

Ataccama version

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the Ataccama Community

No account yet? Create an account

For Ataccama Customers and Partners

or

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK