Skip to main content

Product Updates
Events
Training Resources
Groups

Home
Learn & Share
Data Quality & Governance ⚙️

Data Quality & Governance ⚙️

Discuss, learn, ask, suggest, share all things DQG 🔍

539 Topics
1,291 Replies

When you subscribe we will email you when there is a new topic in this category

539 Topics

Recently active

Most replies Most views Newest first

RoelantVosData Voyager

asked in Data Quality & Governance ⚙️

Reproducing the Term level Data Quality overview from GraphQL

Hi Ataccama community,I’m hoping someone with better grip on the Ataccama GraphQL model can shine a light on this. For internal reporting, we are looking to retrieve data quality details via the Ataccama GraphQL API. One of the requests it to produce an overview of the DQ score across the (rule’s) DQ dimensions. This is basically the same as is shown in the web UI when navigating to the ‘Data Quality’ tab for a Term. It shows the score broken down by dimensions, based on how the rules are defined.What I’m trying to do is retrieve the passed/failed rows at this level, so for each term and each dimension. This would allow me to summarise this into a score per dimension.However, I am unable to get this out of the API.I’ve approached this from both the ‘terms’ and ‘rules’ pathway. In both approaches I always end up with the ‘Overall Quality’ score for each term - not the per dimension details.The ‘rules’ query looks promising, but ultimately only returns the passed/failed records for the

greglvaughanData Enthusiast

asked in Data Quality & Governance ⚙️

monitoring project processing custom label

Issue: We receive files to be tested against our DQ rules (Monitoring Projects) every 2 weeks. Each set of files is referenced by its date created (aka 2024-10-01,2024-10-14...). When running Monitoring Projects against these items we only get the Execution Date which can be different from the file date. When extracting data for reporting from ONE we only have the Extraction Date and don’t necessarily know which file date the plan results are for (sometimes have to rerun old files for new rules etc..). Question: Is there a way to add a custom attribute / parameter for when we run the Monitoring Project (at the processing id level) so we can report our DQ results by File date? The 2 options we could think of: In extraction component connect to the catalog items and get the file date which is in the table itself (tried this but we need to extract history so would need file date, catalog item, and Monitoring Project processing id to join with DQ Aggregation Results tasks in the com

manikanta vData Enthusiast

asked in Data Quality & Governance ⚙️

How to Implement Logic in Ataccama to Process One Record and Flag Another as Error for Duplicate Correlation ID and Version Number

Two records as same correation id and same version number one should process another one should error out how to implement this logic in ataccama

Szymon OlejniczakUniverse Traveller

asked in Data Quality & Governance ⚙️

Debugging - setting breakpoints

Debugging - is this possible to i.e. setup some breakpoint and let the application run all the previous steps before selected one?I like using debug function on steps (CTRL+Shift+D). To debug below Step i need to first debug all the previous steps and then i am able to debug the current one.However, if I happen to update logic on some step from the top i then would need to run & debug all the previous steps before i could debug i.e. Multiplicative Regex Matching.Is this possible to automate the manual work here? I would like to create a breakpoint on Multiplicative Regex Matching and each time i could just click a button and it would self-debug from the start to my breakpoint. I looked in your documentation but didn’t found any solution for this. I found some debugging view in ONE Desktop but it doesn’t allow me to add breakpoints.Do i need to configure my debugger somehow to achieve the above? Szymek

asked in Data Quality & Governance ⚙️

Runtime error in JDBC reader: Error creating record

Encountering with Runtime error in JDBC reader, while running the basic profiling for a table from SQL Server. Full Error details -Runtime error in JDBC reader: Error creating record: column type: 'STRING', read value: '<null value>', read object type: '<null value type>', cause: Connection reset Looking for ideas troubleshooting this error.

MelData Enthusiast

asked in Data Quality & Governance ⚙️

Join a ONE Data table with a Catalog Item in OneWeb

I have a One Data table created by importing data from a .csv file. I need to use some of the data in this table by joining it with an existing Catalog Item by creating a SQL Catalog Item so that I can implement some rules on it. Where you select the source when starting to create the SQL Catalog Item I do not get an option to select oneData. Is it even possible to do this in OneWeb?

ritguptaData Voyager

asked in Data Quality & Governance ⚙️

Access to Transformation Plan

Hi,I am not able to access the transformation plan feature in v15.3 from both the Transformation Plan tab and from the monitoring projects tab. Can someone explain why is there a restriction on creating transformation plans and how can I get access to it?Thanks

Albert de RuiterStar Blazer L2

published in Data Quality & Governance ⚙️

Business glossary – computed content (part 4)

Hi Community,The first posts about the business glossary dealt about applying more term types (attribute and entity terms), and how they get context by adding relationship types. The previous post was about implementing multilingualism. The post ended with a remark about computed content, being the topic of this post.To refresh your memory, at the end of the article I showed this screen print, depicting how the translated terms show in the term’s detail page. So like this.Notice that the translation of the term name is preceded by the language code, so it is a concatenation in other words. I will describe how you can define this. Actually to accomplish this I made use of a document that Ataccama provided and that I will add to this article. The document also describes another use-case, so make sure to have a look! Anyhow, this is how the concatenation of the language code and translated term name is done.The calculation is defined in a new metadata model entity. As naming convention yo

Vanja GilbertUniverse Traveller

asked in Data Quality & Governance ⚙️

Issue Viewing Rule Logic in Sectioned Report View

Hi,I have a question about the rule logic display when a section is applied. In the report view, I can see the rule logic/implementation when I select a rule. However, once I select a section and do the same, I can only see the valid and invalid results, not the rule logic. Is there a reason behind it? Thank you!

C_RUniverse Traveller

asked in Data Quality & Governance ⚙️

Is there a way to Parse Copybook file in order to transform Data File into JSON/CSV

Hi All, Is there a way to parse a copybook file in order to transform the data file into JSON/CSV? The use case is to be able to provide profiling metrics on the web via VCIs? Thank you😁

lawroData Voyager

asked in Data Quality & Governance ⚙️

When creating groups, are the names always public to everyone who uses the instance? Or is there a way to create a group of groups that only the members of that groups can see?

Delbert71Universe Traveller

asked in Data Quality & Governance ⚙️

¬ delimiter issue

Hi all,I have a file with a ¬ delimiter and text qualifiers and in the file it looks fine but when I look at the local file system in Ataccama to bring the file into the catalog the ¬ shows as Z with an accent. Are there different regional settings in the config that need setting or changing or is this something else? Many thanks for any help Des

rajeswarimylaSpace Explorer

asked in Data Quality & Governance ⚙️

Can you please help in getting incremental number in split operation

hi Team, i need to split a word based on a delimiter and try to add inceremental number to it, can you please help in acheiving this scenario.Input : select * from order;select * from customeroutput should be like select * from order - query 1, select * from customer - query 2i itried using split operator , but could not able to append the incremental number

Rohith1217Data Voyager

asked in Data Quality & Governance ⚙️

New Use case implementation in Ataccama

Hi,We have a use case to accept a request from collibra with details like JDBC URL, username, password and query to Ataccama and I need to return the results and size of the results in reponse from Ataccama.Can you please share usecase/documentation for implementing this usecase in Ataccama.Regards,Rohith

JamesBlackwellData Voyager

asked in Data Quality & Governance ⚙️

Alation integration

Has anyone experience of integrating Ataccama with Alation Data Catalog?I’ve searched the support and knowledge base article without any success.

Delbert71Universe Traveller

asked in Data Quality & Governance ⚙️

Fixed width file import to catalog through webui or desktop

Hi all, what about if I had a fixed width file in the catalog with the first row as the header (in error). The column data comes in as one string which renders it unusable for profiling and DQ. How can I get these types of files into the catalog so I can use them in my monitoring project? I have used VCI’s before to add headers to csv files in the catalog and they work well. Kind regards Des

rajeswarimylaSpace Explorer

asked in Data Quality & Governance ⚙️

how to split a word and get the first index value in component

hi,i had a word like this “12345 - Data min”, i need to split this word based on - operator and get the first word.I had another requirement like split and save the words in another varaibleex : select * from emp;select * from customerquery variable should have splitted value like select * from empcan you help in implementing above scenario

may_kwokStar Blazer L2

asked in Data Quality & Governance ⚙️

Planned maintenance advance notification banner?

Hi folks,Is there some way that I can display a notification banner on Ataccama ONE and RDM, to notify users in advance of any platform maintenance?I’d like to notify my users in advance if we have any planned works, including and not limited to platform version upgrades, or RDM model updates.

Albert de RuiterStar Blazer L2

published in Data Quality & Governance ⚙️

Business glossary - multilingualism (part 3)

Hi Community,The previous posts about the business glossary dealt about applying more term types (attribute and entity terms), and how they get context by adding relationship types.This time I want to discuss options to add multilingualism to the glossary. I don’t mean the One Web interface itself, but the content. When your company operates in an international environment, having term names and definitions in just one language may not suffice (although many people understand English, the level of understanding may vary, so translations will contribute to a better understanding of the glossary content).So suppose the glossary contains content in English and, like in my situation, also Dutch translations are required for the term’s name and definition.The most simple solution would be to add properties like nameDutch and businessDefinitionDutch to entity term. And when German translations are required as well, add properties nameGerman and businessDefinitionGerman and so on.The pro of t

clyormzUniverse Traveller

asked in Data Quality & Governance ⚙️

helm on ataccama dpe deployment and configuration

On the last 8 months we were working on deploy cloud resource to integrate databricks solution with ataccama dpe and develop jobs to execute on databricks and see the results on ataccama one. The deployment is executed on cloud provider side (aws - microsoft) and the configuration is manually so it takes time to set up with corresponding values, one of the task we would like to improve is the configuration of one databricks with unity catalog to be part of ataccama dpe. On past forums topics I saw helm is part of the tech stacks of ataccama and I am interested to know more about on. I would like to get more information kind of library to repository to understand helm on ataccama dpe. thank you

may_kwokStar Blazer L2

asked in Data Quality & Governance ⚙️

Using runtimeconfig to encrypt OAuth 2.0 token request

I’m building an API call request, where I first need to obtain an access token, before I make my actual API call.I have tested the 2-step process using 2 separate JSON calls successfully. The message body of the OAuth request looks like this. Sensitive info redacted: scope is a required parameter. The above call successfully retrieves my token. I tried to setup this as a Generic server:But there is no available space to specify the scope parameter. When I run my plan: I get:2024-09-04 09:33:16 INFO: Validating model...2024-09-04 09:33:16 INFO: Creating runtime...2024-09-04 09:33:17 INFO: Running runtime...2024-09-04 09:33:17 [JsonCall] DEBUG: IOException when making request GET https://preprod-api.londonmarketgroup.co.uk/reference-data/acord/v2/version HTTP/1.1java.io.IOException: Cannot get credentials for request. at com.ataccama.dqc.io.http.CommonHttpClient.makeRequest(CommonHttpClient.java:351) at com.ataccama.dqc.tasks.io.json.call.JsonCall$Runtime.run(JsonCall.java:425) at com.at

asked in Data Quality & Governance ⚙️

Full Profiling results

Hi,13.9.4In the Monitoring Project there's a way to only display the data quality of the last run via Advanced settings Is there also such a way to do accomplish this with Full Profiling on (SQL) Catalog Items?

MohtaramData Voyager

asked in Data Quality & Governance ⚙️

Monitoring Projects - Notifications with Dashboard

I want the result of my monitoring project to appear in the dashboard in the Report section when I receive it by mail. Is it possible to amend the mail notification regarding this?The notification in mail should look like this:

may_kwokStar Blazer L2

asked in Data Quality & Governance ⚙️

Connecting to network drive in runtime server

Hi community,Has anyone managed to connect Ataccama (we’re on PaaS), or even ONE Desktop, to a corporate network folder, download files, and work with it in .ewf workflows and components / plans?Yes I know Ataccama connects to S3 buckets / ADLS Gen2 containers, but the environment that I work in people are really used to shared network drives and a lot of applications are integrated with them. It would make my life a lot easier if I can make Ataccama connect into it.When I look at either the SCP or SFTP upload/download steps, it requires a server to be set up with Basic authentication - of course when I connect to the network drive personally I don’t need to enter any of that.Has anyone done something similar?

KaterinaAtaccamer

published in Data Quality & Governance ⚙️

Enhancing Data Quality Evaluation with Component Rules in ONE Web

Speaking about data quality rules, we usually refer to DQ rule configuration in ONE web application. However, when it comes to referencing external sources or performing preliminary work or combining different sources, we may find that the capabilities within ONE Web are insufficient for our needs. For example, consider a scenario where we need to validate whether our data matches an external data source, such as ABN Lookup on https://abr.business.gov.au/. This is where component rules come in handy.The Task at Hand: Validate ABN Information.To implement this data quality (DQ) check, we require a component rule.Here's how you can configure the ABN validation component and use it for DQ evaluation: Configure a placeholder for the component Configuring Logic for the Component using ONE Desktop application Adding Changes to Runtime Configuration in ONE Web Application Configuring the DQ component rule Applying the RuleConfigure a placeholder for the componentNavigate to DQ s

1
2
3
4
...
22

Page 1 / 22

Badge winners

tomas.reichelhas earned the badge First idea 💡
PierreLequyhas earned the badge First idea 💡
NickMikedishas earned the badge First idea 💡
Vanja Gilberthas earned the badge First idea 💡
GastonQhas earned the badge First idea 💡

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more

Powered by Gainsight

Community Terms of Use & Conditions Cookie settings

Sign up

Already have an account? Login

For Ataccama Customers and Partners

or

Username *

E-mail address *

Company *

Job Title *

A bit about yourself

How did you hear about the Data People Community? *

Ataccama version

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the Ataccama Community

No account yet? Create an account

For Ataccama Customers and Partners

or

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK