Skip to main content

Data Quality & Catalog ⚙️

Discuss, learn, ask, suggest, share all things DQ&C 🔍

  • 587 Topics
  • 1,455 Replies
587 Topics
C
Universe Traveller
C_RUniverse Traveller
asked in Data Quality & Catalog ⚙️
Is there a way to Parse Copybook file in order to transform Data File into JSON/CSV

Hi All, Is there a way to parse a copybook file in order to transform the data file into JSON/CSV? The use case is to be able to provide profiling metrics on the web via VCIs? Thank you😁 

292
Cansu
Community Manager
L
Data Voyager
lawroData Voyager
asked in Data Quality & Catalog ⚙️
Private Groups

When creating groups, are the names always public to everyone who uses the instance?  Or is there a way to create a group of groups that only the members of that groups can see?

161
Cansu
Community Manager
D
Universe Traveller
Delbert71Universe Traveller
asked in Data Quality & Catalog ⚙️
¬ delimiter issue

Hi all,I have a file with a ¬ delimiter  and text qualifiers and in the file it looks fine but when I look at the local file system in Ataccama to bring the file into the catalog the ¬ shows as Z with an accent. Are there different regional settings in the config that need setting or changing or is this something else? Many thanks for any help Des

283
Cansu
Community Manager
R
Space Explorer
rajeswarimylaSpace Explorer
asked in Data Quality & Catalog ⚙️
Can you please help in getting incremental number in split operation

hi Team, i need to split a word based on a delimiter and try to add inceremental number to it, can you please help in acheiving this scenario.Input : select * from order;select * from customeroutput should be like select * from order - query 1, select * from customer - query 2i itried using split operator , but could not able to append the incremental number

314
ivysakh
Ataccamer
R
Data Voyager
Rohith1217Data Voyager
asked in Data Quality & Catalog ⚙️
New Use case implementation in Ataccama

Hi,We have a use case to accept a request from collibra with details like JDBC URL, username, password and query to Ataccama and I need to return the results and size of the results in reponse from Ataccama.Can you please share usecase/documentation for implementing this usecase in Ataccama.Regards,Rohith

572
Cansu
Community Manager
J
Data Voyager
JamesBlackwellData Voyager
asked in Data Quality & Catalog ⚙️
Alation integration

Has anyone experience of integrating Ataccama with Alation Data Catalog?I’ve searched the support and knowledge base article without any success.

3604
S
Data Pioneer
D
Universe Traveller
Delbert71Universe Traveller
asked in Data Quality & Catalog ⚙️
Fixed width file import to catalog through webui or desktop

Hi all, what about if I had a fixed width file in the catalog with the first row as the header (in error). The column data comes in as one string which renders it unusable for profiling and DQ. How can I get these types of files into the catalog so I can use them in my monitoring project? I have used VCI’s before to add headers to csv files in the catalog and they work well. Kind regards Des

334
D
Universe Traveller
R
Space Explorer
rajeswarimylaSpace Explorer
asked in Data Quality & Catalog ⚙️
how to split a word and get the first index value in component

hi,i had a word like this “12345 - Data min”, i need to split this word based on - operator and get the first word.I had another requirement like split and save the words in another varaibleex : select * from emp;select * from customerquery variable should have splitted value like select * from empcan you help in implementing above scenario

442
R
Space Explorer
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
Planned maintenance advance notification banner?

Hi folks,Is there some way that I can display a notification banner on Ataccama ONE and RDM, to notify users in advance of any platform maintenance?I’d like to notify my users in advance if we have any planned works, including and not limited to platform version upgrades, or RDM model updates.

233
may_kwok
Star Blazer L3
Albert de Ruiter
Star Blazer L2
Albert de RuiterStar Blazer L2
published in Data Quality & Catalog ⚙️
Business glossary - multilingualism (part 3)

Hi Community,The previous posts about the business glossary dealt about applying more term types (attribute and entity terms), and how they get context by adding relationship types.This time I want to discuss options to add multilingualism to the glossary. I don’t mean the One Web interface itself, but the content. When your company operates in an international environment, having term names and definitions in just one language may not suffice (although many people understand English, the level of understanding may vary, so translations will contribute to a better understanding of the glossary content).So suppose the glossary contains content in English and, like in my situation, also Dutch translations are required for the term’s name and definition.The most simple solution would be to add properties like nameDutch and businessDefinitionDutch to entity term. And when German translations are required as well, add properties nameGerman and businessDefinitionGerman and so on.The pro of t

643
Albert de Ruiter
Star Blazer L2
C
Universe Traveller
clyormzUniverse Traveller
asked in Data Quality & Catalog ⚙️
helm on ataccama dpe deployment and configuration

On the last 8 months we were working on deploy cloud resource to integrate databricks solution with ataccama dpe and develop jobs to execute on databricks and see the results on ataccama one. The deployment is executed on cloud provider side (aws - microsoft) and the configuration is manually so it takes time to set up with corresponding values, one of the task we would like to improve is the configuration of one databricks with unity catalog to be part of ataccama dpe. On past forums topics I saw helm is part of the tech stacks of ataccama and I am interested to know more about on. I would like to get more information kind of library to repository to understand helm on ataccama dpe. thank you

111
Cansu
Community Manager
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
Using runtimeconfig to encrypt OAuth 2.0 token request

I’m building an API call request, where I first need to obtain an access token, before I make my actual API call.I have tested the 2-step process using 2 separate JSON calls successfully. The message body of the OAuth request looks like this. Sensitive info redacted: scope is a required parameter. The above call successfully retrieves my token. I tried to setup this as a Generic server:But there is no available space to specify the scope parameter. When I run my plan: I get:2024-09-04 09:33:16 INFO: Validating model...2024-09-04 09:33:16 INFO: Creating runtime...2024-09-04 09:33:17 INFO: Running runtime...2024-09-04 09:33:17 [JsonCall] DEBUG: IOException when making request GET https://preprod-api.londonmarketgroup.co.uk/reference-data/acord/v2/version HTTP/1.1java.io.IOException: Cannot get credentials for request. at com.ataccama.dqc.io.http.CommonHttpClient.makeRequest(CommonHttpClient.java:351) at com.ataccama.dqc.tasks.io.json.call.JsonCall$Runtime.run(JsonCall.java:425) at com.at

322
Cansu
Community Manager
JTH
Data Pioneer
JTH
asked in Data Quality & Catalog ⚙️
Full Profiling results

Hi,13.9.4In the Monitoring Project there's a way to only display the data quality of the last run via Advanced settings ​​​​​Is there also such a way to do accomplish this with Full Profiling on (SQL) Catalog Items?

705
Cansu
Community Manager
Mohtaram
Data Voyager
MohtaramData Voyager
asked in Data Quality & Catalog ⚙️
Monitoring Projects - Notifications with Dashboard

I want the result of my monitoring project to appear in the dashboard in the Report section when I receive it by mail. Is it possible to amend the mail notification regarding this?The notification in mail should look like this: 

610
Mohtaram
Data Voyager
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
Connecting to network drive in runtime server

Hi community,Has anyone managed to connect Ataccama (we’re on PaaS), or even ONE Desktop, to a corporate network folder, download files, and work with it in .ewf workflows and components / plans?Yes I know Ataccama connects to S3 buckets / ADLS Gen2 containers, but the environment that I work in people are really used to shared network drives and a lot of applications are integrated with them. It would make my life a lot easier if I can make Ataccama connect into it.When I look at either the SCP or SFTP upload/download steps, it requires a server to be set up with Basic authentication - of course when I connect to the network drive personally I don’t need to enter any of that.Has anyone done something similar?

202
may_kwok
Star Blazer L3
R
Data Voyager
RadziahData Voyager
asked in Data Quality & Catalog ⚙️
Configure Path in Attach File of Send Email Workflow

HiI have a workflow with Send Email step configured and this workflow will be pushed to orchestration server to run it. I have an attachment need to be included in the email notification generated from the workflow and realized that if I put only the file name for example RESULTS_ABC.xlsx, by default it will find it the same folder of the workflow that is being created. Let’s say if I put the RESULTS_ABC.xlsx in different folder like Data/Out, how do I configure it correctly at the Attach Files configuration as shown in the image below? I saw a discussion to use filePath variable but I am not too familiar with that. Could you please explain how can we achieve running the workflow and read the file that is stored from different folder so that it can attach in the email. 

582
R
Data Voyager
R
Data Voyager
ritguptaData Voyager
asked in Data Quality & Catalog ⚙️
Profiling on a subset of the data

Hi, I was wondering if there is a way to profile a subset of the data based on a certain criteria in the web application. For example, I want to get the profiling results only for records with the ‘Active’ status in the ‘Status’ field.I am able to do this in the desktop application by writing a custom sql query to filter the results. If I can do this directly in the web app that would be really helpful.Thanks in advance!

425
Cansu
Community Manager
M
Data Pioneer
Maria GolikovaData Pioneer
asked in Data Quality & Catalog ⚙️
Search on code

Hello Dears,Is there a functionality or a tool available to search on the code of VCI, catalog items, DQ rules…. in all the object types that we use in Ataccama.We know that there is possibility to search on the names, but not on the underlying levels.  Thank you for ideas.Kind regards,Maria

161
ivan.kozlov
Ataccamer
R
Data Voyager
rene.requenesData Voyager
asked in Data Quality & Catalog ⚙️
Parse Fixed width File for Profiling

Hello,I was able to parse a fixed width file using the Ataccama One App by editing the metadata.  But when i try to parse the same file using the Ataccama Web App it doesn’t parse correctly.  Is there a way to give the metadata file to the Web App for it to parse the file correctly ? Thanks, Rene Requenes

593
D
Universe Traveller
Albert de Ruiter
Star Blazer L2
Albert de RuiterStar Blazer L2
published in Data Quality & Catalog ⚙️
Business glossary - adding relationship types (part 2)

Business glossary - relationship types (part 2)Hi Community,In my previous post I presented the idea to implement new metadata entities for attribute and entity terms in the glossary. The reason for this is that it makes most sense to have more types of terms that best describe either a catalog item attribute (the attribute term) or a catalog item (the entity term).I finished the post with making a comparison to logical data modelling.In a logical data model the entity is described by its properties, so the attributes. An entity not only has descriptive attributes, but obviously also one or more attributes that identify a record of that entity, the primary key. Besides the primary key also an alternate key may exist (for instance an email address can be used as alternate key next to the client id). Or an attribute can act as foreign key to another entity. Such an attribute will then also exist in its parent entity.With the example of an attribute existing both as foreign key in one ent

731
Cansu
Community Manager
R
Data Voyager
Rohith1217Data Voyager
asked in Data Quality & Catalog ⚙️
Connecting to cosmosDB from Ataccama ONE DESKTOP

Hi Team,I am checking on connecting to Cosmos database from Ataccama One Desktop, can you please share any documentation or usecase available to do that. Thanks,Rohith

303
may_kwok
Star Blazer L3
H
Data Voyager
hbairiData Voyager
asked in Data Quality & Catalog ⚙️
Need an automation to map Active Directory groups to access to groups and monitoring projects on Ataccama UI

Need an automation to map Active Directory groups to access to groups and monitoring projects on Ataccama UI. 

675
may_kwok
Star Blazer L3
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
Runtime server / ONE Desktop connection to Azure Keyvault

I have a use case where I want to store some client id and secret in my own company’s Azure keyvault.Then I have a component which needs to use the client id and secret to authenticate against a 3rd party, so I can issue a GET call to get some data. So I am thinking:Runtime server authenticate against keyvault Runtime server issue a “get secret” call to keyvault Keyvault sends back the secret Runtime server takes the secret and authenticate against 3rd party Runtime server issue a “get data” call to 3rd party 3rd party sends me back the actual data What authentication methods from runtime server to keyvault are supported? I had a look at Azure keyvault’s documentation but I don’t think I understand it fully. https://learn.microsoft.com/en-us/azure/key-vault/general/authentication Has anybody else done it? What’s your experience?

786
may_kwok
Star Blazer L3
J
Data Voyager
jcarrData Voyager
asked in Data Quality & Catalog ⚙️
How does Ataccama interpret duplicate attributes if they have different meanings to different terms?

Good Morning, I have a quick question in regards to loading data and how it appears into Ataccama from a database. I did not see this topic addressed but I wanted to reach out and ask this and see if this question made sense.In the Knowledge Catalog, if terms are “table names” and attributes are “columns” as described in the documentation. How does Ataccama interpret duplicate attributes spread across separate tables of data? For example, how would Ataccama interpret two different tables that have duplicate titled columns within those tables? Especially if those column headers have different meanings across the tables. If any further explanation is needed feel free to send me a message.

292
A
Ataccamer
D
Data Enthusiast
DhavalMData Enthusiast
asked in Data Quality & Catalog ⚙️
How to Transpose columns in a plan?

I have been given a list of emails that I want to transpose—converting the data from rows to separate columns—and then concatenate these columns into a single row. How can I do that in the IDE? Original Data After Transpose Final Output Note: All the emails will be unique, so I don’t want them to be grouped. Could you please suggest how I can transpose these rows into columns and later merge them into one single row?Thank you.Regards,Dhaval

452
Cansu
Community Manager

Badge winners

Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings