Skip to main content

Learn & Share

Discuss data, share best practices, learn, and grow with Ataccama's Community.

1079 Topics
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop Plans: Column Naming & Debugging πŸͺ²
ONE Desktop Plans: Column Naming & Debugging πŸͺ²

Hi everyone!Today, we have a quick best practice on covering the basics of column naming and debugging in ONE Desktop plans.When working with ONE Desktop, maintaining a consistent and meaningful naming convention for columns in your output files is crucial. This practice ensures clarity, organization, and seamless data processing. In this guide, we'll provide recommendations for naming conventions and debugging your expressions and steps for efficient plan execution.Β Column Naming ConventionsOutput files in ONE Desktop can contain numerous columns, especially after processing input data in DQC. To keep your data well-organized, we recommend grouping columns by content. Below are the key prefixes and suffixes used in our naming convention:Β PrefixesAttribute Prefix Explanation src_xxx Source input values (read-only attribute) meta_xxx Source input metadata dec_xxx Decoded source input values dic_xxx Translated master value storage (src_xxx is the source) pur_xxx Pre-clea

2070
A
Albert de Ruiter
Rocket Pioneer L1
Albert de RuiterRocket Pioneer L1
asked inΒ Data Quality & Catalog βš™οΈ
Hiding empty metadata fields in business glossary

Hi,I am wondering if the following is possible. On the glossary's termΒ screen not all properties/metadata will always be filled. For instance in most occasions there will not be an abbreviation for a term.Is there a way, on the view screen only, to hide this abbreviation field when it contains no value?Thanks for your time!Kind regards,Albert de Ruiter

674
Albert de Ruiter
Rocket Pioneer L1
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop: Running Plans πŸƒ
ONE Desktop: Running Plans πŸƒ

Hi Community!We are back with part 2 of One Desktop Plans! If you have missed our first post on how to build plans you can find it by clicking on the link below πŸ‘‡Β When it comes to executing plans within ONE Desktop, understanding the various methods and choosing the right approach is essential for a seamless experience. In this guide, we'll explore the best practices for running plans effectively.Β Understanding Execution OptionsONE users have multiple ways to run a plan: Local Launch: This is done using Ataccama ONE runtime (formerly known as DQC) from the ONE Desktop interface. ONE Platform Launch: You can employ DPE or Spark DPE for remote execution via the ONE Platform. Command Line Execution: Utilize the runcif.sh|.bat script via command line when employing Ataccama ONE runtime. DPM CLI Tool: Employ the DPM CLI tool for remote execution using DPE or Spark DPE. This article primarily focuses on executing plans. For information on running plans within workflows, please refer t

2200
A
G
Data Enthusiast
GersonData Enthusiast
asked inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Property with name 'KafkaResources' doesn't exist in com.ataccama.dqc.streaming.KafkaProviderComponent!

Hello Ataccama Community,I'm currently using Ataccama One Desktop version 13.9.1, and I'm encountering an error while trying to execute a plan that involves a Kafka Reader component. I have configured a Kafka server(kafka integration) with the following properties:ssl.truststore.location=LOCATION_TO_TRUSTSTOREssl.truststore.password=TRUSTSTORE_PASSWORDssl.protocol=TLSsecurity.protocol=SASL_SSLsasl.mechanism=GSSAPIsasl.kerberos.service.name=kafkasasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \Β Β Β Β Β Β Β Β useKeyTab=trueΒ \Β Β Β Β Β Β Β Β storeKey=trueΒ \Β Β Β Β Β Β Β Β keyTab="LOCATION_OF_THE_KEYTAB"Β \Β Β Β Β Β Β Β Β principal="USERNAME_HERE@EUROPE.XYZXYZ.CORP";Β The error message I'm seeing is: "Property with name 'KafkaResources' doesn't exist in com.ataccama.dqc.streaming.KafkaProviderComponent!"I've reviewed my configuration, but I'm still unable to resolve this issue. Can someone please provide guidance on how to troubleshoot and resolve this error? Any insights or suggestions would be gre

641
A
vcosta
Data Voyager
vcostaData Voyager
asked inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Ways to get related entities in transformation proccess

Hi,Β I have below use case:A party can have multiple phones related to it, when any of them change, we need to update a flag on every related phones (same party_id)Β I am having problems to get those related phones during the merge process, what would be the better way to do it?Β Β I have tried it using Mdc Read Step, but I'm not able to start the server due below errorjava.lang.IllegalStateException: Cannot find INmeEngine: Failed to create step 'Mdc Read' (com.ataccama.nme.dqc.steps.MdcRead) (Cannot find INmeEngine) at com.ataccama.nme.internal.dqc.PlanExecutor.createRunner(PlanExecutor.java:137) at com.ataccama.nme.internal.dqc.PlanExecutor.<init>(PlanExecutor.java:83) at com.ataccama.nme.internal.dqc.PlanExecutor.<init>(PlanExecutor.java:69) at com.ataccama.nme.internal.dqc.pool.PlanExecutorCreator.createResource(PlanExecutorCreator.java:34) at com.ataccama.nme.internal.dqc.pool.PlanExecutorCreator.createResource(PlanExecutorCreator.java:17) at com.ataccama.dqc.commons.reso

632
vcosta
Data Voyager
I
Data Voyager
Ilia.StepanichevData Voyager
asked inΒ Data Quality & Catalog βš™οΈ
Simple maker-checker process (approval workflow)

Hi team!I'm trying to implement a simple maker-checker process for Business Glossary Management:Β there are different Data Domains, each with its own responsible: Data Steward and Data Owner.Β So, for each Data Domain, the corresponding Data Steward creates the term --> the corresponding Data Owner approves it. Therefore, it's essential to consider the separation of permissions by data domains.Please give me some ideas on how I can implement it in the current product version.Β As far as I could investigate, grouping terms by different folders/categories in the Glossary doesn't help since I can't assign any roles to a particular folder/category: I can't assign Steward1 and Owner1 to the BusinessTerms group and Steward2 and Owner2 to the SecurityTerms, for example.How should it be implemented, then?Β Many thanks for considering my request!

2012
M
Star Blazer L2
M
Star Blazer L2
Marnix WisselaarStar Blazer L2
asked inΒ Data Quality & Catalog βš™οΈ
Term suggestions overwhelming - How to tweak and make it stop.

Hi,We've loaded 2000+ terms and assigned those with desktop as a migration from a data dictionary. When we now have new catalog items, Ataccama suggests 20+ terms for a CI Attribute. This is not usable.How we tweak it to:Limit to only top 5 Stop when a term is already assigned.And here's an idea: If the column header is the same or can be matched using a simple fuzzy match, that is a better indicator than most AI.Another: Please skip some of the datatypes, a boolean … probably not a good idea.Any idea's?We are also wondering if we can turn it off.Thank you in advance!Marnix

1288
M
Star Blazer L2
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop: Building Plans πŸ—οΈ
ONE Desktop: Building Plans πŸ—οΈBest Practice

Hi Community!Β This week is all about ONE Desktop Plans πŸ—οΈWe will start with how to build plans on ONE Desktop and work our way to DQ Scoring.Next week, we will continue to cover different features of plans and ONE Desktop so don’t forget to subscribe to our forums to receive a daily notification when a new article is published ✨Let’s kick it off!Β Understanding Plan EditorThe Plan Editor is your workspace for crafting plans. It comprises two core components:Canvas (1): This is where you define the logic of your plan by connecting different steps together. Palette (2): Here, you'll find a wide array of steps and actions that can be added to your plan. Creating a New Plan FileStarting a new plan file is a breeze:Right-click on a project or folder in the explorer panel. Β  Select New > Plan. Alternatively, use the toolbar and select Plan. Β  Specify a Name for your plan and designate its storage location (Container). Β  Adding Steps to the CanvasBringing functionality to your plan invol

1030
A
A
Data Voyager
anagha.kData Voyager
asked inΒ Data Quality & Catalog βš™οΈ
Need only Valid rule names in Monitoring Project Report Export

HiΒ I want only the valid rule names in valid_rules column for Monitoring project export. How to implement that?Β 

621
P
Space Explorer
M
Star Blazer L2
Marnix WisselaarStar Blazer L2
posted inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Iterating rows of a table (Record-by-record - SQL Iterator) in a workflow

At Stater we needed to iterate a table in a database and perform some action on it. This needed to be orchestrated from a workflow. In our case trigger a set of MP's and write the output to a DQ Data warehouse.So I created a β€˜pattern’ which we use for our cookbook.Β With thanks to Brian and Matej!Β Β Now if somebody can tell me how to add a PDF and ZIP with the sample, I'd be happy ;)

21513
M
Star Blazer L2
C
Data Pioneer
C_RData Pioneer
asked inΒ Data Quality & Catalog βš™οΈ
I'm trying to retrieve only 10 or so rows from source - SQL Catalog Item

Hi, I have created a SQLΒ Catalog itemΒ in UI however I'm trying to retrieve only 10 or so rows from the database.Β I am using the SELECT TOP/LIMIT clauseΒ to choose how many rows I want to retrieve both queries are having errors, I'mΒ not sure what the problem is, does anyone have any ideas?Β Thank you.Β SELECT TOP 10 FROM xΒ Β SELECT * FROM x LIMIT 10Β Β Β Β 

601
Albert de Ruiter
Rocket Pioneer L1
F
Data Pioneer
FCB developerData Pioneer
asked inΒ Data Quality & Catalog βš™οΈ
Overwrite the data using Jdbc Writer

Hello,everyone.Β When you load data into the database using Jdbc Writer and execute the plan every time, it keeps adding the same data from above. How can I overwrite the data? For instance, I have a table like this.I saved it in the database, made some changes to the table, and when I re-executed the plan, it added the changes on top.Β Duplicates appeared, to put it bluntly.I will be very grateful for your help Thank you!

894
F
Data Pioneer
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop: Projects Part 2️⃣
ONE Desktop: Projects Part 2️⃣Best Practice

Hi community!Β We’re happy to share the second part of our ONE Desktop Projects guide. In this section, we'll delve into essential techniques for project creation, workspace management, and resource configuration. If you have missed the first part you can find it here:Β Creating a New ProjectBefore you embark on data transformations and analyses, a structured project is your starting point. A project is a collection of plans, components, and data files, organized logically into folders. To initiate a new project:Navigate to the File Explorer. Click onΒ New > Project. Β  In the ensuing dialog, specify the project's Name and type. Regular projects encompass various categories: Empty project: A blank slate where you define your structure. Project from template: Predefined layout, useful for component projects. Tutorials project: Generates predefined tutorial projects. Β  Click Finish, and your project will appear in the File Explorer. See the figure below for a template project based

1820
A
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
Monitoring Projects - Lifecycles πŸ”Best Practice

Hi community!We are almost halfway into our Monitoring Project Best Practice series, today we are going to talk about Lifecycles 🎒Let’s dive in!Β Introduction to Monitoring Project LifecyclesThe introduction of Monitoring Project Lifecycles allows you to simulate a multi-environment process (Configure > Test > Send for approval > Make public, and repeat) within a single physical environment. While you can create two physical environments, the non-production environment should primarily be used for specific purposes, such as testing new sources, integration plans, model changes, and service re-configurations.Β It's important to note that non-production environments are not intended for developing data quality monitoring projects and migrating between environments. For the best results, we recommend having one production environment for all metadata and a non-production environment for model changes, API work, and testing. This setup differs from other modules like RDM and MDM, w

1162
A
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
Working with Time Series Data πŸ“ˆBest Practice

HelloΒ Community!Β As part of our ongoing Data Quality Best Practice series, today we'll dive into the world of time series data analysis.Time series data analysis is a powerful tool that enables you to unlock valuable insights from your transaction data by identifying trends and detecting anomalies. To make the most of this feature, here are some best practices to follow:Β Understanding Transaction Data and Time Series AnalysisTransaction data encompasses internal and external events or transactions within a business, often related to financial and logistical records. Common examples include sales orders, invoices, purchase orders, shipping documents, payments, and insurance claims.The benefits of time series analysis include:Visualizing time series data. AI-driven anomaly detection. Β  Keep in mind that time series analysis supports columns with timestamp information, such as DATETIME, DATE, or DAY data types.Β Configure and Run Time Series AnalysisTo get started with time series analysis

1362
A
B
Data Voyager
BhumiData Voyager
asked inΒ Data Quality & Catalog βš™οΈ
Can β€˜SQL Select’ stage only be used to connect with SQL Server DB

Can β€˜SQL Select’ stage only be used to connect with SQL Server DB ?

823
anna.spakova
Ataccamer
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop: Projects Part 1️⃣Best Practice

Happy Wednesday community!Β The Power of Projects: A PrimerBefore you dive into the exciting world of data transformations and analysis, let's lay down the foundation: projects. Think of projects as your organized toolkit – a collection of plans, components, data files, workflows, and custom scripts, all neatly arranged within logical folders. These projects live within a workspace folder, ensuring everything is in its right place.Β Structuring ProjectsImagine the harmony of a well-structured project. Here's how it can look:A Glance at a Well-Structured Projectbat: Home to batch or shell scripts for triggering plans. bin: Houses DQC plans for data cleansing and deduplication. components: Stores the goldmine of component plans for main plans. data: Nurtures your data directories. ext: Contains external lookup files and replacement lists. bat: Serves batch or shell scripts to trigger plans for lookup files. build: Holds DQC plans for crafting lookups. lkp: The indexed version of looku

930
A
F
Data Pioneer
FCB developerData Pioneer
asked inΒ Data Quality & Catalog βš™οΈ
Π‘hanging stewards for all business terms

Hi everyone!How can we avoid manually changing stewards for all business terms and catalog items in Ataccama One?Β I will be very grateful for your helpThank you!

621
Albert de Ruiter
Rocket Pioneer L1
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
ONE Desktop: Environments 🌍Best Practice

Hi everyone!Β I hope you are all excited to take one more step into ONE Desktop today πŸ‘£Today on the menu we have…..Environments!Imagine effortlessly crafting and testing your solutions across different operational landscapes like DEV, TEST, or PROD. That's what Environments are all about! These versatile tools offer a seamless platform for:Running plans on diverse databases without building separate plans. Testing plans, workflows, and web services across testing and production servers. Crafting and exporting runtime configurations for various environments (perfect for those snazzy folder shortcuts).How Environments Work: A Guided TourEnvironments are your game-changers. Here's a sneak peek at how they operate:Local Launch EnvironmentWhen you're running your plans or workflows on your local machine through ONE Desktop, this environment kicks in. It's your go-to for efficient local testing and development.ONE Platform Launch EnvironmentFor those times when your plans need to be executed

1170
A
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
#1 Adding Data SourcesGet certified with Ataccama

πŸ‘‹ Welcome to our new article series, where we will go through the individual topics that are part of the Ataccama Data Quality & Governance Certified Professional: Foundation Level Ataccama ONE v13Β certification exam.We will focus on practicing and pointing out the most common mistakes. Our first topic isΒ Adding Data Sources.Β πŸ”ŽΒ LET’S TEST YOUR SKILLSUsing ONE Desktop Application, create a database connection to your database. ​​Using the ONE Desktop Application, upload the 'bloodtypes.xlsx' into your database. Create the database table <your_name>_bloodtypes, and upload the contents of the 'bloodtypes.xlsx' file.Β πŸ’‘ How to do it?Open Ataccama ONE Desktop Application and create a new projectIn your newly created project, create a plan and name it loaddata.planΒ Add (drag and drop from the palette on the left side) steps Excel File Reader and Jdbc WriterUpload bloodtypes.xlsx (attached to this article) into your project Open Excel File Reader and fill in File Name, Sheet Name

3881
F
Data Pioneer
J
Universe Traveller
Jason.SupticUniverse Traveller
posted inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Greyed out fields appear to change with Enrichment... but they do not (thankfully)!

Howdy Ataccama Community:If you are needing to know how to grey out fields for edit, please navigate here:Β Grey out fields for edit | Community (ataccama.com)Problem: After greying out fields, I noticed that fields an Enrichment acts on appeared to change in the User Interface, even if greyed out.Solution: Working with our Ataccama professional support person, it was found that although the fields change in the UI, when saved the greyed out fields are not updated in the record being edited. This is great news, as the purpose (at least for our team) of greying out a field, is so that it cannot be changed once created.So that’s it, probably the shortest post I will put on here, but an important one for record reliability.Thank you,Jason R.S.

170
J
Universe Traveller
A
Ataccama Community AdminIntergalactic Expert
published inΒ Data Quality & Catalog βš™οΈ
Tasks πŸ“ƒβœ”οΈBest Practice

Happy Wednesday everyone!Today’s topic is Tasks! We will cover how to optimize your task management on Ataccama to effectively organize and track your work. Let's dive in!Β A Glimpse into Task ManagementAt the heart of the Ataccama ONE PlatformπŸ’œΒ lies the Task management system. This tool brings together your tasks into a Kanban board, offering an intuitive visual representation of your workload and its progress. From prioritizing tasks to altering their status and engaging in collaborative discussions, this system is your gateway to productivity.Β Unlocking the Potential of TasksWithin the Task management system, you can accomplish a multitude of actions: Task Creation and Management: Create tasks and their subtasks, assign them to team members, and even solicit assistance from colleagues. Change Management: Request changes like entity creation or access adjustments seamlessly. Workflow Task Management: Efficiently handle tasks related to access requests, approval workflows, and mor

910
A
A
Data Voyager
anagha.kData Voyager
asked inΒ Data Quality & Catalog βš™οΈ
Where do we get Monitoring project ID?

Hi Team,I want to use the DQ monitoring project results component. From where do we get theΒ  Monitoring project IdΒ for the component?Thanks in AdvanceAnaghaΒ 

985
anna.spakova
Ataccamer
alexAguilarMx
Data Pioneer
alexAguilarMxData Pioneer
asked inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Multi-load operation :: Why data acquisition is slow?

Hi Every one,Β Why the data acquisition in a multi-load operation could be so slow?Checking the database, the query to get the data is taking less than 10 minutes. Do you have any guidance about it?

512
alexAguilarMx
Data Pioneer
J
Universe Traveller
Jason.SupticUniverse Traveller
posted inΒ Master Data Management & Reference Data Management πŸ›°οΈ
Errors in LKP file with multiple columns

Howdy Ataccama community:Today I am creating an LKP file with multiple columns for the first time, and came across a couple rookie mistakes I was making, and wanted to share the warnings and errors I am getting.The file I am making is a simple mapping file going from the key to a field β€œPH1”Rookie mistake number 1: The file being used to make the lkp is a csv, and I made the assumption the Text File Reader in the lookup builder planΒ would recognize this from the file type. It did not, it had the separation of columns as \tΒ (tab) still. This created the warningΒ [EHS:SHORT_LINE][step:Text File Reader][line:1]Β with a row for ever row in my file. To fix this one was simply setting the β€œLine Separator” field in the Text File Reader to a comma instead of \t.Rookie mistake number 2:Β After correcting my first mistake, I was now getting the following errorΒ β€œThere were logged 1 runtime warnings and 1 runtime errors.” With the details beingΒ β€œError sending response” with no info on why. I got frus

942
A

Badge winners

  • 10 ideas πŸ’‘
    akumahas earned the badge 10 ideas πŸ’‘
  • First idea πŸ’‘
    romizhas earned the badge First idea πŸ’‘
  • First idea πŸ’‘
    kellymremzilybhas earned the badge First idea πŸ’‘
  • First idea πŸ’‘
    Francois Ascanihas earned the badge First idea πŸ’‘
  • 20 ideas πŸ’‘
    ayounghas earned the badge 20 ideas πŸ’‘
Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, andΒ have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings