Discuss data, share best practices, learn, and grow with Ataccama's Community.
Recently active
Hi everyone,Β In this post, we will continue our introduction to Master Data Management (MDM), if you have missed our first post you can find it here ππ»This article will focus onΒ the architectural details of MDM, breaking down the components, dependencies, and practical management tips β¨Β The Pillars of MDMAt the heart of MDM, we find four essential components:Keycloak: Our third-party identity management tool. MDM Server (formerly known as MDC): The robust Ataccama MDM processing engine. MDM Webapp (formerly known as MDA): The user-friendly Ataccama MDM web application. Relational Database: The sturdy backend database, supporting PostgreSQL, MS SQL, or Oracle.Each of these components is configured as a Linux service, allowing independent restarts and individual log access for issue identification.Β The Backbone of DatabasesMDM is compatible with PostgreSQL, MS SQL, and Oracle relational databases. However, it's important to note that Ataccama officially supports only PostgreSQL Standal
Hi everyone, happy Monday ππ»Β After publishing more than 20 articles onΒ ONE Desktop best practices, Iβm happy to share that we are ready to take the next step into Ataccama's Master Data Management (MDM), a robust, scalable, and highly available platform designed to simplify and enhance your master data management tasks. Letβs start with the basics β‘Β Β What is MDM and how is it different?Β Unlike the traditional "data flow driven" Data Quality Center (DQC), Ataccama MDM stands out as a fully metadata-driven data processing system. It incorporates an array of powerful tools, including the Advanced Metadata Editor (AME), a cutting-edge data processing engine (NME), and the DQC Plan Executor (complete with its steps and components).By doing so, MDM naturally extends the capabilities of the DQC, offering comprehensive data quality management functionality. In essence, MDM combines Master Data Management with a strong emphasis on Data Quality as a fundamental part of the process, providing a
HiΒ I am not able to configure Ataccama Server. Can anyone help me on this?Β I am getting below message when I double click on βstart-server.batβΒ Using java at: "C:\ataccama-one-desktop-14.5.0\jre\bin\java.exe"Using DQC Β at: "C:\ataccama-one-desktop-14.5.0\runtime"Using java at: "C:\ataccama-one-desktop-14.5.0\jre\bin\java.exe"Using DQC Β at: "C:\ataccama-one-desktop-14.5.0\runtime"Picked up _JAVA_OPTIONS: -Djavax.net.ssl.trustStore=C:\Windows\Sun\Java\Deployment\trusted.certsPicked up _JAVA_OPTIONS: -Djavax.net.ssl.trustStore=C:\Windows\Sun\Java\Deployment\trusted.certsWed Nov 01 16:25:19 IST 2023 : Could not listen on port 1529 on host localhost:Β java.net.BindException: Address already in use: bind2023-11-01 16:25:20 [INFO] Β Β com.ataccama.dqc.server.bin.OnlineCtl messageId=generic_msg message=Starting server on localhost2023-11-01 16:25:21 [ERROR] Β com.ataccama.dqc.commons.license.LicenseManager messageId=generic_msg message=DQC license not verified! Program will exit.2023-11-01 16:2
I am following the tutorial toΒ install and run the latest MDM example projectΒ onΒ Windows.When I am starting the MDM Server using start-mdm-server.bat, I am facing the below error:Β Can anyone guide me on how to add a user role to Keycloak for this ? Keycloak demo version is being used in this case (Β Keycloak 21.0.1)
Iβve created a basic workflow that utilizes the API to execute an existing project/plan.Β The project had an post process component that writes the invalids to an Excel spreadsheet.The other steps in the workflow wait for the file to be created and load it into DQIT.Β We can write directly to DQIT due to an issue with Databricks/Spark.Β The workflow works great, but I need to manually execute it.Β Iβd like to us Admin Center to βSchedulerβ to execute the workflow.How do I do that?The environment is Azure / Linux.Β Β
I am trying to use Splitter to transform the stringExample : Input: BeWhoYouAreSplitter output - βe ho ou reβ - skipping the first letter as βUpper caseβ is the separatorDesired output - βBe Who You AreβHow do I achieve this? Also if the string is all lower how to split it?Example: Input: βbewhoyouareβdesired output - βBe Who You Areβ
Hi everyone, happy Wednesday!Β In this post, we will walk you through Model projects! Model projectsΒ play a crucial role in configuring various Ataccama products like DQ Issue Tracker, ONE RDM, and ONE MDM. These model projects are structured based on model templates, which define a specific set of nodes, folders, and files during project creation or import. It's important to note that you cannot modify or remove nodes created by a template.The configuration within model projects is developed using the Advanced Metadata Editor (AME). In this context, metadata refers to the configurations made within the model project nodes, which are used to generate plans, components, web service configurations, XML files, and other necessary components for a particular solution.Developing solutions based on metadata is carried out in the Model Explorer view, and detailed instructions for each product can be found in the respective Developer Guides.Every Ataccama build includes a default model project
πΒ Links to the legacy documentation portal are no longer available.Β Read about the changes and how to access our documentation here.Hi community!In this post, we will discover how to create server connections in ONE Desktop for various tasks and workflow steps. Read on to find out more πΒ How to register a new server?To set up a new server connection, follow these steps:Right-click Servers in the File Explorer and select New Server. Β Choose the desired Environment. Learn more about Environments. Β Provide a unique server Name and select the appropriate Implementation. Configure the server connection according to your implementation, and click Finish.Once registered, the new server connection will appear in the File Explorer tree.Please note that, during development, right-clicking the server connection and selecting Connect is not considered a valid connectivity test, and it's not necessary for enabling step communication.Β Implementation TypesGenericUse this implementation to conn
Hello. i have a general material data csv file, and in that i have to check for duplicates based on material data. Currently i'm trying with the matching step by creating partitions and key rules but still i'm not sure about it. So is there anyone knows about solving this process?? Thanks!
Hi everyone,Β Β We are here with Part II of our VCIs best practice series and today we will delve deeper into two significant aspects of working with VCIs - connecting to natively supported data sources and non-natively supported data sources.In the first post, we explored the creation and transformation of Virtual Catalog Items (VCIs) within Ataccama ONE. You can check it out here ππ»Β Connecting to Natively Supported SourcesAtaccama ONE makes it simple to connect with data sources that are natively supported by following these steps: Create a VCI, as explained in the previous section. To read data from a natively supported source, add the "JDBC Reader" step to your plan. Depending on your transformation requirements, you can use the JDBC Reader step alone or combine it with existing catalog items. Configure the "JDBC Reader" step as follows: On the "General" tab, in "Data Source Name," provide the source identifier of the connected database. You can find the source identifier in
Hi everyone,In this post, we will talk about Virtual Catalog Items π¦ You might have participated inΒ our live training session that took place a few months back where we talked about how to create and work with VCIs. You can always watch back the session and check out the live demo hereΒ Β ππ»Virtual Catalog Items (VCIs) provide flexibility and efficiency when working with data in the ONE Desktop platform and in this article you can find best practices for creating and transforming Virtual Catalog Items to make the most of your data management efforts. Letβs start with the basics:Β Why Use Virtual Catalog Items?VCIs are particularly useful in several scenarios: Applying Transformation Logic: When you need to apply ONE Desktop and in-source (SQL) transformation logic before working with raw data. This includes tasks such as filtering data, joining tables, and standardizing values. Integrating Non-Native Sources: When you want to integrate data from sources that are not natively supporte
πΒ Links to the legacy documentation portal are no longer available.Β Read about the changes and how to access our documentation here.Β Hi there!Β Lisa from Ataccama here. Iβm a Solution Consultant and I love exploring the different ways our platform can become part of the business and technology ecosystem for our customers.In this post, I will share a framework for integratingΒ Ataccama with a work management or issue tracking software like Jira. Delivering data quality insights and alerts from Ataccama to an issue tracking tool can help to maintain transparency and accountability, and to facilitate cross-functional team work.Β Letβs explore an example!Β Use caseΒ Ataccamaβs Data Quality Monitoring Projects are a powerful tool for regular data quality evaluation and reporting on the results. You can configure DQM Projects to run periodic evaluation of data against DQ Rules, and provide various insights to the business. Do you need to send a digest whenever fresh DQM results are ready? Or do
Hi everyone, happy Monday!Β In this post, we will go through a few best practices onΒ how you can work with data in ONE Desktop. Before getting into it, donβt forget to check out some posts you might find useful:Β Browsing Data in ONE ExplorerFrom ONE Explorer, you have the ability to access various components of your data, much like you would in the Knowledge Catalog of the ONE Web Application. ONE Explorer in ONE Desktop includes:Catalog Items Components Lookup Items Monitoring Projects Rules Terms Β This seamless access to your data provides a comprehensive overview of your information, enabling you to manage your data assets with precision.Working with Data in ONE DesktopThere are two primary methods for reading and working with your data within ONE Desktop:1. Catalog Item Reader Step:This step allows you to access data from different sources defined in the Metadata Management Module (MMM). When you drag and drop a Catalog Item from MMM into your plan, the Catalog Item Reader step is
Hello,We are trying to give access for invalid samples in our monitoring projects to our colleagues.Β The coleagues are in a role that they have data access to the Catalog items, and data access in the monitoring project, but they still are not able to see the invalid rows.Which access type should be granded for invalid samples in monitoring project ?
Needed to know how to create a rule having an attribute of STRING datatype for extractingΒ current date and past dateΒ in One Web. Tried using date functions in string using advanced expression (today() is ATTRIBUTE) ..showing error in testcase. But when tried using sameΒ functions for DATETIME datatype working fine. Can anyone help?
I have scheduled certain workflows and running on daily. Due to change in component/ plan optimisation I expect run time changes in the workflow comparing to previous version of the workflow., similarly I did for many workflows and would like to see overall performance improvement in the workflow run.Β Is there any way IΒ can go ahead and download or view the finished instances run times Β for all the workflow in a single place?Β
How I can use create sql catalog on s3 files (csv, parquet) in Ataccama One Web Knowledge Catalog.Do Ataccama use AWS S3 CData drivers
Whenever IΒ run the plan file the output file get updated and the older output file data get lost . Is there any way ataccama can keepΒ the files that is generated for all the time the plan file ran ?Β Β Β
Hi everyone,Β One of the core capabilities of ONE Desktop is metadata handling through ONE Metadata Reader and ONE Metadata Writer Steps. These steps allow users to read and write data from the Web Application, enabling a range of use cases and implementations. In this post, youβll find out how to read & write metadata using the steps in ONE Desktop ππ½πΒ ONE Metadata ReaderThe ONE Metadata Reader Step serves as a tool to read data from the Web Application, offering flexibility in usage:Common Implementations:Enriching data for external Business Intelligence tools (e.g., Tableau, Cognos) to create customized reports with additional attributes such as description, owner, and custom properties. Easily incorporating DQ rules into DQC plans, simplifying the process of finding and integrating rules based on name and input attributes. Constructing DQC plans with data transformations by utilizing data sources from the Metadata Management Module (MMM). Generating comprehensive HTML documen
Hi community!Β In this post, we are covering working and searching in metadata. Efficiently working with metadata is crucial for ensuring data quality, governance, and effective decision-making.Ataccama has several tools within ONE Desktop to assist in metadata management, including the ONE Metadata Reader and ONE Metadata Writer Steps for reading and writing metadata, as well as the ONE Metadata Explorer for seamless navigation. Letβs dive in πΒ Navigating with ONE Metadata ExplorerONE Metadata Explorer allows you to connect directly to your Ataccama ONE Platform server from ONE Desktop, granting you access to your critical metadata and even "Meta Metadata" for more in-depth analysis.Β Metadata Exploration by DefaultWhen you first access ONE Metadata Explorer, it's configured for exploring your primary metadata.Delving into Meta MetadataHowever, if you wish to explore your "Meta Metadata," you can easily adjust the settings:In the ONE Metadata Explorer, click the "View Menu" icon. Selec
Hi Community!This week on the menu we have how to work with the Ataccama ONE Platform. We will cover basics like how to connect your ONE Desktop to a bit more advanced topics like working with metadata.Connecting your ONE Desktop to the Ataccama ONE Platform allows you to seamlessly work with metadata, create data quality rules, and utilize virtual catalog items.Β Create a New ConnectionPrerequisites:Ensure ONE Desktop is properly set up and running. Your ONE instance should be running and accessible from the server where ONE Desktop is installed. Ensure that both ONE Desktop and ONE are using the same version.Follow these steps to create a new connection: Open ONE Desktop. In the ONE Explorer, select Create a new server. Β If you are adding a second ONE server, follow these steps instead: In File Explorer > Servers, right-click Servers and select New > ONE Platform. You can also find the same option in the Servers and Databases tab, which is part of your workspace by default.
Hi, we need to see if for terms (or other entities) certain roles have viewer rights. (...and if not set them)Below is a GraphQL, but I need to filter only the term.assignedIdenties.capability.OperationSet where the name is "viewerβ. How would I go about doing this? (outside filtering in ONE Desktop of course)query GetCapabilities_term($gid: GID!) {Β term(gid: $gid) {Β Β gidΒ Β assignedIdentities {Β Β Β grantingStrategyΒ Β Β capability {Β Β Β Β nameΒ Β Β Β idΒ Β Β Β operationSets {Β Β Β Β Β nameΒ Β Β Β Β displayNameΒ Β Β Β Β idΒ Β Β Β }Β Β Β }Β Β Β identity {Β Β Β Β ... on Role {Β Β Β Β Β nameΒ Β Β Β Β __typenameΒ Β Β Β Β ... on GroupRole {Β Β Β Β Β Β nameΒ Β Β Β Β Β numberOfUsersΒ Β Β Β Β Β gidΒ Β Β Β Β }Β Β Β Β }Β Β Β }Β Β }Β }}
For a use case we need to run DQ on CSV's loaded into HDFS (only). So no hive externally managed tables are available. Our use case is much like the desktop tutorial Hadoop (01.0.0 Read whole folder - MAP.plan), but we'd like to create VCI's and update them once a dayΒ for profiling, DQE and sample data.Two Q's:How do we connect tot HDFS? (Simple JDBC we use in DBBeaver doesn't work) How do we update the VCI's?
Hi,I need to create one rule which is used to check whether last date of current month is available in that column or not. For that I need expression to get the last day of current month.Note:Today is 2023-10-11. So I need to get 2023-10-31.orΒ if today is 2023-09-04 then I should get 2023-09-30 as output.Β Can anyone provide a solution for this?ThanksAnagha K
Hi everyone,In this post we will cover databases that play a vital role in Ataccama solutions, serving as data sources, and repositories, and enabling data profiling. Here are some tips & tricks to help you effectively work with databases in Ataccama.Β Connecting to a Database To establish a connection with a database, right-click the Databases node in the File Explorer and select New Database Connection. Β Configure the connection by providing the following information: Database type: Specify the database connection type. Connection name: Define a name for the new database connection. Host: Specify the hostname or IP of the database host. Port: Provide the port allocated for the database connection. Database name: Catalog: Specify the name of the database instance located on the host. Username: Enter the username to connect to the database. Password: Enter the password to connect to the database. Β Use the Edit Driver Properties... option to set additional JDBC propert
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.