Discuss, learn, ask, suggest, share all things DQ&C π
Recently active
Hello everyone, and happy Wednesday!Β Today, we are continuing our Getting Started with ONE DG series with Catalog Items πΒ Catalog items are essentially records within the data catalog. To access your data profiles, simply click on "Catalog Items" located on the left sidebar. By default, Catalog Items are organized into different groups within their respective tabs:Anomaly Overview: This tab displays a list of all catalog items that have anomalies. Domain Suggestions: Here, you'll find a list of catalog items with suggested terms that you can accept or reject. Master Data: This tab contains a list of catalog items sourced from MDM (Master Data Management) data sources integrated with the Ataccama platform. Reference Data: In this tab, you can find a list of catalog items sourced from RDM (Reference Data Management) data sources integrated with the Ataccama platform. Not Monitored: This tab displays a list of catalog items that are currently not included in any monitoring projects. Repo
Hi everyone!Β This week, we are continuing to explore the Ataccama ONE platform basics. Today, we are looking into Tasks and Collaboration.Β Collaborate with Your TeamCollaboration plays a vital role in any project, and ONE DG makes it simple for you. You can easily comment and initiate conversations with your team members by using the comment section. To notify someone specifically, just tag them using the "@" symbol. You can even add emojis and gifs to make your discussions more lively! Don't worry about missing any updates - you'll receive in-app notifications whenever someone tags you or responds to your comments.Β Streamline Your Workflow with TasksThe Task management system in the Ataccama ONE Platform helps you keep track of your work and ensures efficient progress management. The system organizes tasks into a Kanban Board, providing a visual representation of your workload. With tasks, you can perform the following actions:Create and manage tasks and their subtasks. Assign tasks t
Hi Everyone,Β Β Β Β My DQΒ and KeyCloak is running integrated for authentication but the password in the attribute name=βtruststore-passwordβ is using password without encryption.Β Β I tried to encrypt via command encrypt.bat on DQΒ Β butΒ without success. I just have a success in the authentication using clear password.Β Β Is there someone that have success to encrypt the value for the property truststore-password with keycloak?Β <config class="com.ataccama.server.keycloak.KeycloakDeploymentContributor">Β Β Β Β <keycloakConfigs>Β Β Β Β Β Β <keycloakConfig name="keycloak-admin-center.json">Β Β Β Β Β Β Β Β <!-- Define common parameters for all clients. They can be overridden by client-specific settings.-->Β Β Β Β Β Β Β Β <url>https://srvkeycloak:8446/auth</url>Β Β Β Β Β Β Β Β <realm>ataccamaone</realm>Β Β Β Β Β Β Β Β <attributes>Β Β Β Β Β Β Β Β Β Β <!--<attribute name="ssl-required" value="external"/> -->Β Β Β Β Β Β Β Β Β Β Β Β Β Β Β <
Hi all!Β Hope you are enjoying our getting started series π In this post, we will cover Business Glossary and learn more about Business Terms.Β Business terms play a crucial role in analyzing data, as they provide the basis for evaluation and quality checks. You can establish expected values and gain deeper insights by assigning terms to specific data elements. The Business Glossary automatically maps these terms to real data sources, ensuring that your catalog is always up to date and synchronized with the latest assigned terms. Plus, an unlimited hierarchy structure lets you visualize the relationships among different data elements, systems, and processes.Β Let's explore the different types of terms you'll encounter in the Business Glossary:Business Terms: These are predefined labels that help you understand and organize your data. They are assigned based on set conditions. Custom-Created Terms: This term group allows you to create your own terms according to your specific needs. For e
Hello all, IΒ have a task of detecting duplicate records from particular tables. So is there anyone who know about creating rule for this, a rule to detect duplicate values and mark it using One Web and One Desktop?
I have a workflow(.ewf file) which in turn calls several components and workflows. How do i run the workflow and get to know the trace of which all components and workflows are executed?Is there any property that can be enabled which will give the components and workflows that are getting executed? If there is some kind of information like input to each step and output from the step that also would be usefulΒ
From the ataccama one url, click environment settings, create environmentΒ we have the below propertiesΒ executor endpointadmin centerexecution modeΒ then we have the option to add a datamart, with a jdbc string.Β Β What exactly is the executor endpoint, admin center and execution mode?is the datamart a place(database) to store something from the executor(server)?Please provide some links which explains this.
I need the jar files to create a neo4j database connection from ONEΒ desktop.Can anyone please help?
Hello,I want to use βSFTP Download Fileβ and use a wildcard for the filename and a date.Β IsΒ that possible ?Β Β For example :/<mydirectory>/input/MYFILE_<TODAY_YYMMDD>_*.txtΒ When i enter an asterisk symbol in theΒ Β βFile or Directoryβ section, IΒ get an error.Β Thanks,Rene RequenesΒ
Hi there! Shafeeq from Ataccama here. Iβm a Software Support Engineer and occasionally I come across issues that are shared among different customers.Β One of those issues is how one can get Ataccama tools to preserve spaces before and/or after certain strings. This might be when trying to perform certain operations using the Column Assigner in the IDE, or in a monitoring project running on the Web App where a DQ rule checks that a field does not have trailing spaces, etc.Β Β The solution to this problem is simple, in the form of Java VM arguments. These Β are typically values that change the behaviour of the Java Virtual Machine (JVM) which is used by our products. Below, Iβll shareΒ the solution.Β Β When this issue arises for a plan, workflow, etc. in the IDE, you may follow the steps below:Β Β Click on the arrow next to the run button and then click on "Run Configuration". Β Click on the Runtimes tab and add the following parameter in the VM arguments text box:Β -Dengine.trimStringValues=f
Can anyone provide a document or link on how to effectively navigate and interpret the logs in the OperSearch Discovery portal? Our client is usingΒ Ataccama PAAS. And sometimes we go toΒ OperSearch Discovery for logs.Β I find challenging to understand. For instance, to locate runtime errors, I need to input "Kubernetes.container_name = runtime-server" in the filter section, followed by "error" in the search section. This will display any errors related to the runtime server. Otherwise, the log page appears to be quite intricate. Any help or link is very much appreciated.
HiIβm Using the Windows DQ Analyzer - free version.I have used the business rules functionality, but I have been wondering what the idea behind the βnullβ value is? And what makes data fall into the βnullβ value?Br.Gregers
Which ONE Metadata Reader entity type will let me read the all suggested business terms against all items?Β Β Β
DPM and DPE - Ataccama ONE Gen2 Platform LatestOnce DPE connects to a data source, it creates metadata and optional sampling of records that do not satisfy DQ rules.Could you please answer the following questions or direct me to a link that can answer the below questions?1. Does DPE create other temporary files as a result of processing as well?2. What happens to the metadata, sample records that do not satisfy DQ rules and any temporary files generated during processing after the connection to the data source get closed?3. How does purging of data works in Ataccama?
In Ataccama ONE Desktop, we have steps like JSON Reader, or JSON Parser, to read JSON data and put them into a data flow.We also have a JSON Call step that allows you to construct a JSON template to send JSON Calls outwards.But what if you wanted to transform the data into a JSON string but not necessarily send it anywhere? If thatβs what you want to do, read onβ¦Our dataLetβs take an example of this data:I have some fruits:id name 1 Apple 2 Orange 3 Pear 4 Grape Β I have also some fruit variety names:id variety fruit_id 1 Gala 1 2 Golden Delicious 1 3 Jazz 1 4 Pink Lady 1 5 Navel 2 6 Blood 2 7 Seville 2 8 Conference 3 9 Comice 3 Β Join them together as our base data:(See json_1.comp in zip)Output looks like this:Β π‘ I want to now turn this into a JSON string. Variety should be an array within each fruit.To achieve this, we first need to aggregate the variety back into fruit, and then a final aggregation to get the whole set
What does the id mention in invalid_rules column in Monitoring Project Export refer to?the alphanumeric term within() after rule name as in the image belowIs this the ruleId of the rule? If not, what does this refer to?Is there a way I can configure Metadata Export step to give rule id?
The idea is to divide the invalid_rules column as obtained in export for Monitoring Project into two columns- one containing the rule name and the other containing the explanation; and implement this for all the rules mentioned in a cell.For example,Β The below cell would be divided into invalid_rule_name column containing two records-validation [USA] State and calidation[Noth America] Phone Numbers, and invalid_rule_explanation column containing explanation for the respective rules.The challenge I am facing here is that the rule ID mentioned in Monitoring Project report export is not the same as the rule ID mentioned in the list of rules I export; which I am trying to use to identify the rule in invalid_rules column from Monitoring Project report.Please help me find a solution for this
βHello everyone!Β Foreword and problem statementThere are several subcategories of terms. I need to set up access for managerΒ and stewardΒ ofΒ each subcategory so that:The steward of certain subcategory is able to create new terms, edit old onesΒ in his subcategory only. However, he can not publish any changes The manager, in addition to above, can publish changes, but also in his own subcategory only.I have theΒ list ofΒ questions concerning access management in v.14.2:Is it possible to grant ownership to node? And if itβs yes, how can I implement it? Do you know any way to solve my manager-steward task? (there are my understanding of the solution below) In States of approval workflowwhat are theΒ βUser rolesβ : governance or identity provider ones? (looks like the second ones, but doesnβt work) what isΒ the βOWNERβ for this version of workflow?_______________________There is a long prelude to where the questions came from.Β Version: 13.9There is the ability to create a special capabilityΒ to b
METADATA MODELΒ The metadata model is used toΒ configure the application entities and their properties. TheΒ Metadata ModelΒ describes the entities and their properties in the application.Β Β Web Application) use theΒ Metadata ModelΒ to create the frontend user interface.Β The metadata model is also responsible for the configuration of the left navigation menu of the application.Β ONE METADATA PROPERTY REFERENCESEntities consist of properties of SimpleΒ types, such as strings and integers, andΒ ObjectΒ types, which link the entity to other entities.Β ADDING NEW PROPERTY TO THE MMM MODELGo to left panel of the Webapp and choose Global settings Β Choose metadata model under Maintenance CenterChoose the Custom filed where you need to add the new property to CatalogItem, Glossary and sources. In our case we adding to CatalogItem Β Click on the Custom field CatalogItem. We can see General Information and Property. Click add property and type in the property to be added. Β Fill in the name and choose the
Hello,Β I have a speedΒ of profiling data for using Oracle DB of about 4000-7000 rows per minute.I useΒ Ataccama DQ Analyzer (Version:Β 11.1.1.ga-2017-06-14).Β Β Please, can you take me to advise me on howΒ to increase the speed of profiling data for using Oracle DB?I hope to take the profiling speed to about 100 000Β rows per minute...Β Β Connation config:Β Β Β Log:17.03.2023 12:56:04 [INFO] Switching to logging setup from C:\Users\ECHERN~1\AppData\Local\Temp\loggingConfig3307204922660014311.tmp17.03.2023 12:56:04 [INFO] Using following licenses:17.03.2023 12:56:04 [INFO] C:\Ataccama DQ Analyzer 11\runtime\license_keys\license.plf17.03.2023 12:56:04 [INFO] Using DQ Analyzer configuration file C:\Ataccama DQ Analyzer 11\workspace\sqb_prof_pilot\data\out\sqb_all_table.plan17.03.2023 12:56:04 [INFO] Using runtime configuration file C:\Users\echernoskutov\AppData\Local\Temp\runtimeCfg1065525328424309714.tmp17.03.2023 12:56:04 [INFO] Changing CPU affinity to 4 CPU(s).17.03.202
We are using some GraphQL to βdo stuffβ. Turns out the easiest way to find the GraphQL is using the Developer-tools in Chrome or Edge.Β See enclosed.Β
Hello everyone, Anna here! I work as a projects consultant in Ataccama and today I put together a guide for you on how to export a metadata model definition from Ataccama ONE.Ataccama ONE allows to export metadata model definition from the platform into a JSON file. However, the JSON file is not an ideal format for business users. For that reason, parsing of the JSON into more user-friendly format, for instance Excel, is desired. The current JSON parser step in Ataccama DQC, however, is not able to parse this JSON structure.This article introduces a Python script that was developed to parse the JSON structure of the metadata export and provide the metadata model in Excel and CSV formats. It is supported for v13 of Ataccama ONE.As a first step, the metadata model has to be exported from Ataccama. Next step is to run the Python script on top of the exported metadata model to obtain the CSV and Excel format. Chapter JSON parser in Python describes in detail the Python script as well as th
What kind of MCQs are asked in Certification exam?
How are NULLβs considered? I have a case statement which gives the right value matching measure -used debug to test. But when it comes to matching rules, it skips the rule.Β Example:NameΒ Β Β NPIRamonaΒ 12345678NULLΒ Β Β Β 12345678the 2nd row should match the 1st on NPI value and considering on of them is null.Even the scenarioNameΒ Β Β NPINULLΒ Β Β Β 12345678NULLΒ Β Β Β 12345678these two should match together based on NPI. I do not see a special character. If I replace NULL with a value βAβ , they match with rule Matching on NPI. but does not work for NULL.Β Β Any thoughts or suggestions appreciated.Β
Hello,Β Iβve got a question while using DQ evaluation rules.I have been creating rules with valid and invalid messages but just the invalid message at the end does not display as intended.Β But when I view this in the results tab it just says OTHER as the explanation.Β Β Β Β Is there a workaround to this problem?Β Should I edit my rule logic to include some specific condition and leave the rest to fall under the βIF none of the conditions above apply THENβΒ TIA for the replies.Β Β
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.