Discuss data, share best practices, learn, and grow with Ataccama's Community.
Recently active
Hello,Has someone usedΒ $environment$Β variable in the subject line in RDM notification or any other notification emails?Β We are experiencing an issue with the RDM when using theΒ environment variable within our email notification template.Β When we configure the template to include $environment$ in the subject line, the received emails do not display the variable.Β Β Thanks,Ojaswini
Speaking about data quality rules, we usually refer to DQ rule configuration in ONE web application. However, when it comes to referencing external sources or performing preliminary work or combining different sources, we may find that the capabilities within ONE Web are insufficient for our needs. For example, consider a scenario where we need to validate whether our data matches an external data source, such as ABN Lookup on https://abr.business.gov.au/. This is where component rules come in handy.The Task at Hand: Validate ABN Information.To implement this data quality (DQ) check, we require a component rule.Here's how you can configure the ABN validation component and use it for DQ evaluation: βββββββConfigure a placeholder for the component Configuring Logic for the Component using ONE Desktop application Adding Changes to Runtime Configuration in ONE Web Application Configuring the DQ component rule Applying the RuleβββββββConfigure a placeholder for the componentNavigate to DQ s
Hi,I am looking for a plan to export column names for a catalog item in One Web. Can someone help me with it?Thanks
Hi Ataccama community,Iβm hoping someone with better grip on the Ataccama GraphQL model can shine a light on this.Β For internal reporting, we are looking to retrieve data quality details via the Ataccama GraphQL API. One of the requests it to produce an overview of the DQ score across the (ruleβs) DQ dimensions. This is basically the same as is shown in the web UI when navigating to the βData Qualityβ tab for a Term. It shows the score broken down by dimensions, based on how the rules are defined.What Iβm trying to do is retrieve the passed/failed rows at this level, so for each term and each dimension. This would allow me to summarise this into a score per dimension.However, Β I am unable to get this out of the API.Iβve approached this from both the βtermsβ and βrulesβ pathway. In both approaches I always end up with the βOverall Qualityβ score for each term - not the per dimension details.The βrulesβ query looks promising, but ultimately only returns the passed/failed records for the
Hi, Iβm trying to add the data value to the file extension in the component but Iβm not able to get the expected results. I have added a new string column using alter format step (tostring(today(), 'yyyy-MM-dd'). When I pass the column as $date or ${date} in the output filename, IβmΒ not seeing the data on the output filename.Β
Can anyone guide me on how to generate an audit for every action, log it, and integrate the logs with the UI in Ataccama One Desktop? Any best practices or examples would be greatly appreciated!
Unable to start keycloak locally on a Azure DevBox, When I run the keycloak.bat file from OneDesktop, it is immediately terminated as per console logs.Β Can anyone help with the diagnosis?
We are getting errors in the delta update flow for party and policy master and this is the error message.Β βcom.ataccama.nme.core.NmeExecutionException: Plan "/opt/ataccama/one/mdm-server-config-13.9.1.221107-11165-6e37fd24/Files/engine/export/Party_Master_Delta.comp" execution failed: runtime.error.sql.instance.processError - Transaction (Process ID 901) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim.βΒ Please advice if any help can be provided.Thanks !!
Encountering with Runtime error in JDBC reader, while running the basic profiling for a table from SQL Server.Β Full Error details -Runtime error in JDBC reader: Error creating record: column type: 'STRING', read value: '<null value>', read object type: '<null value type>', cause: Connection resetΒ Looking for ideas troubleshooting this error.
Hi All,We are facing performance issue while loading and exporting data in MDM. We have around 30 million records. We are loading data in chunks and initial few execution were very quick. After processing 20M we are observingΒ severe degrade in performance. Now even for 10K records the job is taking 8-10 hours for both load and export. We have applied parallelism wherever its possible. Is there any other thing which we need to consider/implement which will enhance the performance?TIA.
Hi folks,Iβve been given details to an SFTP server, and I can successfully connect to it using Winscp on my laptop:Β I tried to setup connection in ONE desktop:I put workflow task:But I get an error:14.11.2024 15:15:21 [INFO] Using following licenses:14.11.2024 15:15:21 [INFO] C:\Users\mkwok\license_hamilton_USER_v14_subscription_2025-01-29.plf14.11.2024 15:15:21 [INFO] ============================================================14.11.2024 15:15:21 [INFO] This is a subscription USER license for developers' laptops for "Hamilton". It expires on 2025-01-29.14.11.2024 15:15:21 [INFO] ============================================================2024-11-14 15:15:21 [INFO] com.ataccama.lib.securityutil.javaencryption.DefaultEncryptionEngine messageId=createDefaultKeyProvider reason=the 'properties.encryption.keystore' property not specified2024-11-14 15:15:21 [INFO] com.ataccama.lib.securityutil.javaencryption.DefaultEncryptionEngine messageId=createDefaultKeyProvid
Hi Team need your expert opinion on how to tag Policies to Business Terms or any other assets(Applying policies to asset types) in 14.5 version? I cannot see any options for the same is it any version limitation or any customization needs to be turned on. Any help is appreciated.
Hello all,I want to create a DQ rule, In this rule I need to check a numeric field if the value is in the predefined interval or not. Let me give an example of logic.Β I have a column X contains βtypes of productβ and column Y contains βvalue of productβ. And have a list of valid values. If the value of X is βabcβ then Y must be in the interval 1-30. If hte value of X is βxyzβ then Y must be in 1-200 and so on.I was going to create a lookup file for this one, but couldnβt be sure how to define intervals. Does someone know how to implement my DQ rule for this logic ?Β Thanks a lot !
I have an ETL process thatβs run on-demandΒ and refreshesΒ master data referenced by several Ataccama catalog items. Thereβs about 60 lookup items that are created from catalog item attributes that I need to update immediately following the master data refresh. I know thereβs a lookup item βUpdateβ button in the Ataccama web interface and a method to schedule refreshes; however, I havenβt been able to find a way to update lookup items on-demand, such as through a GraphQL mutation.Is there an available mutation for updating lookup items on-demand, or another method that could be utilized?
Taking reference of CDI Example, I have multiple contact values for the same person and I want all of these contacts to be a part of the contact master entity.Β I am not sure how the same instance record id will affect this. Will ataccama load both these records? Will both these records partivipate in match logic(same master_party_id, same_contact_type,same contact_value) or will it consider just the first one?Also, if someone can please confirm if the master_party_id is id for master record derived on processing of all party records?
Hi Ataccama community. Could you please share examples of plans used for adding a new metadata entity and bulk loading values from excel file.We have an excel file with sensitivity level definedΒ for each attribute. We need to add an entity called e.g. sensitivity level and to import this from excel file.Β Β
Issue: We receive files to be tested against our DQ rules (Monitoring Projects) every 2 weeks.Β Each set of files is referenced by its date created (aka 2024-10-01,2024-10-14...).Β When running Monitoring Projects against these items we only get the Execution Date which can be different from the file date.Β Β When extracting data for reporting from ONE we only have the Extraction Date and donβt necessarily know which file date the plan results are for (sometimes have to rerun old files for new rules etc..).Β Β Question:Β Is there a way to add a custom attribute / parameter for when we run the Monitoring Project (at the processing id level) so we can report our DQ results by File date?Β Β The 2 options we could think of:Β In extraction component connect to the catalog items and get the file date which is in the table itself (tried this but we need to extract history so would need file date, catalog item, and Monitoring Project processing id to join with DQ Aggregation Results tasks in the com
Iβve recently noticed that some jobs Iβm trying to kill, jobs which happen to running for much too long (~2 days), take a long time to kill. Is that a bug in the kill function or maybe related to the jobs themselves taking so long to run?
Hi All,I have a few observations regarding the source_id column in matching step of an entity in a MDM project.For context, source_id is a column with unique values (Primary Key in most cases) that we define to uniquely identify each record in a source system, connected to mdm hub.I was recently trying to test the match logic on OneDesktop using plan files. IΒ used a very small subset of records from each source.Load ComponentIn the project, source_id column(defined in load component) for one of these source system is βSOURCE_SYSTEM_NAME~^~pk_idβwhereΒ SOURCE_SYSTEM_NAME is as string value and pk_id is the primary key.Β match componentWhen we defined a match componentΒ in mdm project, we do not define standalone bindings but assumably βId Columnβ would be mapped to source_id column if we are testing on OneDesktop as a plan.So, when I created a plan to test the match logic, I mapped βId Columnβ to source_id (with source_id having string values as mentioned above). ButΒ when I run the plan wi
Hi everyone,Β In this post, we will cover how to do Bulk Profiling πΎProfiling vast databases with numerous tables can be a daunting task, but Ataccama's Bulk Profiling extension streamlines the process. This extension simplifies the profiling of database data sources with multiple tables. Instead of configuring analyses individually for each table, it enables you to import data structures directly from the database dictionary.Bulk Profiling then automatically generates a profiling plan for each table. To top it off, the tool creates an execution batch file that can run the individual plans. This means you can perform the entire profiling process unattended, whether overnight or at a scheduled time using a task scheduler.Β Step 1: Creating a Bulk Profiling ProjectIn the Model Explorer view, click on New Model Project in the toolbar. This will open the New Model Project dialog. In the dialog, select the Bulk Profiling model template. Click Finish. Β This initiates the process of creating
Hello everyone,Β We recently upgraded to version 13.9.3, and gained the ability to export DQ rules from one Ataccama environment and import them into another. This works great, but I need to do the same thing for our monitoring projects. I would like to be able to export them from DEV and import them into a higher environment. Β Unfortunately, there doesnβt seem to be a plan available in my version to export monitoring projects.Β Has anybody figured out a way to migrate monitoring projects without recreating them in each environment?Β Alternately, I am considering trying to write my own plan. Is there any documentation out there on how to write the JSON for export plans?Β Thanks!
If we use matillion ETL tool forΒ data integration to snowflake. Can we still see the lineage and transformation code in Ataccama?
Hi -Β I wanted more information on how to use the slices vs a SQL Catalog Item.The use case being often when Iβm leveraging a slice I want to be able to do a start date ofΒ >2023 , if it is part of aΒ Β quality program I want the data to keep updating i.e as new records come in. However, it seems like the slice is only really designed for the βpreviousβ so ignores current records i.e previous year = 2023, and I canβt set the custom date without an end date so it would be limited to a fixed time range.It seems like for the quality program is better served with the SQL CAT ITEM but I was curious if there was value with the slice I was missing or if I could accomplish the same objective with the slice ?Β Thank you,Β LindsayΒ Β
Hi,I want to create a DQ aggregation rule that checks fields from two different tables for the group by clause. Is there a way to do this directly in Ataccama One Web UI.Thanks!
Hi,I am not able to access the transformation plan feature in v15.3 from both the Transformation Plan tab and from the monitoring projects tab. Can someone explain why is there a restriction on creating transformation plans and how can I get access to it?Thanks
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.