Data Quality & Catalog ⚙️

Hey all,I was wondering if you ever came accross the following requirement and has some tips maybe to implement it.The Text File Writer step can be used to create for instance log files. For tracking or governance reasons it may be necessary to keep all log files. But because the output file name is fixed, each time the file will be overwritten whenever a new one is created. Of course it is possible to manually add a datetimestamp to the file name after its creation, but that is a rather cumbersome approach. So I would like to be able to add a datetimestamp to the file name automatically. Does anyone have a suggestion?Kind regards,Albert

Hello,Has someone used $environment$ variable in the subject line in RDM notification or any other notification emails? We are experiencing an issue with the RDM when using the environment variable within our email notification template. When we configure the template to include $environment$ in the subject line, the received emails do not display the variable.  Thanks,Ojaswini

Hi,I am looking for a plan to export column names for a catalog item in One Web. Can someone help me with it?Thanks

Hi Ataccama community,I’m hoping someone with better grip on the Ataccama GraphQL model can shine a light on this. For internal reporting, we are looking to retrieve data quality details via the Ataccama GraphQL API. One of the requests it to produce an overview of the DQ score across the (rule’s) DQ dimensions. This is basically the same as is shown in the web UI when navigating to the ‘Data Quality’ tab for a Term. It shows the score broken down by dimensions, based on how the rules are defined.What I’m trying to do is retrieve the passed/failed rows at this level, so for each term and each dimension. This would allow me to summarise this into a score per dimension.However,  I am unable to get this out of the API.I’ve approached this from both the ‘terms’ and ‘rules’ pathway. In both approaches I always end up with the ‘Overall Quality’ score for each term - not the per dimension details.The ‘rules’ query looks promising, but ultimately only returns the passed/failed records for the

Hi, I’m trying to add the data value to the file extension in the component but I’m not able to get the expected results. I have added a new string column using alter format step (tostring(today(), 'yyyy-MM-dd'). When I pass the column as $date or ${date} in the output filename, I’m not seeing the data on the output filename. 

Can anyone guide me on how to generate an audit for every action, log it, and integrate the logs with the UI in Ataccama One Desktop? Any best practices or examples would be greatly appreciated!

Encountering with Runtime error in JDBC reader, while running the basic profiling for a table from SQL Server. Full Error details -Runtime error in JDBC reader: Error creating record: column type: 'STRING', read value: '<null value>', read object type: '<null value type>', cause: Connection reset Looking for ideas troubleshooting this error.

Hi folks,I’ve been given details to an SFTP server, and I can successfully connect to it using Winscp on my laptop: I tried to setup connection in ONE desktop:I put workflow task:But I get an error:14.11.2024 15:15:21 [INFO] Using following licenses:14.11.2024 15:15:21 [INFO] C:\Users\mkwok\license_hamilton_USER_v14_subscription_2025-01-29.plf14.11.2024 15:15:21 [INFO] ============================================================14.11.2024 15:15:21 [INFO] This is a subscription USER license for developers' laptops for "Hamilton". It expires on 2025-01-29.14.11.2024 15:15:21 [INFO] ============================================================2024-11-14 15:15:21 [INFO] com.ataccama.lib.securityutil.javaencryption.DefaultEncryptionEngine messageId=createDefaultKeyProvider reason=the 'properties.encryption.keystore' property not specified2024-11-14 15:15:21 [INFO] com.ataccama.lib.securityutil.javaencryption.DefaultEncryptionEngine messageId=createDefaultKeyProvid

Hi Team need your expert opinion on how to tag Policies to Business Terms or any other assets(Applying policies to asset types) in 14.5 version? I cannot see any options for the same is it any version limitation or any customization needs to be turned on. Any help is appreciated.

Hello all,I want to create a DQ rule, In this rule I need to check a numeric field if the value is in the predefined interval or not. Let me give an example of logic. I have a column X contains ‘types of product’ and column Y contains ‘value of product’. And have a list of valid values. If the value of X is ‘abc’ then Y must be in the interval 1-30. If hte value of X is ‘xyz’ then Y must be in 1-200 and so on.I was going to create a lookup file for this one, but couldn’t be sure how to define intervals. Does someone know how to implement my DQ rule for this logic ? Thanks a lot !

Hi Ataccama community. Could you please share examples of plans used for adding a new metadata entity and bulk loading values from excel file.We have an excel file with sensitivity level defined for each attribute. We need to add an entity called e.g. sensitivity level and to import this from excel file.  

Issue: We receive files to be tested against our DQ rules (Monitoring Projects) every 2 weeks.  Each set of files is referenced by its date created (aka 2024-10-01,2024-10-14...).  When running Monitoring Projects against these items we only get the Execution Date which can be different from the file date.   When extracting data for reporting from ONE we only have the Extraction Date and don’t necessarily know which file date the plan results are for (sometimes have to rerun old files for new rules etc..).  Question:  Is there a way to add a custom attribute / parameter for when we run the Monitoring Project (at the processing id level) so we can report our DQ results by File date?  The 2 options we could think of: In extraction component connect to the catalog items and get the file date which is in the table itself (tried this but we need to extract history so would need file date, catalog item, and Monitoring Project processing id to join with DQ Aggregation Results tasks in the com

I’ve recently noticed that some jobs I’m trying to kill, jobs which happen to running for much too long (~2 days), take a long time to kill. Is that a bug in the kill function or maybe related to the jobs themselves taking so long to run?

Hello everyone, We recently upgraded to version 13.9.3, and gained the ability to export DQ rules from one Ataccama environment and import them into another. This works great, but I need to do the same thing for our monitoring projects. I would like to be able to export them from DEV and import them into a higher environment.  Unfortunately, there doesn’t seem to be a plan available in my version to export monitoring projects. Has anybody figured out a way to migrate monitoring projects without recreating them in each environment? Alternately, I am considering trying to write my own plan. Is there any documentation out there on how to write the JSON for export plans? Thanks!

If we use matillion ETL tool for  data integration to snowflake. Can we still see the lineage and transformation code in Ataccama?

Hi - I wanted more information on how to use the slices vs a SQL Catalog Item.The use case being often when I’m leveraging a slice I want to be able to do a start date of  >2023 , if it is part of a  quality program I want the data to keep updating i.e as new records come in. However, it seems like the slice is only really designed for the ‘previous’ so ignores current records i.e previous year = 2023, and I can’t set the custom date without an end date so it would be limited to a fixed time range.It seems like for the quality program is better served with the SQL CAT ITEM but I was curious if there was value with the slice I was missing or if I could accomplish the same objective with the slice ? Thank you, Lindsay  

Hi,I want to create a DQ aggregation rule that checks fields from two different tables for the group by clause. Is there a way to do this directly in Ataccama One Web UI.Thanks!

Hi,I am not able to access the transformation plan feature in v15.3 from both the Transformation Plan tab and from the monitoring projects tab. Can someone explain why is there a restriction on creating transformation plans and how can I get access to it?Thanks

I have a One Data table created by importing data from a .csv file.  I need to use some of the data in this table by joining it with an existing Catalog Item by creating a SQL Catalog Item so that I can implement some rules on it.  Where you select the source when starting to create the SQL Catalog Item I do not get an option to select oneData.  Is it even possible to do this in OneWeb?

Debugging - is this possible to i.e. setup some breakpoint and let the application run all the previous steps before selected one?I like using debug function on steps (CTRL+Shift+D). To debug below Step i need to first debug all the previous steps and then i am able to debug the current one.However, if I happen to update logic on some step from the top i then would need to run & debug all the previous steps before i could debug i.e. Multiplicative Regex Matching.Is this possible to automate the manual work here? I would like to create a breakpoint on Multiplicative Regex Matching and each time i could just click a button and it would self-debug from the start to my breakpoint. I looked in your documentation but didn’t found any solution for this. I found some debugging view in ONE Desktop but it doesn’t allow me to add breakpoints.Do i need to configure my debugger somehow to achieve the above? Szymek

Hi,I have a question about the rule logic display when a section is applied. In the report view, I can see the rule logic/implementation when I select a rule. However, once I select a section and do the same, I can only see the valid and invalid results, not the rule logic. Is there a reason behind it? Thank you!