Discuss, learn, ask, suggest, share all things DQ&C π
Recently active
Hi everyone, me again π I work as a professional services consultant in Ataccama and I prepared another article for you, this time about some recommendations for integration with git, mainly for self-managed deployments. Specifically, when you have the DQG suite and you want to integrate your orchestration server. You can probably reuse some of these recommendations for other modules as well.Before you start reading, please note that I reused some of the already existing articles:https://community.ataccama.com/master-data-management-reference-data-management-92/one-desktop-git-recommendations-187?tid=187&fid=92I also recommend checking out this article in our documentation for working with git in our IDE:https://docs.ataccama.com/one-desktop/latest/projects/git.html#introduction-to-git-and-egitAlso please note, that this article doesnβt cover how to work with git in general, but more on how to work with the Ataccama projects, what files to version, what to look out for etc.Recomme
I have certain Business terms that i want to track on which fields are getting mapped from which datasets on a given day.Is there a way, i can send a notification to myself (and any others interested) on the occurance details of a term - and i schedule this to run periodically like every monday?Β To be specific, Iam want to track Social Security Number. I want to get a report every monday morning - with list of Catalog items and attributes that are mapped to this term. (from the occurances details).Β Is it possible?? How??Β Β
Hello! I am looking for assistance with applying filters to rules. Currently, I understand that the "where" clause can be used in rules, but it seems to apply to all records uniformly.Β What I need is a way to create a rule that only affects a subset of the data, so that the total count of records affected by the rule is different from the total count of records in the Catalog Item. For example, if I have a dataset with 1000 records, I want to create a rule that only applies to, say, 200 of those records based on specific criteria, without affecting the other 800 records. So if there is 100 records failing the result should be 100/200 instead of 100/1000.Is there a way to achieve this selective application of rules? Any guidance or examples on how to set this up would be greatly appreciated.Β Thank you!
is there a way to change the date format in ataccama one. i want the format to beΒ dd mm yyyy, what i have currently is mm dd yyyy
How can I run a .vbs file workflow. May be either βRun Shell Scriptβ or βRun Windows Commandβ. Or is there any other options available?Β I tied to run this through βRun Shell Scriptβ but it showing an issue with Interpreter(/bin/sh)Β
Hello can you please assist in finding out how to findΒ the duplicates in a column using one desktop.
For terms like USA Zipcode, USA Social Security Number etc., - the term is enabled with AI detection and it is getting tagged to hundreds of fields - most of them are NOT zipcode or SSN.Β Β If it was giving a score and recommendation, I would have discarded the recommendation, but it is assigning the term automatically and I have to go in and remove that - which became painful. I have now disabled the AI detection but is there a better way to tune it to make it work better?Β
hello everyone , I am unable to understand how to use representative creator step can you please explain it in detail with an example so that I am able to use it .Β Thank you.
HI Team,I am using one metadata writer for one of my use-case. In one-metadata writer there is an out option called recovery audit.Β Β May I know the use of this functionality. I tried checking in help contents I was not able to find any document related to this. If IΒ donβt want to use this recovery_audit is it possible?Β Β This is the error I am getting while running the plan. Let me know if more information is needed from my end.Β Thanks.
How can I run a .vbs file workflow. May be either βRun Shell Scriptβ or βRun Windows Commandβ. Or is there any other options available?Β I tied to run this through βRun Shell Scriptβ but it showing an issue with Interpreter(/bin/sh)Β Β
Hi everyone!I wanted to share some best practices when it comes to GIT version control, how you can organize your branches in a way that merges and rollbacks are more granular, making it easier as well to see what code changes were done whereΒ Typically, what I tell clients when they are using GIT integrated with Ataccama, is that they would essentially need to have a standard flow control process for their branches. What I mean by that, is that they should set up 1 main production branch, 1 development branch, and many feature branches. You can think about it like this, your Master branch can be thought of as your βProductionβ branch, your Development branch can be thought of as your βTestβ server, and each of those feature branchesΒ you can see it as your βDevβ server where you develop new code. How this works is simple yet effective:You start off with a main branch in your repo, we can call it βmasterβ Master can have either prior code or be empty, let's say it has some code in it alr
I have been trying for some time to build a data quality check/rule in Ataccama Desktop (using an advanced expression) to check where a defiend attribute is unique/distinct and have been struggling to find the correct function to use for this.In a tool like excel I woudl simply count the number of instances of each value and following that I would ensure that anything where the count was >1 failed this check but I have thus far been unable to find a way to perform a distinct count in this way. e.g.Β DistinctCount([Install_And_Leave_Flag]) = 1I understand that the profiling step nativley gives you this information but I require a way to do that via an advanced expression for my use case.Β Β
Hello everyone, I have a requirement of exporting ONE webapp's audit logs but not using the MinIO storage, is there any other way to export logΒ files through ONE Desktop IDE using any Api calls such as graphql converted into curl command in json call? Thanks!
Hello,I want to set up a single general email notification for various monitoring projects. Typically, I receive separate email notifications for each monitoring project, but I would like to receive only one emailΒ notification that consolidates the results of different monitoring projects. I know that I am supposed to use One Desktop but I donβt know which steps exactly. I would be grateful if you could help.Thanks in advanceπ
Hello CommunityΒ πI am trying to write a DQ rule using a regular expression, I have been using the βONE Expressionsβ documentation to help with this but I canβt seem to get it working. Can anyone advise where I am going wrong please??ExampleInput data:Value1 Value2 Value3 -1137.86 -5689.33 0.2 Β DQ rule I am trying to achieve:Value2 * Value 3 = -1137.866 Take only the first 2 decimals of the above bullet WITHOUT rounding = -1137.86 Check if value1 equals the value in the bullet above. (i.e.Β -1137.86 =Β -1137.86 so it should show as a valid result)How I have written this in Ataccama using a regular expression:Value1 is NOTΒ (substituteAll("(\d+\.\d{2})(\d+)","${1}",toString(Value2 * Value3))) Then result is INVALIDError I get:Using the test rule feature with the example input data above, it says that the result is INVALID when I am expecting it to be VALID. And only if I change Value1 to equalΒ -1137.866 then it will show VALID which is not what I want to achieve.Can anyone see wh
Hi,This may sound very easy, but: how do I retrieve records that have code = βAβ and more than one empty start dates? So in the end I want to see records 3 & 4 since they have the same id_#, code = βAβ and more than 1 empty start date. Would be ideal if this was in ONE WebΒ and not through ONE Desktop.Person id_# code start_date 1 1 A 1-1-2000 2 1 A 1-2-2000 3 1 A Β 4 1 A Β 5 2 A 1-1-2000 6 2 A 1-2-2000 7 2 A Β Β select *from Personwhere start_date is emptyand code = βAβgroup by id_#, codehaving count (*) > 1I tried to use count in advanced expressions, but can't seem to get it to work.
I am facing the problem that I have output from 4 components and the I am writing them on different text file writers is there any way I can combine the outputs into a single text file writer if not can I combine the outputs from the different components and write it in a single text file writer.
Hi Community,We have a scenario where an entire schema was imported as an Ataccama source. This resulted in over 700 tables/views brought in as catalog items.Is there a way to bulk delete all 700?Thank you.
Is it possible to create a data asset from an ETL job (SSIS or similar) so that this is included in the data lineage to allow for full impact analysisΒ
Currently only 4 sources are supported for data stories and they are mentioned below: Amazon Aurora PostgreSQL ONE Data PostgreSQL Snowflake We also use synapse as source for a lot of catalog items and it would be great if you can add synapse as well to the above.Β
Hello,How we can delete one of the components in One Desktop? I created virtual catalogΒ item through One Desktop. Virtual catalog item has already been deleted but its component still remains.Β Β I want to delete its component as well. Β For example, I want to delete component βbranch_dwh_vs_lakeβ. How can I delete it from desktop?Thanks in advance,Β
Hi,Β I notice that for some of my terms, the visual Insights is visible and for others it is not. What is the reason for this?Second question: I would want to automatically assign terms. if the column is named wnplts then I want to assign the term Residence to it. Is there an easy way to do this? Detection rules seem to look at the value and not at the column name.kind regards, Jur DΓΆrdregter
Hi,I have certrain fields in my data that should be masked for all users. This is information like social security number and IBAN. Now users can see this data from the sample data tab and from the profile. Can I apply some some masking of these attributes without have to create a VCI or SCI for this? Ideally masking should be applied based on the term assignment: as soon as I map the data to IBAN it should be masked for all users.Regards, Jur DΓΆrdregter
Hi,We have several Monitoring Projects that are scheduled to run at 07:00. If for instance 1000 jobs are still in the queue those scheduled MP will run after the 1000 jobs (assuming the priority is the same; 0). In order to change this I have to go to DPM and change the priority of the scheduled MPs. Is there a way where I can schedule a Monitoring Project at a specific time, 07:00, with a certain priority (automatically)?
how do i do an exact field search?For Ex: I have a field named trd_settle_dateΒ - and when i search for that in the data catalog, it is Giving me all fields in catalog that either started with βtrdβ or had βsettleβ word some where in it etc.,Because of that, the search is giving so many unnecesary values in the list, and making it difficult to find the right one.How can i get Ataccama to perform exact match to my search string? If it doesnt find - its OK for it to give me βno items foundβ
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.