Discuss data, share best practices, learn, and grow with Ataccama's Community.
Recently active
Hello,I want to set up a single general email notification for various monitoring projects. Typically, I receive separate email notifications for each monitoring project, but I would like to receive only one email notification that consolidates the results of different monitoring projects. I know that I am supposed to use One Desktop but I don’t know which steps exactly. I would be grateful if you could help.Thanks in advance😊
Hello Community 😀I am trying to write a DQ rule using a regular expression, I have been using the “ONE Expressions” documentation to help with this but I can’t seem to get it working. Can anyone advise where I am going wrong please??ExampleInput data:Value1 Value2 Value3 -1137.86 -5689.33 0.2 DQ rule I am trying to achieve:Value2 * Value 3 = -1137.866 Take only the first 2 decimals of the above bullet WITHOUT rounding = -1137.86 Check if value1 equals the value in the bullet above. (i.e. -1137.86 = -1137.86 so it should show as a valid result)How I have written this in Ataccama using a regular expression:Value1 is NOT (substituteAll("(\d+\.\d{2})(\d+)","${1}",toString(Value2 * Value3))) Then result is INVALIDError I get:Using the test rule feature with the example input data above, it says that the result is INVALID when I am expecting it to be VALID. And only if I change Value1 to equal -1137.866 then it will show VALID which is not what I want to achieve.Can anyone see wh
Hi,This may sound very easy, but: how do I retrieve records that have code = ‘A’ and more than one empty start dates? So in the end I want to see records 3 & 4 since they have the same id_#, code = ‘A’ and more than 1 empty start date. Would be ideal if this was in ONE Web and not through ONE Desktop.Person id_# code start_date 1 1 A 1-1-2000 2 1 A 1-2-2000 3 1 A 4 1 A 5 2 A 1-1-2000 6 2 A 1-2-2000 7 2 A select *from Personwhere start_date is emptyand code = ‘A’group by id_#, codehaving count (*) > 1I tried to use count in advanced expressions, but can't seem to get it to work.
I am facing the problem that I have output from 4 components and the I am writing them on different text file writers is there any way I can combine the outputs into a single text file writer if not can I combine the outputs from the different components and write it in a single text file writer.
Hi Community,We have a scenario where an entire schema was imported as an Ataccama source. This resulted in over 700 tables/views brought in as catalog items.Is there a way to bulk delete all 700?Thank you.
Is it possible to create a data asset from an ETL job (SSIS or similar) so that this is included in the data lineage to allow for full impact analysis
Currently only 4 sources are supported for data stories and they are mentioned below: Amazon Aurora PostgreSQL ONE Data PostgreSQL Snowflake We also use synapse as source for a lot of catalog items and it would be great if you can add synapse as well to the above.
Hello,How we can delete one of the components in One Desktop? I created virtual catalog item through One Desktop. Virtual catalog item has already been deleted but its component still remains. I want to delete its component as well. For example, I want to delete component “branch_dwh_vs_lake”. How can I delete it from desktop?Thanks in advance,
I have a data set created and the query runs fine in a sql editor against my local database. I can deploy the configuration with no errors, but when I go to look at the data in Ataccama One I get Fatal error occurred and click on Details just gives me Internal error. The only thing I can think of is the query is rather long. Is there a max for data set sql? What is the best way to try to troubleshoot this? The error messages are not at all helpful.
Hi, I notice that for some of my terms, the visual Insights is visible and for others it is not. What is the reason for this?Second question: I would want to automatically assign terms. if the column is named wnplts then I want to assign the term Residence to it. Is there an easy way to do this? Detection rules seem to look at the value and not at the column name.kind regards, Jur Dördregter
Hey All!I am aware that only Stewardship owners have the ability to use the “Document” button on a data source level, but is there a way that we can disable this button all together? If a user accidentally uses this feature, it eats up a lot of processing to import the metadata, and is a best practice that we are telling everyone not to use. Is there a way within the admin console to do this?Thanks!
Hi,I have certrain fields in my data that should be masked for all users. This is information like social security number and IBAN. Now users can see this data from the sample data tab and from the profile. Can I apply some some masking of these attributes without have to create a VCI or SCI for this? Ideally masking should be applied based on the term assignment: as soon as I map the data to IBAN it should be masked for all users.Regards, Jur Dördregter
Hi,We have several Monitoring Projects that are scheduled to run at 07:00. If for instance 1000 jobs are still in the queue those scheduled MP will run after the 1000 jobs (assuming the priority is the same; 0). In order to change this I have to go to DPM and change the priority of the scheduled MPs. Is there a way where I can schedule a Monitoring Project at a specific time, 07:00, with a certain priority (automatically)?
how do i do an exact field search?For Ex: I have a field named trd_settle_date - and when i search for that in the data catalog, it is Giving me all fields in catalog that either started with “trd” or had “settle” word some where in it etc.,Because of that, the search is giving so many unnecesary values in the list, and making it difficult to find the right one.How can i get Ataccama to perform exact match to my search string? If it doesnt find - its OK for it to give me “no items found”
Please advise if there is an easy way to capture how much volume of data in total we have to report up to our CIO? I was thinking creating a plan but that will execute forever. Any ideas?
Is there a way to put columns in a view with a static value? Like you can in a sql statement, such asSELECT col1, ‘this is a static value’ AS col2
Is there a way to create views in RDM that contain union statements combining queries to different tables?
We're currently working on a setup where different customers can see their own data in our Cloudera data lake. So when a customer looks at sample data or profiling data they should only see the row of their own data. I'm currently working on a solution for the profiling data. For this I want a service account for that customer to do the profiling. This service account can also only see the rows for that customer. I was thinking of creating a component for this that reads a database table with lists of customers, url resources and CIs (Hive tables) to profile. And then use a Json call that runs a GraphQL command to run the profiling as the correct service account. Unfortunately the url resource in the Json Call is the only part that cannot be parameterized (as far as I can see). The url resource has the authentication (One Desktop server connection), so that will not be the way to go then. Alternatively I was thinking of using authorization in the HTTP Header. Like this.But some experim
Hi everyone, Today we will cover how to use the search function in ONE Web Application 🔍Whether you're a business user looking for specific information or a technical user executing complex queries, our search engine has you covered. Let's dive in!There are two areas for search:Global Search: full-text search is enabled on the whole application and is a good starting point for searching entities and locating the area for the future detailed search if necessary. Search in Navigation Sections: combines full-text and filtering capabilities to query entities. Search options are restricted to the currently selected navigation section: for example, using the search bar in Catalog Items will only return results found within that entity and configured areas of search on this entity.1️⃣ Global Search: Discovering Catalogued EntitiesThe Global Search is your go-to tool for searching use cases across cataloged entities. It allows users to explore preconfigured entities like catalog items, terms,
Hello Ataccama team I would like to know if there is a reference about how to configure an external dpe using databricks on azure. Thank you
Hi community 👋 We are so excited to announce our next event, our first Ask Me Anything session on Data quality and governanceWe are bringing our best experts to gather together and discuss the best practices, use cases, challenges, and workarounds on all things DQG.You'll have the opportunity to ask our Product Managers, Professional Services Consultants, and Community DQG experts anything related to data quality and governance in Ataccama. Whether you're looking for advice on overcoming specific challenges, seeking insights into improving your processes, wanting to broaden your knowledge, or simply finding inspiration from other members, this AMA is the perfect opportunity to connect with like-minded professionals and gain valuable insights.👇Find more about the event and how to register here 👇 Already have your questions lined up? Great! Share with us what you’d like to learn more about, your current challenges in DQG, and anything you’d like us to cover in the comments below. ⚠️A
Hello:I would like to use GraphQL queries to extract results of a monitoring project - and load the results into my own database table for reporting and other purposes. I need to make it split the invalid records - in a way - that each record should be for one failed rule - even if the primary key record - has multiple DQ issues (failed rules).Can some one help me with sample code on how you may have accomplished it - if you did such a thing?
The response message was:null and response error was 'errorMsg:java.net.NoRouteToHostException: No route to host at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:579) at java.base/sun.nio.ch.Net.connect(Net.java:568) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:605) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)
Hello hive mind,I am calling an API that gets me some foreign exchange rate data (all from USD). The data that comes back is in this format:timestamp AUD EUR GBP 2024-04-24 12:00:00 1.538766 0.935636 0.803964 I’ve highly simplified this. There’s a possible169 currencies here, I’m only showing 3. I need to unpivot it and get it into this format:timestamp from_ccy to_ccy rate 2024-04-24 12:00:00 USD AUD 1.538766 2024-04-24 12:00:00 USD EUR 0.935636 2024-04-24 12:00:00 USD GBP 0.803964 My current plan looks like this: It is fine for 3 currencies. But not fine for 169!Anyone got any ideas for steps I can use to keep my plan clean and tidy? (And that I don’t have to make 169 copies of Alter Format 😱)
Hi everyone!Data quality management often requires comparing data from various sources and performing cross-entity checks to ensure accuracy and consistency. In this post, I’ll walk you through the process of creating cross-entity checks. Let’s get started 👇🏻 An Overview of the General StepsTo perform cross-entity checks, you need to follow these five key steps: Create a Virtual Catalog Item (VCI) in IDE: The Virtual Catalog Item acts as a container to hold the tables you want to compare. Join the Necessary Tables: Identify and join the relevant tables from your data sources that can be merged for comparison. Sync the VCI with the Web Application: Ensure that the Virtual Catalog Item you created is synchronized with the web application for further processing. Create a Rule to Compare Data: Develop a rule that specifies the comparison logic for data across different tables. Create a Monitoring Project: Set up a monitoring project to compare table values at a specific periodi
Already have an account? Login
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.