Skip to main content

Solved

681 Topics
aysel_jafarzade
Data Pioneer
aysel_jafarzadeData Pioneer
asked in Data Quality & Catalog ⚙️
Combination of Email Notifications

Hello,I want to set up a single general email notification for various monitoring projects. Typically, I receive separate email notifications for each monitoring project, but I would like to receive only one email notification that consolidates the results of different monitoring projects. I know that I am supposed to use One Desktop but I don’t know which steps exactly. I would be grateful if you could help.Thanks in advance😊

1293
aysel_jafarzade
Data Pioneer
R
Universe Traveller
RiannaUniverse Traveller
asked in Data Quality & Catalog ⚙️
DQ rule and Regular expression - Truncate

Hello Community 😀I am trying to write a DQ rule using a regular expression, I have been using the “ONE Expressions” documentation to help with this but I can’t seem to get it working. Can anyone advise where I am going wrong please??ExampleInput data:Value1 Value2 Value3 -1137.86 -5689.33 0.2  DQ rule I am trying to achieve:Value2 * Value 3 = -1137.866 Take only the first 2 decimals of the above bullet WITHOUT rounding = -1137.86 Check if value1 equals the value in the bullet above. (i.e. -1137.86 = -1137.86 so it should show as a valid result)How I have written this in Ataccama using a regular expression:Value1 is NOT (substituteAll("(\d+\.\d{2})(\d+)","${1}",toString(Value2 * Value3))) Then result is INVALIDError I get:Using the test rule feature with the example input data above, it says that the result is INVALID when I am expecting it to be VALID. And only if I change Value1 to equal -1137.866 then it will show VALID which is not what I want to achieve.Can anyone see wh

723
R
Universe Traveller
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Master Data Management & Reference Data Management 🛰️
RDM Extended Reader row filter syntax

We are on RDM v14.5.1, on PaaS, so underlying db is postgres.I need to query a table using the RDM Extended Reader step with a filter. The volume of this table will grow, so I much prefer to filter at the reader step, rather than bring the whole table into the plan and then filter, to manage performance.I have a column, from_date, that is datetime type. I have from_date in my data flow, and I want to query RDM for a day prior.Expecting that this is an SQL where clause, I tried to put in syntax compatible with Postgres So I created a column called “row_filter”, with type string. Using postgres to_timestamp functionday_prior:=dateAdd(from_date,-1,'DAY');day_prior_string:=tostring(day_prior,'yyyy-MM-dd HH:mm:ss');"from_date=to_timestamp('"+day_prior+"','YYYY-MM-DD HH:MI:SS')" This one gave an error of:13.05.2024 13:10:00 [INFO] Ataccama DQC engine initialized.13.05.2024 13:10:00 [INFO] Creating runtime...13.05.2024 13:10:00 [INFO] Starting runtime...13.05.2024 13:10:00 [INFO]

363
may_kwok
Star Blazer L3
JTH
Data Pioneer
JTHData Pioneer
asked in Data Quality & Catalog ⚙️
Record having max one empty attribute

Hi,This may sound very easy, but: how do I retrieve records that have code = ‘A’ and more than one empty start dates? So in the end I want to see records 3 & 4 since they have the same id_#, code = ‘A’ and more than 1 empty start date. Would be ideal if this was in ONE Web and not through ONE Desktop.Person id_# code start_date 1 1 A 1-1-2000 2 1 A 1-2-2000 3 1 A   4 1 A   5 2 A 1-1-2000 6 2 A 1-2-2000 7 2 A    select *from Personwhere start_date is emptyand code = ‘A’group by id_#, codehaving count (*) > 1I tried to use count in advanced expressions, but can't seem to get it to work.

934
JTH
Data Pioneer
Y
Data Voyager
Yasharth MisraData Voyager
asked in Data Quality & Catalog ⚙️
combining the output of more than 3 or more text file write or component output

I am facing the problem that I have output from 4 components and the I am writing them on different text file writers is there any way I can combine the outputs into a single text file writer if not can I combine the outputs from the different components and write it in a single text file writer.

341
A
Ataccamer
A
Data Voyager
akumaData Voyager
asked in Data Quality & Catalog ⚙️
Mass Delete Catalog Items

Hi Community,We have a scenario where an entire schema was imported as an Ataccama source. This resulted in over 700 tables/views brought in as catalog items.Is there a way to bulk delete all 700?Thank you.

2124
anna.spakova
Ataccamer
Alan Raubenheimer
Data Voyager
Alan RaubenheimerData Voyager
asked in Data Quality & Catalog ⚙️
Data Asset from ETL job

Is it possible to create a data asset from an ETL job (SSIS or similar) so that this is included in the data lineage to allow for full impact analysis 

371
F
Ataccamer
A
Data Voyager
AkuljotData Voyager
asked in Data Quality & Catalog ⚙️
Data stories on synapse source catalog items

Currently only 4 sources are supported for data stories and they are mentioned below: Amazon Aurora PostgreSQL ONE Data PostgreSQL Snowflake We also use synapse as source for a lot of catalog items and it would be great if you can add synapse as well to the above. 

301
A
aysel_jafarzade
Data Pioneer
aysel_jafarzadeData Pioneer
asked in Data Quality & Catalog ⚙️
Deleting Component

Hello,How we can delete one of the components in One Desktop? I created virtual catalog item through One Desktop. Virtual catalog item has already been deleted but its component still remains.  I want to delete its component as well.  For example, I want to delete component “branch_dwh_vs_lake”. How can I delete it from desktop?Thanks in advance, 

402
aysel_jafarzade
Data Pioneer
P
Data Enthusiast
Patty BData Enthusiast
asked in Master Data Management & Reference Data Management 🛰️
Troubleshooting data sets

I have a data set created and the query runs fine in a sql editor against my local database. I can deploy the configuration with no errors, but when I go to look at the data in Ataccama One I get Fatal error occurred and click on Details just gives me Internal error. The only thing I can think of is the query is rather long. Is there a max for data set sql? What is the best way to try to troubleshoot this? The error messages are not at all helpful.

663
ivan.kozlov
Ataccamer
jdordregter
Data Voyager
jdordregterData Voyager
asked in Data Quality & Catalog ⚙️
Term insights (term suggestions)

Hi, I notice that for some of my terms, the visual Insights is visible and for others it is not. What is the reason for this?Second question: I would want to automatically assign terms. if the column is named wnplts then I want to assign the term Residence to it. Is there an easy way to do this? Detection rules seem to look at the value and not at the column name.kind regards, Jur Dördregter

412
jdordregter
Data Voyager
T
Data Pioneer L2
ThomasProData Pioneer L2
asked in Master Data Management & Reference Data Management 🛰️
Data Source "Documentation"

Hey All!I am aware that only Stewardship owners have the ability to use the “Document” button on a data source level, but is there a way that we can disable this button all together? If a user accidentally uses this feature, it eats up a lot of processing to import the metadata, and is a best practice that we are telling everyone not to use. Is there a way within the admin console to do this?Thanks! 

372
T
Data Pioneer L2
jdordregter
Data Voyager
jdordregterData Voyager
asked in Data Quality & Catalog ⚙️
Masking of sensitive data

Hi,I have certrain fields in my data that should be masked for all users. This is information like social security number and IBAN. Now users can see this data from the sample data tab and from the profile. Can I apply some some masking of these attributes without have to create a VCI or SCI for this? Ideally masking should be applied based on the term assignment: as soon as I map the data to IBAN it should be masked for all users.Regards, Jur Dördregter

2045
A
JTH
Data Pioneer
JTHData Pioneer
asked in Data Quality & Catalog ⚙️
Priority DPM

Hi,We have several Monitoring Projects that are scheduled to run at 07:00. If for instance 1000 jobs are still in the queue those scheduled MP will run after the 1000 jobs (assuming the priority is the same; 0). In order to change this I have to go to DPM and change the priority of the scheduled MPs. Is there a way where I can schedule a Monitoring Project at a specific time, 07:00, with a certain priority (automatically)?

632
ivan.kozlov
Ataccamer
Catherine
Data Pioneer
CatherineData Pioneer
asked in Data Quality & Catalog ⚙️
Total Record Volume (all Data Source)

Please advise if there is an easy way to capture how much volume of data in total we have to report up to our CIO? I was thinking creating a plan but that will execute forever.  Any ideas?

482
Catherine
Data Pioneer
P
Data Enthusiast
Patty BData Enthusiast
asked in Master Data Management & Reference Data Management 🛰️
Static values in RDM views

Is there a way to put columns in a view with a static value? Like you can in a sql statement, such asSELECT col1, ‘this is a static value’ AS col2

101
A
Ataccamer
P
Data Enthusiast
Patty BData Enthusiast
asked in Master Data Management & Reference Data Management 🛰️
UNIONS in RDM Views

Is there a way to create views in RDM that contain union statements combining queries to different tables?

161
A
Ataccamer
Marcel-Jan
Star Blazer L2
Marcel-JanStar Blazer L2
asked in Data Quality & Catalog ⚙️
Profiling with row based access

We're currently working on a setup where different customers can see their own data in our Cloudera data lake. So when a customer looks at sample data or profiling data they should only see the row of their own data. I'm currently working on a solution for the profiling data. For this I want a service account for that customer to do the profiling. This service account can also only see the rows for that customer. I was thinking of creating a component for this that reads a database table with lists of customers, url resources and CIs (Hive tables) to profile. And then use a Json call that runs a GraphQL command to run the profiling as the correct service account. Unfortunately the url resource in the Json Call is the only part that cannot be parameterized (as far as I can see). The url resource has the authentication (One Desktop server connection), so that will not be the way to go then. Alternatively I was thinking of using authorization in the HTTP Header. Like this.But some experim

551
A
Ataccamer
C
Universe Traveller
clyormzUniverse Traveller
asked in Master Data Management & Reference Data Management 🛰️
ataccama dpe connection with databricks

Hello Ataccama team I would like to know if there is a reference about how to configure an external dpe using databricks on azure. Thank you

1041
ivan.kozlov
Ataccamer
S
Data Voyager
sujmazumData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Encountering NoRouteToHostException Error on OnPublish

The response message was:null and response error was 'errorMsg:java.net.NoRouteToHostException: No route to host    at java.base/sun.nio.ch.Net.connect0(Native Method)    at java.base/sun.nio.ch.Net.connect(Net.java:579)    at java.base/sun.nio.ch.Net.connect(Net.java:568)    at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:593)    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)    at java.base/java.net.Socket.connect(Socket.java:633)    at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:121)    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)    at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:326)    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:605)    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:440)  

261
A
may_kwok
Star Blazer L3
may_kwokStar Blazer L3
asked in Data Quality & Catalog ⚙️
Is this the best way to unpivot data?

Hello hive mind,I am calling an API that gets me some foreign exchange rate data (all from USD). The data that comes back is in this format:timestamp AUD EUR GBP 2024-04-24 12:00:00 1.538766 0.935636 0.803964  I’ve highly simplified this. There’s a possible169 currencies here, I’m only showing 3. I need to unpivot it and get it into this format:timestamp from_ccy to_ccy rate 2024-04-24 12:00:00 USD AUD 1.538766 2024-04-24 12:00:00 USD EUR 0.935636 2024-04-24 12:00:00 USD GBP 0.803964  My current plan looks like this: It is fine for 3 currencies. But not fine for 169!Anyone got any ideas for steps I can use to keep my plan clean and tidy? (And that I don’t have to make 169 copies of Alter Format 😱) 

272
A
M
Data Voyager
Mr.DQData Voyager
asked in Master Data Management & Reference Data Management 🛰️
Accessing parent node attributes in XML with repeated tags

I have an xml file with a structure like this:<model xmlns="http://www.ataccama.com">    <entities>        <entity name="table1" />     <columns>             <column datatype="varchar(100)" name="abc" required="false" />             <column datatype="int" name="xyz" required="false" />             <column datatype="datetime" name="123" required="false" />     </columns>...I want to produce a data stream that combines the column name and data_type with the name from the parent entity node.  If I define the columns using xpath nm:model/nm:entitles/nm:entity/nm:columns/nm:column then I cannot access the entity name using ../../@name (or any xpath expression for that matter).  If I define the columns using entity level xpath epxression nm:model/nm:entitles/nm:entity and the columns with xpath nm:columns/nm:column/@name then I only get 1 record per entity with the first column only.   How can I get the entity name, column name, and column data

302
M
Data Voyager
JTH
Data Pioneer
JTHData Pioneer
asked in Data Quality & Catalog ⚙️
Change source on Monitoring Project

Hi,I created a Monitoring Project with a Catalog Item Loan from Source A. In the Monitoring Project I apply 100 DQ checks and I'm very happy. I want to change the source afterwards because I'm done checking the CI Loan from Source A and need to check the same CI (not same data) from Source B. Is it possible/doable to change the underlying Source A to Source B keeping the 100 DQ checks in the same Monitoring Project? I heard it was possible, but couldn't find it anywhere in the community.

381
J
Ataccamer
C
Universe Traveller
clyormzUniverse Traveller
asked in Ataccama Data People Summit 2022 - Virtual Attendees
ataccama architecture

Hello I would like to know if there is documentation about the architecture of ataccama. I know ataccama has different services integrated (dqf,cooment,audit,dmm,dqp,task,workflow..). In order to know how services are integrated and communicate each other , does ataccama has a document?

381
A
Albert de Ruiter
Rocket Pioneer L1
Albert de RuiterRocket Pioneer L1
asked in Data Quality & Catalog ⚙️
Use case for Security terms

Hi Community!I was wondering if anyone has used the Security term (as extension of term) functionality in OneWeb and if so how. There seems no documentation available from the Documentation portal or the Community to clarify the modeling choices. The node has the following properties regulation is a string instead of a reference to node regulation and responsiblePerson is also a string. The effectiveDate is not of type date but string as well. Why would this be?Anyhow, I would appreciate if anyone can shine some light on this.Thanks and kind regards!Albert

662
Albert de Ruiter
Rocket Pioneer L1

Badge winners

  • First idea 💡
    VanNguyenhas earned the badge First idea 💡
  • First idea 💡
    SusanShas earned the badge First idea 💡
  • 10 ideas 💡
    akumahas earned the badge 10 ideas 💡
  • First idea 💡
    romizhas earned the badge First idea 💡
  • First idea 💡
    kellymremzilybhas earned the badge First idea 💡
Show all badges

Let's shape the future of Ataccama together

Join User Research at Ataccama!
See new features and prototypes before everyone else, and have an opportunity to shape the future of new products.

Read more
Community Terms of Use & ConditionsCookie settings