Learn & Share

If we had the following Instance Data containing the source system and master id groups for each of the instance records and we would like to calculate the count of source per master id. Step by step guide. Create our input data as a text file. Create a new plan and add the following steps to the plan; 1x Text File Reader 1x Representative Creator 2x Text File Writer Connect the steps as follows Configure the Text File Reader to read from the input data we created in step 1. and add 2 shadow columns. These shadow columns could alternatively be added via an Alter Format step. The columns will hold our count per source system (A | B)   Double click on the Representative Creator step and click on the Group By tab. select the master_id as the column to group by.     Add a Rule to the Representative Creator step by clicking on the Add button or double clicking on the asterisk.  Double click on the newly created Attribute rule and then on the Representative Attributes tab

The standard deviation measures the spread of the data about the mean value. It is useful in comparing sets of data which may have the same mean but a different range.For example, the mean of the following two is the same:18 19 20 21 22 = 18+19+20+21+22 / 5 = 2010 15 20 25 30 = 10+15+20+25+30 / 5 = 20However, the second is clearly more spread out. If a set has a low standard deviation, the values are not spread out too much. Lets do the calculations outside of ONE Desktop, so we know our expected result.ExampleFind the standard deviation of 4, 9, 11, 12, 17, 5, 8, 12, 14?First work out the mean: 4+9+11+12+17+5+8+12+14 / 9 = 10.222Now, subtract the mean individually from each of the numbers given and square the result. This is equivalent to the (x - )² step. x refers to the values given in the question.x 4 9 11 12 17 5 8 12 14 (x -  )2 38.7 1.49 0.60 3.16 45.9 27.3 4.94 3.16 14.3 Now add up these results (this is the 'sigma' in the formula): 38.7+1.

Did you know that you can setup email notifications when certain events occur in your DQ Monitoring Projects? There are 4 types of notifications that can be assigned to events; Notification + Email sends both an in-app notification and an email to the recipients in case of the event occurrence. Email sends an email to the recipients in case of the event occurrence. Notification sends an in-app notification to the recipients in case of the event occurrence. Leave the events notification options blank if you do not want to be notified of a particular event occurrence. Process Steps Configure the MMM Module for generating links in emails.Ensure the MMM Module has been configured for SMTP (There is a link to the documentation at the end of this post)Underneath your existing configuration for plugin.email.ataccama.one.addressAdd a new configuration line for plugin.email.ataccama.one.base.url which is the base URL for generating links in emails. The default value is http://localhost:80

In this Tips & Tricks post, I will show you how to configure a plan to evaluate your data and make a decision on whether to continue processing your data as planned or to stop processing data and exit the plan safely without the plan throwing an error.In this scenario we will generate a data set consisting of 1 column and 100 records. The column will have either a true or false value. We’ll have a business rule that checks the values of the 100 records and stops processing the data if at least one false value is found in the data stream.  Let’s prepare the data pipe initial so that we have input and output data flows.Starting with a blank .plan file we will add a Text File Reader step to store our results and a Random Record Generator step to generate the 100 records.We’ll configure the Random Record Generate step to create 1 column named value, and to create the values true 99 in 100 frequency and false 1 in 100 frequency. This configuration will ensure that when we run the plan 1