Learn & Share

Happy Wednesday everyone! We are halfway through the week and wrapping up our profiling best practices with this post (but honestly are you ever done with profiling?). So let’s dive in to learn how to read a DQ profile, column analyses, and advanced analyses. Data quality profiles are crucial tools for understanding and improving the quality of your data. Once you've generated a profile, it's important to know how to read and interpret the information it provides. 1. Accessing the Profile ViewerAfter running a profiling plan or creating a one-time profile, a file with a .profile extension will appear in your File Explorer. Double-clicking this file opens the Profile Viewer. 2. Inputs and Roll UpsThe Profiling Step may have multiple inputs, each of which may have associated "roll ups" that group the data by specific parameters. Understanding these inputs and roll ups is vital to gaining a comprehensive view of your data. For detailed information on adding inputs and configuring roll ups

Hi community! In this post, we will cover a fundamental practice of configuring the profiling step 👣If you have missed it you can check the first post on the introduction to data profiling here👇🏽 The Profiling StepWhen you create a profile using a plan, the Profiling step becomes a pivotal part of your data journey, connecting your data source to insights. Let's explore how to make the most of it.General CategoryBasic TabIn the Basic tab, you can set the step's name, output file name, location, and the default locale for generated files. Masks TabThe Masks tab is where you define and edit masks. Masks help reveal data structure without displaying the actual content. For instance, you can use "D" to represent a digit and "L" for a letter. Configure characters, symbols, repeated symbols, and thresholds for your masks here. Right-click on the row to add, delete, or edit the characters: Drill-through TabThe Drill-through tab enables drill-through functionality, allowing you to inspect i

Happy Monday everyone! This week we are delving deep into one of the most important steps in working with data - profiling 🚀 We will start with the basics and move on to more advanced practices so don’t forget to follow the forum by hitting the Subscribe button to get the latest article in your inbox 🔔 When embarking on a data project, understanding your data's current state is paramount. It's the compass that guides you through the winding path of data cleansing and transformation. This understanding also serves as a yardstick to measure the effectiveness of your cleansing rules. Enter the Data Quality Profile – a snapshot of your data's health. It unveils duplicates, dependencies, business rule evaluations, data patterns, and much more. In this post, we will cover how to create and configure a data quality profile, complete with additional analyses like masks, dependencies, and business rules. Step 1: Select Data to Profile Launch ONE Desktop. In the File Explorer view, pick one

Hi everyone! This week’s last best practice is also the final post on components! Check out yesterday’s post to catch on if you have missed the series so far:  Nested components are like puzzle pieces that fit together to create sophisticated data transformations within a single entity. In this guide, we'll explore how to work with nested components in Ataccama ONE Desktop, focusing on their creation, import, configuration, and use cases. Understanding Nested ComponentsImagine having a main (parent) component that uses multiple component files to achieve complex data transformations. Nested components enable you to do just that. This approach allows you to maintain the original data while applying intricate logic to your datasets. For instance, you can use predefined components provided by Ataccama ONE Desktop to cleanse, filter, or standardize specific data attributes, all while preserving the integrity of your original data. Configuring a Nested Component in ONE DesktopIn Ataccama ON

Happy Monday community! Today we are going to talk about Remote Plans. Let’s start with the basics, what are remote plan executions? The remote plan execution lets you run any ONE plan on the Ataccama ONE Platform. Plans are typically developed and tested on small data sets or on dedicated environments. This helps you validate the plans and assess how well they perform so that you can make the necessary adjustments before applying them on your actual data. Once the plan works as expected, you can run it remotely on production data in ONE.Jobs are first sent to the Data Processing Manager (DPM) module, which forwards the job information to a suitable Data Processing Engine (DPE). Actual data processing takes place only in DPE instances. How to configure Remote Plan ExecutionRemote plan execution leverages existing controls and concepts of ONE Desktop. Once you connect to the Ataccama ONE Platform, you can choose between the following launch options:Local: A suitable DPE is selected auto

Happy Wednesday Community! We are halfway into the week and ready with Part 2 of Component Steps. If you have missed Part 1 don’t worry, check it out on the link below 🔽 In the second part of our journey into Component Steps, we will delve deeper into their creation, configuration, parameter mapping, running from the command line, and working with nested components. Creating a ComponentBuilding a component step is akin to constructing a plan, but certain rules dictate its placement within a plan: At the Beginning: A component at the start of a plan should commence with an input step connected to actual data, not Integration Input. In the Middle: For components in the middle of a plan, they must have at least one input endpoint with Integration Input and one output endpoint with Integration Output. At the End: Components concluding a plan must start with an input endpoint with Integration Input and conclude with an output step that isn't Integration Output (ensuring data is written

Hi community, happy Tuesday! Today, we are covering one of the essential functions of ONE Desktop - Expressions & Steps ✨Steps and functions enable you to craft plan files tailored to your project's unique needs. Here, we provide a comprehensive overview of the frequently used steps and functions, guiding you towards best practices.Frequently Used Steps:Flow Control StepsData Parsing StepsAnalysis StepsMatch and Merge StepsUsing FunctionsWithin these steps, you can leverage various functions to further enhance your data processing capabilities. Here are some commonly used functions:matches: Performs a full match of input data with a regular expression, returning true or false. find: Conducts a partial match of a regular expression within the input string, returning true or false. substr: Extracts a substring from the input string, starting from zero.Using Regular ExpressionsAtaccama's products support regular expressions for powerful pattern matching. Here are some fundamental regu