Skip to main content

I’m hoping anyone has some good GraphQL examples to share. For us, it is necessary to integrate Ataccama into our standard reporting. To do so I’m trying to get a GraphQL working that returns the scores for each item in a monitoring project. Ideally, listing out all monitoring projects and items with their scores - for the latest processing (although I’ll take all at this point and filter the old ones later). This is all about the statistics (# records passed/failed etc. not the individual results - I am aware I can create post-processing plans to export data, but this is not applicable here).

It’s basically the overview from the report tab in the monitoring project, at item level:

I can get everything I need, including the total row count except the passed or failed figure. I have tried and looked at all GraphQL examples everywhere, including favicon-16x16.pngONE API :: Ataccama ONE, and have pieced this together based on the structures visible in the desktop app. Things like DqAggregationResultOverTime don’t really cut it, because it already aggregates across dq dimensions, and I’m interested in the per-item information.

How can I report on passed/failed for each item in a monitoring project, ideally without explicitly listing the catalog item (because we have hundreds)?

The query below is close, all it’s missing is the passed and/or failed count.

query listMonitoringProject {
    monitoringProject(gid: "<gid>") {
        gid
        publishedVersion {
            name
            processings {
                edges {
                    node {
                        gid
                        publishedVersion {
                            __typename
                            startedAt
                            state
                            result
                            dqResult {
                                publishedVersion {
                                    __typename
                                    startedCount
                                    statistics {
                                        publishedVersion {
                                            successCount
                                        }
                                    }
                                }
                            }
                            items {
                                edges {
                                    node {
                                        publishedVersion {
                                            __typename
                                            catalogItem {
                                                publishedVersion {
                                                    name
                                                }
                                            }
                                            result
                                            dqResult {
                                                __typename
                                                gid
                                                publishedVersion {
                                                    result
                                                    dqResult {
                                                        storedVersion {
                                                            __typename
                                                            ruleCount
                                                            recordCount

                                                        }
                                                    }                  
                                                }
                                            }
                                        }
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

 

Hi all,

In case others attempt something similar, this is not directly possible and requires multiple data sets to be retrieved and combined.

Ataccama support sent a helpful component for the desktop application, and that works. I’ve also refactored this into GraphQL calls, and it’s possible to piece this overview together by combining the following:

Get the passed/failed indicator for each dimension pathway.

query listDataQualityDimensions {
dqDimensionResults(
    versionSelector: { publishedVersion: true }
  ) {
    edges {
      node {
        gid
        publishedVersion {
          _displayName
          passedOverallResult
        }
      }
    }
  }
}

Get the monitoring projects and catalog items following the configuration details. 

query listMonitoringProject 
            {
            monitoringProjects (versionSelector: { draftVersion: false }) {
                edges {
                    node {    
                        gid
                        publishedVersion {
                            __typename
                            name  
                            configuration {                
                                __typename
                                publishedVersion {
                                    items {
                                        edges {
                                            node {
                                                gid
                                                publishedVersion {                                    
                                                    displayName
                                                    dqChecks {
                                                        edges {
                                                            node {
                                                                __typename
                                                                gid
                                                                publishedVersion
                                                                {
                                                                    __typename
                                                                    displayName
                                                                    mappings {
                                                                        edges {
                                                                            node {
                                                                                __typename
                                                                                gid
                                                                                publishedVersion
                                                                                {
                                                                                    catalogItemAttribute 
                                                                                    {
                                                                                        publishedVersion 
                                                                                        {
                                                                                            name
                                                                                        }
                                                                                    }
                                                                                }                                                     
                                                                            }
                                                                        }
                                                                    }
                                                                }
                                                            }
                                                        }
                                                    }
                                                }
                                            }
                                        }
                                    }
                                }
                            }          
                        }
                    }
                }
            }
        }

Use these details to collect the results per project / item / check using the dqCheckResultsOverTime query:

query CheckResults {
          monitoringProject(gid: "<monitoring_project_gid>") {
            publishedVersion {
              dqCheckResultOverTime(
                selector: { limit: 1 }
                catalogItemId: "<item_gid>"
                checkIds: I"<dq_check_gid>"]
              ) {
                processingId
                startedAt
                results {
                  dqCheckId
                  results {
                    id
                    count
                    name
                  }
                }
              }
            }
          }
        }

Combining these three sets can re-create this overview.

The desktop app component uses the same building blocks.

Hi ​@RoelantVos, thank you for coming back and sharing the solution here - really helpful for anyone looking to implement a similar reporting 🙌

Reply


Community Terms of Use & ConditionsCookie settings