Skip to main content

Hi!

I am profiling tables and am looking at the profiles. I noticed that for date and datetime the profile does not show the MIN or MAX values. That is something I had expected though. Working on the data quality of this data, I really would like to know the MIN and MAX values for these attributes. Is there a way to change the configuration and include these profile statistics for these datatypes?

 

kind regards, Jur Dördregter

Hi ​@jdordregter !

Thank you for your question! 

As you are correctly pointing out, numerical statistics are calculated only for numeric attributes.
However, you can have a look at the Quantiles - They display the values that occur at designated intervals in the ordered data set, with the first value on the list shown at 0% and the last value at 100%.
If your date type attributes have standardized format, like YYYY-MM-DD, it will work for you, as well. 

Hope this will help you!

Kind regards, 
Ekaterina

Hi ​@ekaterina.ponomareva,

I tried your solution, however it does not show the quantiles in my profiling, is this due to our version or the source used?

kind regards, Jur Dördregter

Hi ​@jdordregter ,

I can see that your data source is Hadoop, so it is indeed because of that.

Kind regards,
Ekaterina

Hi ​@jdordregter I’m closing this thread for now. If you have any follow up questions please feel free to share the in the comments or create a new post 🙋🏻‍♀️

​@ekaterina.ponomareva Good for you to know: in the backend this information is stored. Should not be rocket science to show this information also in web application. 

 

Hi ​@jdordregter, ONE Desktop and Ataccama ONE does have some differences and thank you for taking the time to share your feedback. I’ve passed this on to our team. 

Community Terms of Use & ConditionsCookie settings