Skip to main content
Solved

Confusing date data

  • February 4, 2025
  • 7 replies
  • 75 views

The Business wants this date attribute validated.  I donโ€™t know why, itโ€™s a key field and I assume managed by the DB.  Nonetheless, I need to build a rule.

But I think itโ€™s strange that the Data Type is DATE while the date values are YYYY-MM-DD. I would expect to see DATE format data in the MM-DD-YYY HH:MM:SS AM format.

I donโ€™t know how to code it in the date functions since it doesnโ€™t have the standard date format and it doesnโ€™t work in any string functions.

Any ideas? 

 

Best answer by sam.dahl

Hi Tim,

It sounds like this is already a proper DATE type, so thereโ€™s no need for time or string functions. The way itโ€™s displayed (YYYY-MM-DD vs. MM-DD-YYYY HH:MM:SS AM) is just a formatting choice, not usually how it's actually stored. In most DBMS, DATE fields are strictly typed and inherently validate as proper Gregorian dates so thereโ€™s no โ€œformatโ€ in storage, just a native date representation.

If the goal is validation using Evaluation rules, then the usual things to check would be:
a) Missing values - are there nulls where there shouldnโ€™t be?
b) Expected ranges - do dates fall within a logical timeframe (e.g., no orders from 1800)?
c) Common default or disallowed values - things like 1900-01-01, 9999-12-31, or placeholders that might indicate bad data.
d) Relativity & dependencies - does this date make sense in relation to others? (e.g., a ship date shouldnโ€™t be before an order date.)
e) Uniqueness - probably not relevant unless this is actually a key field (which seems unlikely-maybe itโ€™s just important rather than a true key).

If these dates were loaded from another source and incorrectly interpreted during ingestion, there could be a mix of issues-not just valid-looking but incorrect dates, but also records that were dropped entirely if parsing failed. That would likely have been framed as a bigger problem, though, so itโ€™s probably not the case here.

One other possible issue is truncation or timezone shifts. If the original source included a time component and it was stripped to just a DATE, you could have unintended offsets-some systems default to midnight UTC when stripping time, which might cause subtle shifts depending on how time zones were handled. Itโ€™s an edge case, but worth considering if unexpected discrepancies show up.

Otherwise, if the field is already a strict DATE type in the DB, this is more of a standard data integrity check rather than a formatting issue.

Does that help?

View original

  • Data Voyager
  • February 4, 2025

Hi Tim,

It sounds like this is already a proper DATE type, so thereโ€™s no need for time or string functions. The way itโ€™s displayed (YYYY-MM-DD vs. MM-DD-YYYY HH:MM:SS AM) is just a formatting choice, not usually how it's actually stored. In most DBMS, DATE fields are strictly typed and inherently validate as proper Gregorian dates so thereโ€™s no โ€œformatโ€ in storage, just a native date representation.

If the goal is validation using Evaluation rules, then the usual things to check would be:
a) Missing values - are there nulls where there shouldnโ€™t be?
b) Expected ranges - do dates fall within a logical timeframe (e.g., no orders from 1800)?
c) Common default or disallowed values - things like 1900-01-01, 9999-12-31, or placeholders that might indicate bad data.
d) Relativity & dependencies - does this date make sense in relation to others? (e.g., a ship date shouldnโ€™t be before an order date.)
e) Uniqueness - probably not relevant unless this is actually a key field (which seems unlikely-maybe itโ€™s just important rather than a true key).

If these dates were loaded from another source and incorrectly interpreted during ingestion, there could be a mix of issues-not just valid-looking but incorrect dates, but also records that were dropped entirely if parsing failed. That would likely have been framed as a bigger problem, though, so itโ€™s probably not the case here.

One other possible issue is truncation or timezone shifts. If the original source included a time component and it was stripped to just a DATE, you could have unintended offsets-some systems default to midnight UTC when stripping time, which might cause subtle shifts depending on how time zones were handled. Itโ€™s an edge case, but worth considering if unexpected discrepancies show up.

Otherwise, if the field is already a strict DATE type in the DB, this is more of a standard data integrity check rather than a formatting issue.

Does that help?


Albert de Ruiter
Star Blazer L3
Forum|alt.badge.img+4

Hi Tim,

In addition to what Sam already explained, there is also a distinction between a date and a datetime (or timestamp) datatype. In this case apparently the date is displayed without the time details, but it can also be displayed as date with 00:00:00 as time details.

If the business is only requesting to make sure that the values are only date values, so Tim's examples a-e don't apply, and indeed the date datatype applies for this field in the database, you can simply tell the business that their requirement is already met. 

Kind regards,

Albert


  • Universe Traveller
  • February 6, 2025

Thank you, your replies are in line with my experience.


Cansu
Community Manager
Forum|alt.badge.img+3
  • Community Manager
  • February 13, 2025

Thank you โ€‹@sam.dahl and โ€‹@Albert de Ruiter for sharing your solutions here!


  • Universe Traveller
  • February 18, 2025

Thank Sam and Albert. 

My experience aligns with your responses.

I have passed your comments onto my mgr, who has the task (if he chooses) to convince the business users. 


  • Universe Traveller
  • March 13, 2025

Another analyst developed this code to validate a datetime datatype.  It seems valid to me:

In the Variables section:

 

In the logic section:

Not (VARIABLE_1 is not null)

 

 


Cansu
Community Manager
Forum|alt.badge.img+3
  • Community Manager
  • March 20, 2025

Thank you for sharing โ€‹@TimBrown74 โœ”๏ธ๐Ÿ™Œ


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings