Skip to content

Metadata/data validation should produce a warning for an unused datetime_format entry #2930

Description

@npatki

Problem Description

For datetime columns, the metadata specification allows me to input a datetime_format string that specifies exactly how my datetimes are to be formatted. For example, '21 Jul 2026' vs '21-07-2026'.

If the data is represented as a dtype object (containing string values), then SDV can explicitly validate that the provided format matches the data.

However if for some reason the data is represented as a datetime64, then there is no format really associated with those values. So in this case, SDV does not explicitly validate whether the data matches any explicitly provided format string. But as a user, I don't know that this validation is skipped.

Expected behavior

The metadata.validate_data functions should raise a warning if:

  • The metadata has a specified datetime_format string for a particular column AND
  • The data is represented as a dtype object (or any other dtype where the format cannot be checked)

The warning should specifically indicate that the datetime format cannot be verified and will be skipped.

Warning: The datetime format for column 'checkin_date' (table 'guests') could not be verified because the data is represented as dtype 'datetime64[ns]'.
Please omit the datetime format string from the metadata or cast the data to strings with the right format.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions