Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert bag metadata datetime fields to date fields #506

Closed
helrond opened this issue Sep 20, 2021 · 5 comments · Fixed by #638
Closed

Convert bag metadata datetime fields to date fields #506

helrond opened this issue Sep 20, 2021 · 5 comments · Fixed by #638

Comments

@helrond
Copy link
Member

helrond commented Sep 20, 2021

Is your feature request related to a problem? Please describe.

Bag metadata date fields use Django datetime fields. This is confusing for end users.

Describe the solution you'd like

Convert datetime fields to date fields.

Additional context

Will need to investigate how this impacts existing data.

@McDaPick
Copy link
Contributor

McDaPick commented Jan 17, 2022

Converted the DateTime fields to Date fields in the BagInfoMetadata model.

After a refresh and taking a look at the metadata section in the transfers detail view, it seems bagging_date was pushed forward five hours while date_start and date_end were pushed four.

Before field change:

DateTimeStart

After field change:

DateField

Going to grab a burrito 🌯 but will be looking through bag_transfers to see what could be causing this afterwards.

@helrond
Copy link
Member Author

helrond commented Jan 18, 2022

@McDaPick Hey, thanks for working on this! I'm guessing there is some timezone/daylight savings time stuff going on. I would suggest the following:

  • Look at the underlying values for these Transfers in the Django shell to confirm they are correct. What you see in the browser is getting filtered through templates, which also need to be updated.
  • Some work may need to be done in the TransferRoutine (which is where these values get saved) to save dates rather than datetimes. I think this might be where you'd find what's going on with bagging_date.

@McDaPick
Copy link
Contributor

You would not believe how long that burrito line was!

Jumped back into this and believe I found the issue above.

After updating the date_start and date_end fields in the BagInfoMetadata model from

date_start = models.DateTimeField()
date_end = models.DateTimeField()

to

date_start = models.DateField()
date_end = models.DateField()

and running a migration, the date was being saved as one day earlier than what was in the bag-info file.

Transfer's Bag Info:

External Identifier: records-2017-12-11T20:09:48.238148
BagIt-Profile-Identifier: http://localhost:8000/api/bagit_profiles/2/
Bagging-Date: 2017-12-11T20:09:48.238148
Date-End: 2005-06-22
Date-Start: 2003-05-14

The Transfer's data through the Django shell after being imported:

>>> from bag_transfer.models import BagInfoMetadata
>>> x = BagInfoMetadata.objects.get(external_identifier="records-2017-12-11T20:09:48.238148")
>>> print(x.date_end)
2005-06-21
>>> print(x.date_start)
2003-05-13

Inside of the Transfer's detail view:

image

After reading @helrond comment about timezone/daylight, I started to take a look around and saw that config.py has the time zone set as America/New_York on line 15.

DJANGO_TIME_ZONE = "America/New_York"  # TZ database name for local timezone (string)

Updating this value to UTC allowed for the date_field to save correctly.

DJANGO_TIME_ZONE = "UTC"  # TZ database name for local timezone (string)

Transfer's Bag Info:

External Identifier: records-2017-12-11T20:09:48.238148
BagIt-Profile-Identifier: http://localhost:8000/api/bagit_profiles/2/
Bagging-Date: 2017-12-11T20:09:48.238148
Date-End: 2005-06-22
Date-Start: 2003-05-14

The Transfer's data through the Django shell after DJANGO_TIME_ZONE was updated:

>>> from bag_transfer.models import BagInfoMetadata
>>> x = BagInfoMetadata.objects.get(external_identifier="records-2017-12-11T20:09:48.238148")
>>> print(x.date_end)
2005-06-22
>>> print(x.date_start)
2003-05-14

Inside of the Transfer's detail view:

image

Side Note:

This update to the DJANGO_TIME_ZONE also seems to fix the bagging_date time being fives hours off of its value in bag-data.

I pushed up a branch and submitted a PR with the changes.

Pull Request #634

@helrond
Copy link
Member Author

helrond commented Nov 1, 2023

This all makes sense @McDaPick! My only question is if that burrito was worth the wait?

@McDaPick
Copy link
Contributor

McDaPick commented Nov 1, 2023

I'd wait until the end of my days for a large double queso! 🌯🌯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants