Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure date string format is consistent #57

merged 15 commits into from Mar 8, 2019


None yet
2 participants
Copy link

commented Mar 6, 2019

Types of changes

  • Bug fix ( addresses issue #56 )


Added function in to convert any parse-able date string into the desired format dd-mm-YYYY. Added call to that function as part of sanitize_fatality_entity() in

These changes ensure that the dates returned will all be formatted the same, improving readability.


  • I have updated the documentation accordingly
  • I have written unit tests


Fixes #56

@rgreinho rgreinho self-requested a review Mar 6, 2019

mrengler added some commits Mar 6, 2019

Added tests for date reformat; older tests will still fail (assert 'D…
…ecember 7, 2017' == '07/12/2017' is False)
@@ -166,7 +166,7 @@ def parse_twitter_description(twitter_description):
# Parse the Deceased field.
if d.get(Fields.DECEASED):

This comment has been minimized.

Copy link

rgreinho Mar 7, 2019


Nice catch ;)

:rtype: str
dt = parse_date(date)
return datetime.datetime.strftime(dt, "%d/%m/%Y")

This comment has been minimized.

Copy link

rgreinho Mar 7, 2019


I apologize I made a mistake, the English format for the date is MM/DD/YYYY, so if you can just reverse these two first fields that would be great.

That would also make your test pass.


This comment has been minimized.

Copy link

commented Mar 7, 2019

Good job on your first PR @mrengler! It looks very good.

I apologize again for the mistake I made in the issue regarding the date format. Luckily it is an easy one to fix.

Aside of that, just a few minor remarks:

  • As you normalized the date format, you will also have to update all the other tests which involve a date field.
  • You will need to update the commit message of your first commit:
    • Just add a small paragraph describing the what you just did (and "why" if applicable).
    • The last last line of the message should be Fixes scrapd/scrapd#56. This is a line used by GitHub to automatically close the issue. Here are 2 good examples:
  • Edit the CHANGELOG with a one line desciption of what you did. If you are not familiar with the KeepAChangelog format, you can read more about it here. It should look like this:
## [Unreleased]

### Fixed

- Fix inconsistent date format. All the new date fields now follow the English format `MM/DD/YYYY`. [#57]

## [[1.4.2]] - 2019-03-03

And at the end of the CHANGELOG, add a link to the issue:


@rgreinho rgreinho added the bug label Mar 7, 2019

mrengler added some commits Mar 6, 2019

This commit adds a function in date_utils to convert a date string of…
… any format into the standard English format mm-dd-YYYY and adds a call to this function during data sanitization. This improves legibility of the data as all dates no longer appear in various formats.

Fixes #56
Updated CHANGELOG, changed date format to English standard mm/dd/YYYY…
… and updated tests to reflect date format change.
Should check only DOB for future date parsed; this way bad input for …
…FROM and TO date ranges will have expected behavior.
Copy link

left a comment

Great job! Thanks a lot for your contribution and all the efforts keep the code base clean 👍

@rgreinho rgreinho merged commit c34d3a0 into scrapd:master Mar 8, 2019

6 checks passed

ci/circleci: docs Your tests passed on CircleCI!
ci/circleci: format Your tests passed on CircleCI!
ci/circleci: lint Your tests passed on CircleCI!
ci/circleci: prepare Your tests passed on CircleCI!
ci/circleci: test Your tests passed on CircleCI!
coverage/coveralls First build on date_format_fix at 96.813%
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.