Skip to content
This repository has been archived by the owner on Feb 2, 2022. It is now read-only.

Extract deceased field to its own function #136

Merged
merged 4 commits into from Jun 8, 2019

Conversation

anthonybaulo
Copy link
Member

@anthonybaulo anthonybaulo commented Jun 6, 2019

Types of changes

  • Code cleanup / Refactoring

Description

Extract deceased field to its own function with tests.
Rename previous parse_deceased_field() to process_deceased_field()

Checklist:

  • I have updated the documentation accordingly
  • I have written unit tests

Fixes: #106

@rgreinho rgreinho self-requested a review June 7, 2019 02:38
@anthonybaulo
Copy link
Member Author

@rgreinho
There's a lot going on with parsing the deceased field (L344-470 in apd.py), including a function named parse_deceased_field().
Is this issue solely concerned with extracting a parsing function for use inside of parse_page_content() on L504, which would literally just be moving the existing regex from the searches list into a new function?

searches = [
        (Fields.DECEASED, re.compile(r'>Deceased:\s*(?:</span>)?(?:</strong>)?\s*>?([^<]*\d)\s*.*\)?<')),
        (Fields.LOCATION, re.compile(r'>Location:.*>\s{2,}(?:</strong>)?([^<]+)')),
    ]

If so, what would you like to call the new function, since parse_deceased_field() is being used for something else?

@rgreinho
Copy link
Member

rgreinho commented Jun 7, 2019

Oh you're right! so let's do the following:

  • Rename the existing parse_deceased_field() function to process_deceased_field(). Be sure to catch all the references to this functions. VS Code will be your friend :)
  • Name the new function you will create parse_deceased_field().

And we should be good to go!

The function parse_deceased_field() was created to pull content from the
corresponding field on the fatality page. There was an existing function
of the same name, which has now been renamed process_deceased_field().
Test cases have been made for the new parse_deceased_field().

Fixes scrapd#106
@anthonybaulo anthonybaulo marked this pull request as ready for review June 7, 2019 23:16
Copy link
Member

@rgreinho rgreinho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good job overall! Just one minor comment to check whether or not we succeeded to parse the field before adding it do the results.

scrapd/core/apd.py Outdated Show resolved Hide resolved
The deceased field will not be created in the dictionary
unless a match was found.

Fixes scrapd#106
@rgreinho
Copy link
Member

rgreinho commented Jun 8, 2019

And merged! Great work @anthonybaulo!

@mergify mergify bot merged commit 2fa140e into scrapd:master Jun 8, 2019
@anthonybaulo anthonybaulo deleted the issues/106/deceased-field branch June 9, 2019 16:30
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extract deceased information parsing into its own function
2 participants