Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add crawl_time and keep_collection_open spider parameters #462

Merged
merged 4 commits into from Jul 24, 2020
Merged

Add crawl_time and keep_collection_open spider parameters #462

merged 4 commits into from Jul 24, 2020

Conversation

yolile
Copy link
Member

@yolile yolile commented Jul 23, 2020

No description provided.

@yolile
Add custom_collection_data_version spider parameter
226865f 
Signed-off-by: Yohanna Lisnichuk <yohanitalisnichuk@gmail.com>
@yolile
Remove prints
705e305 
Signed-off-by: Yohanna Lisnichuk <yohanitalisnichuk@gmail.com>
@yolile yolile requested a review from jpmckinney July 23, 2020 15:20
jpmckinney
jpmckinney reviewed Jul 23, 2020
View reviewed changes
kingfisher_scrapy/extensions.py Outdated Show resolved Hide resolved
kingfisher_scrapy/extensions.py Outdated Show resolved Hide resolved
kingfisher_scrapy/base_spider.py Outdated Show resolved Hide resolved
@yolile
Rename parameter, add keep_collection_open, update tets and get_start…
bdc3753 
…_date method

Signed-off-by: Yohanna Lisnichuk <yohanitalisnichuk@gmail.com>
@yolile yolile requested a review from jpmckinney July 23, 2020 18:23
@yolile
Update extension to use crawl_time
ebb34da 
Signed-off-by: Yohanna Lisnichuk <yohanitalisnichuk@gmail.com>
jpmckinney
jpmckinney approved these changes Jul 23, 2020
View reviewed changes
Copy link
Member

@jpmckinney jpmckinney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very clean!

@yolile yolile changed the title Add custom_collection_data_version spider parameter Add crawl_time and keep_collection_open spider parameters Jul 24, 2020
@yolile yolile merged commit 3b74a82 into master Jul 24, 2020
@yolile yolile deleted the 441-same-collection-feature branch July 24, 2020 12:23
@yolile yolile mentioned this pull request Jul 24, 2020
Closed
@jpmckinney jpmckinney mentioned this pull request Sep 8, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jpmckinney jpmckinney jpmckinney approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@yolile @jpmckinney