Skip to content

Commit

Permalink
Apply suggestions from code review
Browse files Browse the repository at this point in the history
Co-authored-by: James McKinney <26463+jpmckinney@users.noreply.github.com>
  • Loading branch information
yolile and jpmckinney committed Jul 13, 2023
1 parent 119ff84 commit 324fe91
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 6 deletions.
2 changes: 1 addition & 1 deletion kingfisher_scrapy/extensions/database_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class DatabaseStore:
If the ``DATABASE_URL`` Scrapy setting and the ``crawl_time`` spider argument are set, the OCDS data is stored in a
PostgreSQL database, incrementally.
This extension stores data in the "data" column of a table named after the spider or ``table_name``.
This extension stores data in the "data" column of a table named after the spider, or ``table_name`` (if set).
When the spider is opened, if the table doesn't exist, it is created. The spider's ``from_date`` attribute is then
set, in order of precedence, to: the ``from_date`` spider argument (unless equal to the spider's
``default_from_date`` class attribute); the maximum value of the ``date`` field of the stored data (if any); the
Expand Down
8 changes: 3 additions & 5 deletions tests/extensions/test_database_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,11 +164,9 @@ def test_spider_closed(cursor, caplog, tmpdir, data, data_type, sample, compile_
spider.data_type = data_type
spider.sample = sample
spider.compile_releases = compile_releases
spider.table_name = table_name

if table_name:
table = spider.table_name = table_name
else:
table = spider.name
expected_table = table_name if table_name else spider.name

extension = DatabaseStore.from_crawler(spider.crawler)

Expand Down Expand Up @@ -209,7 +207,7 @@ def test_spider_closed(cursor, caplog, tmpdir, data, data_type, sample, compile_
expected_messages = [
f'Reading the {tmpdir}/test{suffix}/20210525_000000 crawl directory with the {prefix} prefix',
f'Writing the JSON data to the {tmpdir}/test{suffix}/20210525_000000/data.csv CSV file',
f'Replacing the JSON data in the {table} table',
f'Replacing the JSON data in the {expected_table} table',
]
if compile_releases:
expected_messages.insert(1, 'Creating generator of compiled releases')
Expand Down

0 comments on commit 324fe91

Please sign in to comment.