Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Converting create_instance to 2 separate functions #6169

Merged
merged 33 commits into from
Dec 18, 2023

Conversation

sa2415
Copy link
Contributor

@sa2415 sa2415 commented Dec 5, 2023

Converted create_instance to 2 separate functions (build_from_crawler and build_from_settings).
Changed other calls to create_instance in other files.
Added tests for the new functions.

Fixes #5523, closes #5884, closes #6162.

sa2415 and others added 4 commits November 30, 2023 20:26
Attempted build_from_crawler
Separates the create_instance into 2 separate functions 
1) build_from_settings
2)build_from_crawler
Commented out function duplicates
Edited import statement and changed create_instance call to build_from_settings
@sa2415 sa2415 marked this pull request as draft December 5, 2023 01:46
monicaq21 and others added 12 commits December 5, 2023 10:59
Testing 2 new functions 
build_from_settings 
build_from_crawler
edited import statement and changed create_instance calls to build_from_settings
edit import statements and change create_instance calls
Create_instance changed to build_from_crawler
edit create_instance function
@sa2415 sa2415 marked this pull request as ready for review December 5, 2023 02:32
@sa2415 sa2415 marked this pull request as draft December 5, 2023 02:32
sa2415 and others added 3 commits December 4, 2023 21:37
create_instance changed to build_from_crawler
edit import sentence and change create_instance call
@sa2415 sa2415 marked this pull request as ready for review December 5, 2023 02:49
@wRAR
Copy link
Member

wRAR commented Dec 5, 2023

You should use pre-commit and you should run the tests locally before submitting.

@wRAR wRAR marked this pull request as draft December 5, 2023 09:21
@wRAR
Copy link
Member

wRAR commented Dec 6, 2023

Have you run the tests locally?

@sa2415
Copy link
Contributor Author

sa2415 commented Dec 6, 2023

Attempted to run the tests locally but are running into issues. Will attempt to debug and will make another commit once resolved.

monicaq21 and others added 2 commits December 11, 2023 02:35
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
@sa2415 sa2415 requested a review from wRAR December 10, 2023 17:39
scrapy/core/downloader/contextfactory.py Show resolved Hide resolved
scrapy/utils/misc.py Outdated Show resolved Hide resolved
scrapy/utils/misc.py Outdated Show resolved Hide resolved
scrapy/utils/misc.py Outdated Show resolved Hide resolved
scrapy/utils/misc.py Outdated Show resolved Hide resolved
@sa2415 sa2415 requested a review from wRAR December 10, 2023 18:55
@wRAR wRAR closed this Dec 10, 2023
@wRAR wRAR reopened this Dec 10, 2023
scrapy/utils/misc.py Outdated Show resolved Hide resolved
Co-authored-by: Andrey Rakhmatullin <wrar@wrar.name>
@sa2415 sa2415 requested a review from wRAR December 10, 2023 21:56
@sa2415
Copy link
Contributor Author

sa2415 commented Dec 11, 2023

Hi @wRAR - thank you for the detailed feedback - is our commit merge ready or would you like us to make more changes?

@monicaq21
Copy link

Hi! Sorry to bother you again, this PR is actually part of our final assignment that has significant credit, and having it merged would help us immensely. Could you check over the changes again and, if possible, merge it? Thank you so much @wRAR

Comment on lines 106 to 109
if hasattr(m, "from_crawler"):
m.from_crawler.assert_called_once_with(crawler, *args, **kwargs)
self.assertEqual(m.call_count, 0)
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this test only checks from_crawler while test_build_from_settings checks from_crawler and from_settings, I think this is an error, this one should check both while test_build_from_settings should only check from_settings.

@@ -111,47 +142,23 @@ def _test_with_settings(mock, settings):
else:
mock.assert_called_once_with(*args, **kwargs)

def _test_with_crawler(mock, settings, crawler):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this logic should be kept in _test_with_crawler.

# 1. with no alternative constructors
# 2. with from_settings() constructor
# 3. with from_crawler() constructor
# 4. with from_settings() and from_crawler() constructor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Including the fourth scenario.

@@ -95,14 +96,44 @@ class TestItem(Item):
list(arg_to_iter(TestItem(name="john"))), [TestItem(name="john")]
)

def test_create_instance(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think the create_instance test should be kept as long as we keep the function itself.

Copy link
Member

@Gallaecio Gallaecio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also log a deprecation warning when create_instance is called.

Copy link
Member

@wRAR wRAR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@sa2415
Copy link
Contributor Author

sa2415 commented Dec 16, 2023

Hi @wRAR - apologies for another ping - Was wondering if our PR can be merged with the base branch now if it's ready? Sorry for the slight rush, our deadline is in a day and we would receive credit if it is successfully merged...however, if there are still changes to be made, we will work accordingly to get those done as well. Thank you for your understanding!

@wRAR
Copy link
Member

wRAR commented Dec 17, 2023

@sa2415 our policy is to merge contributor PRs after two approvals from maintainers so we can merge it once @Gallaecio approves it.

@Gallaecio Gallaecio merged commit c67f730 into scrapy:master Dec 18, 2023
26 checks passed
@Gallaecio
Copy link
Member

Awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Simplify create_instance calls
6 participants