Move from optparse to argparse

[BurnzZ]

The optparse module has been deprecated since 3.2 in lieu of argparse.

At the time of this writing, we only have 3 modules that use it directly:

• https://github.com/scrapy/scrapy/blob/master/scrapy/cmdline.py
• https://github.com/scrapy/scrapy/blob/master/scrapy/commands/__init__.py
• https://github.com/scrapy/scrapy/blob/master/tests/test_commands.py

I'm filing this ticket to keep track of this task. In addition there were several attempts on this in the past that hasn't pushed through:

• [WIP] Upgrading optparse code (deprecated since Py2.7) to argparse #1118
• [WIP] optparse to argparse #829

[kawishqayyum]

Can I work on this, please?

[LusiferAjay]

Hi @Gallaecio I am new in the open source world
can I work on this? if this is not available let me know if anything else is there from which I can start my Journey

[wRAR]

@kawishqayyum @LusiferAjay you don't need a permission to work on any issue

[Jerry-Mah]

Hello, I would like to work on this issue. How should I go about solving this issue?

[alexpdev]

I submitted #5374 which addresses this Issue.

[bryangators]

Hello, I plan to work on this issue. I see a pr is pending but I would like to take a shot at it as well. I am participating in this for a course. Thanks.

[bryangators]

How do I build the scrapy repo locally with the source code. I tried installing locally with 'pip install -e scrapy/' but it does not run. I am able to get it to run when downloading from pip. I guess my question is how do I get a local build where I can modify and view changes from my local repo?

[wRAR]

pip install -e scrapy in a venv should work.

[bryangators]

I must of had a bad setup in virtualenv. I switched to conda and it works now. Thanks!

[bryangators]

I created a pull request #5382 to solve this. It is very similar to #5374 above but this commit solves the issue in scrapy/commands/view.py where "--header" option is to be removed. Still need an admin to approve workflows for codecov to run coverage tests. This is my first commit to open source and look forward to hearing any feedback. Thanks.