Remove deprecated DictItem/BaseItem classes

[elacuesta]

More deprecation removals:

• DictItem deprecated on 1.8
• BaseItem deprecated on 2.2

[codecov]

Codecov Report

Merging #5398 (fca49cc) into master (3533717) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #5398      +/-   ##
==========================================
- Coverage   88.58%   88.54%   -0.04%     
==========================================
  Files         163      163              
  Lines       10646    10627      -19     
  Branches     1816     1809       -7     
==========================================
- Hits         9431     9410      -21     
- Misses        940      941       +1     
- Partials      275      276       +1     

Impacted Files	Coverage Δ
scrapy/exporters.py	100.00% <100.00%> (ø)
scrapy/item.py	98.36% <100.00%> (-0.39%)	⬇️
scrapy/utils/misc.py	97.72% <100.00%> (ø)
scrapy/core/downloader/__init__.py	90.97% <0.00%> (-1.51%)	⬇️

[elacuesta]

For the record, I successfully ran the tests for scrapy-jsonschema (which imports scrapy.item.ItemMeta) after this merge (f7bf741):

$ tox -e py38                    
GLOB sdist-make: /Users/eus/zyte/scrapy-jsonschema/setup.py
py38 create: /Users/eus/zyte/scrapy-jsonschema/.tox/py38
py38 installdeps: -rrequirements.txt, -rdev-requirements.txt, coverage, nose
py38 inst: /Users/eus/zyte/scrapy-jsonschema/.tox/.tmp/package/1/scrapy-jsonschema-0.5.0.zip
py38 installed: arrow==1.2.2,attrs==21.4.0,Automat==20.2.0,cffi==1.15.0,constantly==15.1.0,coverage==6.3.1,cryptography==36.0.1,cssselect==1.1.0,fqdn==1.5.1,hyperlink==21.0.0,idna==3.3,importlib-resources==5.4.0,incremental==21.3.0,iniconfig==1.1.1,isoduration==20.11.0,itemadapter==0.4.0,itemloaders==1.0.4,jmespath==0.10.0,jsonpointer==2.2,jsonschema==4.4.0,lxml==4.7.1,nose==1.3.7,packaging==21.3,parsel==1.6.0,pluggy==1.0.0,Protego==0.1.16,py==1.11.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pycparser==2.21,PyDispatcher==2.0.5,pyOpenSSL==22.0.0,pyparsing==3.0.7,pyrsistent==0.18.1,pytest==7.0.0,python-dateutil==2.8.2,queuelib==1.6.2,rfc3339-validator==0.1.4,rfc3987==1.3.8,

---> Scrapy @ git+https://github.com/scrapy/scrapy@f7bf7414f0f51a80e4055912e93de7b853d6c2a1,

scrapy-jsonschema @ file:///Users/eus/zyte/scrapy-jsonschema/.tox/.tmp/package/1/scrapy-jsonschema-0.5.0.zip,service-identity==21.1.0,six==1.16.0,tomli==2.0.0,Twisted==21.7.0,typing_extensions==4.0.1,uri-template==1.1.0,w3lib==1.22.0,webcolors==1.11.1,zipp==3.7.0,zope.interface==5.4.0
py38 run-test-pre: PYTHONHASHSEED='3431114446'
py38 run-test: commands[0] | nosetests --with-doctest --with-coverage --cover-package=scrapy_jsonschema tests
........................
Name                            Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------------------
scrapy_jsonschema/__init__.py       3      0      0      0   100%
scrapy_jsonschema/draft.py          4      0      0      0   100%
scrapy_jsonschema/item.py          62      0     39      1    99%
scrapy_jsonschema/pipeline.py      30      1     12      0    98%
-----------------------------------------------------------------
TOTAL                              99      1     51      1    99%
----------------------------------------------------------------------
Ran 24 tests in 0.042s

OK
___________________________________________________________ summary ___________________________________________________________
  py38: commands succeeded
  congratulations :)

Also itemadapter, which attempts to import scrapy.item._BaseItem/scrapy.item.BaseItem:

$ tox -e py38
GLOB sdist-make: /Users/eus/zyte/itemadapter/setup.py
py38 create: /Users/eus/zyte/itemadapter/.tox/py38
py38 installdeps: -rtests/requirements.txt
py38 inst: /Users/eus/zyte/itemadapter/.tox/.tmp/package/1/itemadapter-0.4.0.zip
py38 installed: attrs==21.4.0,Automat==20.2.0,cffi==1.15.0,constantly==15.1.0,coverage==6.3.1,cryptography==36.0.1,cssselect==1.1.0,hyperlink==21.0.0,idna==3.3,incremental==21.3.0,iniconfig==1.1.1,itemadapter @ file:///Users/eus/zyte/itemadapter/.tox/.tmp/package/1/itemadapter-0.4.0.zip,itemloaders==1.0.4,jmespath==0.10.0,lxml==4.7.1,packaging==21.3,parsel==1.6.0,pluggy==1.0.0,Protego==0.1.16,py==1.11.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pycparser==2.21,pydantic==1.9.0,PyDispatcher==2.0.5,pyOpenSSL==22.0.0,pyparsing==3.0.7,pytest==7.0.0,pytest-cov==3.0.0,queuelib==1.6.2,

---> Scrapy @ git+https://github.com/scrapy/scrapy@f7bf7414f0f51a80e4055912e93de7b853d6c2a1,

service-identity==21.1.0,six==1.16.0,tomli==2.0.0,Twisted==21.7.0,typing_extensions==4.0.1,w3lib==1.22.0,zope.interface==5.4.0
py38 run-test-pre: PYTHONHASHSEED='3965422145'
py38 run-test: commands[0] | pytest --verbose --cov=itemadapter --cov-report=term-missing --cov-report=html --cov-report=xml --doctest-glob=README.md itemadapter README.md tests
================================================================ test session starts ================================================================
platform darwin -- Python 3.8.6, pytest-7.0.0, pluggy-1.0.0 -- /Users/eus/zyte/itemadapter/.tox/py38/bin/python
cachedir: .tox/py38/.pytest_cache
rootdir: /Users/eus/zyte/itemadapter, configfile: pytest.ini
plugins: cov-3.0.0
collected 115 items                                                                                                                                 

README.md::README.md PASSED                                                                                                                   [  0%]
tests/test_adapter.py::ItemAdapterReprTestCase::test_repr_attrs PASSED                                                                        [  1%]
tests/test_adapter.py::ItemAdapterReprTestCase::test_repr_attrs_init_false PASSED                                                             [  2%]
(...)
tests/test_utils.py::ItemLikeTestCase::test_true_scrapy PASSED                                                                                [100%]

---------- coverage: platform darwin, python 3.8.6-final-0 -----------
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
itemadapter/__init__.py       3      0   100%
itemadapter/adapter.py      198      0   100%
itemadapter/utils.py         65      1    98%   19
-------------------------------------------------------
TOTAL                       266      1    99%
Coverage HTML written to dir htmlcov
Coverage XML written to file coverage.xml


========================================================== 113 passed, 2 skipped in 5.07s ===========================================================
______________________________________________________________________ summary ______________________________________________________________________
  py38: commands succeeded
  congratulations :)

[pawelmhm]

I have code that relies on BaseItem, how much time I have to update it? Which Scrapy version will it be in?

[wRAR]

This will only be included in 2.6, which we are currently preparing (with no certain release date but an almost done changelog).

[pawelmhm]

thanks for info. I'm not importing basitem directly but I see exception coming from https://pypi.org/project/itemadapter/

something like this

my_project/spiders/spider_file.py:161: in parse_product
    item = loader.load_item()

../../../.pyenv/versions/x/lib/python3.9/site-packages/itemloaders/__init__.py:265: in load_item
    adapter[field_name] = value
../../../.pyenv/versions/x/lib/python3.9/site-packages/itemadapter/adapter.py:304: in __setitem__
    self.adapter.__setitem__(field_name, value)
../../../.pyenv/versions/x/lib/python3.9/site-packages/itemadapter/adapter.py:214: in __setitem__
    self.item[field_name] = value
my_project/items.py:115: in __setitem__
    super(Item, self).__setitem__(key, value)

checking why it happens. I reached this commit by doing git bisect.

[pawelmhm]

I think this is just my codebase doing unusual things overriding __setitem__ method, so most likely something I can and should change in my code, not scrapy issue.