Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Field names containing "_items" crash field coverage #428

Closed
rvandam opened this issue Dec 20, 2023 · 0 comments · Fixed by #437
Closed

Field names containing "_items" crash field coverage #428

rvandam opened this issue Dec 20, 2023 · 0 comments · Fixed by #437
Assignees

Comments

@rvandam
Copy link

rvandam commented Dec 20, 2023

  File "/Users/rvandam/.venv/ADD1D703-D26A-4DD9-BEE9-DA619F3B2F68/lib/python3.11/site-packages/scrapy/utils/defer.py", line 348, in maybeDeferred_coro
    result = f(*args, **kw)
  File "/Users/rvandam/.venv/ADD1D703-D26A-4DD9-BEE9-DA619F3B2F68/lib/python3.11/site-packages/pydispatch/robustapply.py", line 55, in robustApply
    return receiver(*arguments, **named)
  File "/Users/rvandam/.venv/ADD1D703-D26A-4DD9-BEE9-DA619F3B2F68/lib/python3.11/site-packages/spidermon/contrib/scrapy/extensions.py", line 128, in spider_closed
    self._add_field_coverage_to_stats()
  File "/Users/rvandam/.venv/ADD1D703-D26A-4DD9-BEE9-DA619F3B2F68/lib/python3.11/site-packages/spidermon/contrib/scrapy/extensions.py", line 182, in _add_field_coverage_to_stats
    coverage_stats = calculate_field_coverage(stats)
  File "/Users/rvandam/.venv/ADD1D703-D26A-4DD9-BEE9-DA619F3B2F68/lib/python3.11/site-packages/spidermon/utils/field_coverage.py", line 29, in calculate_field_coverage
    item_field_coverage = value / root_field_type_total
TypeError: unsupported operand type(s) for /: 'int' and 'NoneType'

Problem seems to be in line 19 of utils/field_coverage.py -> calculate_field_coverage which makes the assumption that if the field name contains "_items" it must be an internal coverage stat.

Perhaps that line should be if "/_items/" in item_key: ??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants