TypeError: 'NoneType' object is not iterable on some pages

[tvfoodmaps]

Hi, I've been using extruct pretty successfully but came across a URL that seems to validate OK but when I run it through I get an error:

  File "/Users/frankapap/KCApp/extruct/RecipeInfoService.py", line 24, in recipeExtract
    data = extruct.extract(r.text, base_url=base_url,syntaxes=['json-ld', 'opengraph'],uniform=True)
  File "/usr/local/lib/python3.7/site-packages/extruct/_extruct.py", line 67, in extract
    output[label] = list(extract(document, base_url=base_url))
  File "/usr/local/lib/python3.7/site-packages/extruct/jsonld.py", line 25, in extract_items
    self._xp_jsonld(document))
  File "/usr/local/lib/python3.7/site-packages/extruct/jsonld.py", line 26, in <listcomp>
    for item in items
TypeError: 'NoneType' object is not iterable

The code is:
data = extruct.extract(r.text, base_url=base_url,syntaxes=['json-ld', 'opengraph'],uniform=True)
The URL being passed in is https://www.tasteofhome.com/collection/keto-diet-recipes/view-all/

[Kebniss]

Thank you for reporting this bug! The issue seems to be that JsonLdExtractor._extract_items returns None and JsonLdExtractor.extract_items tries to iterate on it. I will create a PR to fix it

[tvfoodmaps]

Oh that would be great! Thanks.

[tvfoodmaps]

When the fix is merged, will I be able to get it by just doing a pip3 update?

[whalebot-helmsman]

@tvfoodmaps you don't need to wait until merge, you can install fixed version right now using

pip install git+git://github.com/scrapinghub/extruct@fix-jsonld-return-none-bug

[Kebniss]

@tvfoodmaps the fix was merged into master. It will be included in next release but in the meantime you can use @whalebot-helmsman's solution. Can I close the issue?

[tvfoodmaps]

yes. thanks.