core/source: use import error

[purarue]

uses the more broad ImportError
instead of ModuleNotFoundError

reasoning being if some submodule
(the one I'm configuring currently is
my.twitter.twint) doesn't have additional
imports from another parser/DAL, but it
still has a config block, the user would
have to create a stub-config block in their
config to use the all.py file

I had originally made this ModuleNotFoundError
since reddit had pushshift_comment_export,
so the assumption was that there was an additional
module that could make this fail when additional
sources are used, but that ignores this case when
its just a single file module, not calling out
to some library to handle parsing

[purarue]

To compare: before this PR, it fails since I dont have twint configured

 $ hpi doctor -S my.twitter.all
✅ OK  : my.twitter.all
❗      - stats:                      computing failed
   Traceback (most recent call last):
     File "/home/sean/Repos/HPI-karlicoss/my/core/__main__.py", line 271, in modules_check
       res = stats()
     File "/home/sean/Repos/HPI-karlicoss/my/core/stats.py", line 21, in auto_stats
       return {k: stat(v) for k, v in providers.items()}
     File "/home/sean/Repos/HPI-karlicoss/my/core/stats.py", line 21, in <dictcomp>
       return {k: stat(v) for k, v in providers.items()}
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 454, in stat
       res = _stat_iterable(fr)
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 486, in _stat_iterable
       count = ilen(eit)
     File "/usr/lib/python3.10/site-packages/more_itertools/more.py", line 489, in ilen
       deque(zip(iterable, counter), maxlen=0)
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 469, in funcit
       for x in it:
     File "/home/sean/Repos/HPI-karlicoss/my/twitter/all.py", line 45, in likes
       yield from merge_tweets(
     File "/home/sean/Repos/HPI-karlicoss/my/core/common.py", line 377, in _warn_iterator
       for i in it:
     File "/home/sean/Repos/HPI-karlicoss/my/twitter/common.py", line 21, in merge_tweets
       yield from unique_everseen(chain(*sources), key=key)
     File "/usr/lib/python3.10/site-packages/more_itertools/recipes.py", line 410, in unique_everseen
       for element in iterable:
     File "/home/sean/Repos/HPI-karlicoss/my/core/source.py", line 44, in wrapper
       res = factory_func(**kwargs)
     File "/home/sean/Repos/HPI-karlicoss/my/twitter/all.py", line 28, in _likes_twint
       from . import twint as src
     File "/home/sean/Repos/HPI-karlicoss/my/twitter/twint.py", line 10, in <module>
       from my.config import twint as user_config
   ImportError: cannot import name 'twint' from 'my.config' (/home/sean/.config/my/my/config/__init__.py)

After this PR:

$ hpi doctor -S my.twitter.all
✅ OK  : my.twitter.all
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:55: UserWarning: Module my.twitter.twint (_likes_twint) could not be imported, or isn't configured properly
  To hide this message, add my.twitter.twint to your core config disabled_classes, like:

  class core:
      disabled_modules = ['my.twitter.twint']

    warn(f"""Module {module_name} ({factory_func.__qualname__}) could not be imported, or isn't configured properly\nTo hide this message, add {module_name} to your core config disabled_classes, like:
[INFO    2022-02-09 14:32:12 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip like
[INFO    2022-02-09 14:32:12 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:55: UserWarning: Module my.twitter.twint (_tweets_twint) could not be imported, or isn't configured properly
  To hide this message, add my.twitter.twint to your core config disabled_classes, like:

  class core:
      disabled_modules = ['my.twitter.twint']

    warn(f"""Module {module_name} ({factory_func.__qualname__}) could not be imported, or isn't configured properly\nTo hide this message, add {module_name} to your core config disabled_classes, like:
[INFO    2022-02-09 14:32:12 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip tweet
[INFO    2022-02-09 14:32:12 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
✅     - stats: {'likes': {'likes': {'count': 23}}, 'tweets': {'tweets': {'count': 5}}}

After I disable it in core.disabled_modules:

$ hpi doctor -S my.twitter.all
✅ OK  : my.twitter.all
[INFO    2022-02-09 14:34:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip like
[INFO    2022-02-09 14:34:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
[INFO    2022-02-09 14:34:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip tweet
[INFO    2022-02-09 14:34:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
✅     - stats: {'likes': {'likes': {'count': 23}}, 'tweets': {'tweets': {'count': 5}}}

[purarue]

I guess in the case of my.config errors, could be similar to this

But could probably just check the ImportError.name?

ipdb> err
ImportError("cannot import name 'twint' from 'my.config' (/home/sean/.config/my/my/config/__init__.py)")
ipdb> err.name
'my.config'

[karlicoss]

ooh, yeah good idea, let's try that?

[purarue]

Extracted it out into a helper so it could be used in both __main__.py and source.py, now looks like:

$ hpi doctor -S my.twitter.all
✅ OK  : my.twitter.all
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:56: UserWarning: Module my.twitter.twint (_likes_twint) could not be imported, or isn't configured properly
    medium(f"Module {module_name} ({factory_func.__qualname__}) could not be imported, or isn't configured properly")
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:57: UserWarning: To hide this message, add {module_name} to your core config disabled_classes, like:

  class core:
      disabled_modules = [{repr(module_name)}]

    warn("""To hide this message, add {module_name} to your core config disabled_classes, like:
   You may be missing the 'twint' section from your config.
   See https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#private-configuration-myconfig
    high("DEPRECATED! Please use my.core.common instead.")
[INFO    2022-02-09 16:30:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip like
[INFO    2022-02-09 16:30:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
  /home/sean/Repos/HPI-karlicoss/my/core/source.py:56: UserWarning: Module my.twitter.twint (_tweets_twint) could not be imported, or isn't configured properly
    medium(f"Module {module_name} ({factory_func.__qualname__}) could not be imported, or isn't configured properly")
   You may be missing the 'twint' section from your config.
   See https://github.com/karlicoss/HPI/blob/master/doc/SETUP.org#private-configuration-myconfig
[INFO    2022-02-09 16:30:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip tweet
[INFO    2022-02-09 16:30:33 my.twitter.archive archive.py:138] processing: /home/sean/data/twitter/archive.zip account
✅     - stats: {'likes': {'likes': {'count': 23}}, 'tweets': {'tweets': {'count': 5}}}

[purarue]

Hmm -- Failed to fetch http://azure.archive.ubuntu.com/ubuntu/pool/universe/r/rust-fd-find/fd-find_7.4.0-2build1_amd64.deb Could not connect to azure.archive.ubuntu.com:80 (52.250.76.244), connection timed out

seems to be an apt issue or something? May have been a temporary failure - perhaps retry in a bit

[karlicoss]

ah, annoying, yeah it happened to be a couple of times before. I'll nudge and merge tomorrow morning!