ModuleNotFoundError: No module named 'msgraph'

[laughk]

Description

I have installed parsedmarc but it does not work with the following error:

$ parsedmarc --help
Traceback (most recent call last):
  File "/tmp/parsedmarc/bin/parsedmarc", line 5, in <module>
    from parsedmarc.cli import _main
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/__init__.py", line 31, in <module>
    from parsedmarc.mail import MailboxConnection
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/mail/__init__.py", line 2, in <module>
    from parsedmarc.mail.graph import MSGraphConnection
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/mail/graph.py", line 10, in <module>
    from msgraph.core import GraphClient
ModuleNotFoundError: No module named 'msgraph'

I am seeing this error in at least the following environments

• ArchLinux ( Linux 5.15.133.1-microsoft-standard-WSL2 #1 SMP Thu Oct 5 21:02:42 UTC 2023 x86_64 GNU/Linux ), Python 3.11
• Ubuntu 22.04.3 （Linux 6.2.0-1017-aws #17~22.04.1-Ubuntu SMP Fri Nov 17 21:19:35 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux ）, Python 3.10

Steps to reproduce

This occurs when you install parsedmarc and run the command in the following flow.

$ python -m venv venv
$ source venv/bin/activate
(venv) $ python -m pip install parsedmarc

Running parsedmarc in this state will result in the following

$ parsedmarc
Traceback (most recent call last):
  File "/tmp/parsedmarc/bin/parsedmarc", line 5, in <module>
    from parsedmarc.cli import _main
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/__init__.py", line 31, in <module>
    from parsedmarc.mail import MailboxConnection
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/mail/__init__.py", line 2, in <module>
    from parsedmarc.mail.graph import MSGraphConnection
  File "/tmp/parsedmarc/lib/python3.11/site-packages/parsedmarc/mail/graph.py", line 10, in <module>
    from msgraph.core import GraphClient
ModuleNotFoundError: No module named 'msgraph'

What else I have investigated

I looked up the module name msgraph and found that it refers to the dependent package msgraph-core,
The latest version of msgraph-core, 1.0.0, was released on 2024-01-23, the first major release since 2021-07-27.

• msgraph-core · PyPI
• Release v1.0.0 · microsoftgraph/msgraph-sdk-python-core

On the other hand, if you look at project.toml, you will see that >= 0.2.2.

parsedmarc/pyproject.toml

Line 50 in 7d2b431

"msgraph-core>=0.2.2",

Therefore, it is assumed that a new installation of parsedmarc installs msgraph-core 1.0.0, and that it no longer works.

Workaround

It seems to be possible to avoid this problem by specifying a msgraph-core version less than 1.0.0 as follows.

$ python -m venv venv
$ source venv/bin/activate
(venv) $ python -m pip install parsedmarc msgraph-core==0.2.2

(venv) $ parsedmarc --help
usage: parsedmarc [-h] [-c CONFIG_FILE] [--strip-attachment-payloads] [-o OUTPUT] [--aggregate-json-filename AGGREGATE_JSON_FILENAME] [--forensic-json-filename FORENSIC_JSON_FILENAME]
                  [--aggregate-csv-filename AGGREGATE_CSV_FILENAME] [--forensic-csv-filename FORENSIC_CSV_FILENAME] [-n NAMESERVERS [NAMESERVERS ...]] [-t DNS_TIMEOUT] [--offline]
                  [-s] [-w] [--verbose] [--debug] [--log-file LOG_FILE] [-v]
                  [file_path ...]

Parses DMARC reports

positional arguments:
  file_path             one or more paths to aggregate or forensic report files, emails, or mbox files'

options:
  -h, --help            show this help message and exit
  -c CONFIG_FILE, --config-file CONFIG_FILE
                        a path to a configuration file (--silent implied)
  --strip-attachment-payloads
                        remove attachment payloads from forensic report output
  -o OUTPUT, --output OUTPUT
                        write output files to the given directory
  --aggregate-json-filename AGGREGATE_JSON_FILENAME
                        filename for the aggregate JSON output file
  --forensic-json-filename FORENSIC_JSON_FILENAME
                        filename for the forensic JSON output file
  --aggregate-csv-filename AGGREGATE_CSV_FILENAME
                        filename for the aggregate CSV output file
  --forensic-csv-filename FORENSIC_CSV_FILENAME
                        filename for the forensic CSV output file
  -n NAMESERVERS [NAMESERVERS ...], --nameservers NAMESERVERS [NAMESERVERS ...]
                        nameservers to query
  -t DNS_TIMEOUT, --dns_timeout DNS_TIMEOUT
                        number of seconds to wait for an answer from DNS (default: 2.0)
  --offline             do not make online queries for geolocation or DNS
  -s, --silent          only print errors
  -w, --warnings        print warnings in addition to errors
  --verbose             more verbose output
  --debug               print debugging information
  --log-file LOG_FILE   output logging to a file
  -v, --version         show program's version number and exit

[jose-m-ferreira]

confirm workaround works:
ignore my tasteless non-usage of venv please.

jose@ubuntudmarc:~$ pip3 show -f parsedmarc
Name: parsedmarc
Version: 8.6.4
Summary: A Python package and CLI for parsing aggregate and forensic DMARC reports
Home-page:
Author:
Author-email: Sean Whalen whalenster@gmail.com
License:
Location: /usr/local/lib/python3.11/dist-packages
Requires: azure-identity, azure-monitor-ingestion, boto3, dateparser, dnspython, elasticsearch, elasticsearch-dsl, expiringdict, geoip2, google-api-core, google-api-python-client, google-auth, google-auth-httplib2, google-auth-oauthlib, imapclient, kafka-python, lxml, mailsuite, msgraph-core, publicsuffixlist, requests, tqdm, urllib3, xmltodict
Required-by:
Files:
../../../bin/parsedmarc
parsedmarc-8.6.4.dist-info/INSTALLER
parsedmarc-8.6.4.dist-info/METADATA
parsedmarc-8.6.4.dist-info/RECORD
parsedmarc-8.6.4.dist-info/REQUESTED
parsedmarc-8.6.4.dist-info/WHEEL
parsedmarc-8.6.4.dist-info/entry_points.txt
parsedmarc-8.6.4.dist-info/licenses/LICENSE
parsedmarc/init.py
parsedmarc/pycache/init.cpython-311.pyc
parsedmarc/pycache/cli.cpython-311.pyc
parsedmarc/pycache/elastic.cpython-311.pyc
parsedmarc/pycache/kafkaclient.cpython-311.pyc
parsedmarc/pycache/log.cpython-311.pyc
parsedmarc/pycache/loganalytics.cpython-311.pyc
parsedmarc/pycache/s3.cpython-311.pyc
parsedmarc/pycache/splunk.cpython-311.pyc
parsedmarc/pycache/syslog.cpython-311.pyc
parsedmarc/pycache/utils.cpython-311.pyc
parsedmarc/cli.py
parsedmarc/elastic.py
parsedmarc/kafkaclient.py
parsedmarc/log.py
parsedmarc/loganalytics.py
parsedmarc/mail/init.py
parsedmarc/mail/pycache/init.cpython-311.pyc
parsedmarc/mail/pycache/gmail.cpython-311.pyc
parsedmarc/mail/pycache/graph.cpython-311.pyc
parsedmarc/mail/pycache/imap.cpython-311.pyc
parsedmarc/mail/pycache/mailbox_connection.cpython-311.pyc
parsedmarc/mail/gmail.py
parsedmarc/mail/graph.py
parsedmarc/mail/imap.py
parsedmarc/mail/mailbox_connection.py
parsedmarc/resources/init.py
parsedmarc/resources/pycache/init.cpython-311.pyc
parsedmarc/resources/dbip/README.md
parsedmarc/resources/dbip/init.py
parsedmarc/resources/dbip/pycache/init.cpython-311.pyc
parsedmarc/resources/dbip/dbip-country-lite.mmdb
parsedmarc/s3.py
parsedmarc/splunk.py
parsedmarc/syslog.py
parsedmarc/utils.py

jose@ubuntudmarc:~$ parsedmarc --help
Traceback (most recent call last):
File "/usr/local/bin/parsedmarc", line 5, in
from parsedmarc.cli import _main
File "/usr/local/lib/python3.11/dist-packages/parsedmarc/init.py", line 31, in
from parsedmarc.mail import MailboxConnection
File "/usr/local/lib/python3.11/dist-packages/parsedmarc/mail/init.py", line 2, in
from parsedmarc.mail.graph import MSGraphConnection
File "/usr/local/lib/python3.11/dist-packages/parsedmarc/mail/graph.py", line 10, in
from msgraph.core import GraphClient
ModuleNotFoundError: No module named 'msgraph'

jose@ubuntudmarc:~$ sudo pip3 uninstall msgraph-core --break-system-package
Found existing installation: msgraph-core 1.0.0
Uninstalling msgraph-core-1.0.0:
Would remove:
/usr/local/lib/python3.11/dist-packages/msgraph_core-1.0.0.dist-info/*
/usr/local/lib/python3.11/dist-packages/msgraph_core/*
Proceed (Y/n)? y
Successfully uninstalled msgraph-core-1.0.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

jose@ubuntudmarc:~$ sudo pip3 install msgraph-core==0.2.2 --break-system-packages
Collecting msgraph-core==0.2.2
Downloading msgraph_core-0.2.2-py3-none-any.whl (15 kB)
Installing collected packages: msgraph-core
Successfully installed msgraph-core-0.2.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

jose@ubuntudmarc:~$ parsedmarc --version
8.6.4

[tjillez]

I did all this @jose-m-ferreira but i'm still getting:

parsedmarc.service: Failed with result 'exit-code'.
parsedmarc.service: Main process exited, code=exited, status=1/FAILURE
ModuleNotFoundError: No module named 'msgraph'
from msgraph.core import GraphClient
File "/opt/parsedmarc/venv/lib/python3.10/site-packages/parsedmarc/mail/graph.py", line 10, in
from parsedmarc.mail.graph import MSGraphConnection
JaFile "/opt/parsedmarc/venv/lib/python3.10/site-packages/parsedmarc/mail/init.py", line 2, in
from parsedmarc.mail import MailboxConnection
File "/opt/parsedmarc/venv/lib/python3.10/site-packages/parsedmarc/init.py", line 31, in
Jafrom parsedmarc.cli import _main
File "/opt/parsedmarc/venv/bin/parsedmarc", line 5, in

pip3 list | grep msgraph
msgraph-core 0.2.2

[jose-m-ferreira]

@tjilez

looks like you have msgraph-core 0.2.2 alright, right, but i'm curious as if any would be in the venv environment at /opt/parsedmarc and if so what version.

[tjillez]

@jose-m-ferreira
(venv) root@philip:/home/philip# parsedmarc --version
8.6.4

I think i got it working now :)

But i get:
ERROR:cli.py:799:You must supply input files or a mailbox connection. Do you know where i should put input files?

[jose-m-ferreira]

you have to configure a parsedmarc.ini somewhere. i actually haven't been able to get imap out of gmail to work.

[laughk]

This issue has been resolved in the following commit (version 8.7.0).

77132b3