1.17.1 introduces ModuleNotFoundError: No module named 'starters'

[mtucker502]

Overview

I upgraded to 1.17.1 to fix #729 but now when running icloud I receive a ModuleNotFoundError: No module named 'starters'.

Steps to Reproduce

1. Install icloudpd@1.17.1
2. Run icloud

Expected Behavior

No errors and icloud should display help.

Actual Behavior

starters module error:

/volume1/homes/mike/venv-icloudpd-amber/bin/icloud
(venv-icloudpd-amber) mike@thevault0:~$ icloud
Traceback (most recent call last):
  File "/volume1/homes/mike/venv-icloudpd-amber/bin/icloud", line 5, in <module>
    from starters.icloud import main
ModuleNotFoundError: No module named 'starters'

Context

This was broken in pull 726.

[AndreyNikiforov]

are you using pip to get 1.17.1? Is it linux?

[AndreyNikiforov]

I failed to repro with the following:

docker run --rm -it debian:11

...and then in container:

cd && \
apt-get update && \
apt-get install -y pip && \
pip install icloudpd==1.17.1 && \
icloud -h && \
icloudpd --version

also works on ubuntu:24.04 image with extra installation hoops

[mtucker502]

I tried from Synology:

(venv-icloudpd-amber) mike@thevault0:~$ python -V
Python 3.9.14

And from WSL:

Python 3.8.2

When I check the contents of the icloud look at this:

(venv-icloudpd-amber) mike@thevault0:~$ cat /volume1/homes/mike/venv-icloudpd-amber/bin/icloud
#!/volume1/homes/mike/venv-icloudpd-amber/bin/python3.9
# -*- coding: utf-8 -*-
import re
import sys
from starters.icloud import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
(venv-icloudpd-amber) mike@thevault0:~$

But in the container:

root@8de8a14021d3:~# which icloud
/usr/local/bin/icloud
root@8de8a14021d3:~# cat /usr/local/bin/icloud
#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from icloud import __main__
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(__main__())

[mtucker502]

I did some more testing. On a Linux VM pip install breaks. But within an ubuntu:22.04 container it works.

But I can repro using ubuntu:20.04 (same as host).

docker run --rm -it ubuntu:20.04

cd && \
apt-get update && \
apt-get install -y pip && \
pip install icloudpd==1.17.1 && \
icloud

The ubuntu:22.04 container downloads icloudpd-1.17.1-py2.py3-none-manylinux_2_17_x86_64.whl while the host and ubuntu:20.04 container downloads icloudpd-1.17.1-py3-none-any.whl.

The VM host (Python 3.8.10), ubuntu:20.04 (Python 3.8.10) and my Synology (Python 3.8.10) are running Python 3.8.

[mtucker502]

More testing:

• python 3.9.14 on Linux with a pip install and it's broken.
• python 3.8.10 on MacOS with pip install and it's working
• python 3.10 on Synology with pip install and it's broken.

[AndreyNikiforov]

The ubuntu:22.04 container downloads icloudpd-1.17.1-py2.py3-none-manylinux_2_17_x86_64.whl while the host and ubuntu:20.04 container downloads icloudpd-1.17.1-py3-none-any.whl.

There are two questions actually: a) why manylinux_2_17_x86_64 was not picked for ubuntu:20.04 (and your host) -- IIUC it should b) why src distribution is not working (~original issue)

Context

we package icloudpd as a single binary without python dependency for consistent behavior and distribute that through different mechanisms: Github release page, pypi, npm, dockerhub, etc (the list will hopefully grow).

The fallback for pypi is src distribution. manylinux is a standard pypi mechanism to match binaries to running environment.

For binary distributions we use small scripts (py, js) to start process with actual binary. That was a work around a) permissions on the target b) quirks of the packager for binaries

Fixing & Testing

We need to validate behavior through all distribution paths. There are no automated tests for that and using github action may be a way to go for testing. If not, manual validation can be done, but is tedious.

[yorkday]

I have the same problem as others, no longer able to run it natively on Synology.
I switched to using a Docker on Synology and now I can run it. I'd like to run it natively as it's much faster than running through Docker, but I feel like Docker will reduce compatibility problems going forward, so it is a tradeoff.

[AndreyNikiforov]

I'd like to run it natively as it's much faster than running through Docker

I am curious how did you measure performance and what were your findings comparing host python vs python in docker for icloudpd?

IIUC processes running in docker are just regular linux processes, so no perf impact expected. container filesystem access in docker is special perf consideration, but icloudpd uses bounded volumes, so no perf impact either.

Overall, I expect performance of icloudpd to be bound by internet speed, not CPU.

[yorkday]

TBH I haven't been able to benchmark the two, as since the authentication issues, I've been unable to run icloudpd natively on Synology - so it was only in my testing and setup of Docker icloudpd that I noticed a slowdown in syncing speed.
If I can get icloudpd running natively again, I could try some proper benchmarks.
Appreciate all your work on this invaluable tool.