./mach bootstrap failure, possibly changes in python packages API

[hbina]

From a clean installation of Ubuntu 18.04.3

hbina@akarin:~/git/servo$ ./mach bootstrap
Traceback (most recent call last):
  File "./mach", line 103, in <module>
    main(sys.argv)
  File "./mach", line 31, in main
    sys.exit(mach_bootstrap.bootstrap_command_only(topdir))
  File "/home/hbina/git/servo/python/mach_bootstrap.py", line 227, in bootstrap_command_only
    from servo.bootstrap import bootstrap
  File "/home/hbina/git/servo/python/servo/bootstrap.py", line 11, in <module>
    import distro
ImportError: No module named distro

Simple enough, lets install module distro...

hbina@akarin:~/git/servo$ ./mach bootstrap
Traceback (most recent call last):
  File "./mach", line 103, in <module>
    main(sys.argv)
  File "./mach", line 31, in main
    sys.exit(mach_bootstrap.bootstrap_command_only(topdir))
  File "/home/hbina/git/servo/python/mach_bootstrap.py", line 234, in bootstrap_command_only
    bootstrap(context, force)
  File "/home/hbina/git/servo/python/servo/bootstrap.py", line 403, in bootstrap
    distrib, version = get_linux_distribution()
  File "/home/hbina/git/servo/python/servo/bootstrap.py", line 348, in get_linux_distribution
    distrib = six.ensure_str(distrib)
AttributeError: 'module' object has no attribute 'ensure_str'

[jdm]

Any difference if you remove python/_virtualenv? https://six.readthedocs.io/#six.ensure_str suggests that it should work.

[hbina]

Nope, still the same error.

hbina@akarin:~/git/servo$ rm -rf python/
mach_bootstrap.py      requirements-salt.txt  servo/                 tox.ini                
README.md              requirements.txt       tidy/                  _virtualenv/           
hbina@akarin:~/git/servo$ rm -rf python/_virtualenv/
hbina@akarin:~/git/servo$ ./mach bootstrap
Traceback (most recent call last):
  File "./mach", line 103, in <module>
    main(sys.argv)
  File "./mach", line 31, in main
    sys.exit(mach_bootstrap.bootstrap_command_only(topdir))
  File "/home/hbina/git/servo/python/mach_bootstrap.py", line 234, in bootstrap_command_only
    bootstrap(context, force)
  File "/home/hbina/git/servo/python/servo/bootstrap.py", line 403, in bootstrap
    distrib, version = get_linux_distribution()
  File "/home/hbina/git/servo/python/servo/bootstrap.py", line 348, in get_linux_distribution
    distrib = six.ensure_str(distrib)
AttributeError: 'module' object has no attribute 'ensure_str'

[jdm]

I wonder if bootstrap doesn't go through the whole virtualenv setup. What if you update your installed six package? You need six 1.12, based on https://github.com/benjaminp/six/blob/aa4e90bcd7b7bc13a71dfaebcb2021f4caaa8432/CHANGES#L13.

[hbina]

That seems to have solved it albeit being inconvenient.

hbina@akarin:~$ pip install -I six==1.12
Collecting six==1.12
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.12.0
hbina@akarin:~$ cd git/servo/
hbina@akarin:~/git/servo$ ./mach bootstrap
Dependencies were already installed!

[JoshMcguigan]

I see the same issue (with the same resolution) on Fedora 30, after manually installing python 2. Is this something that could/should be fixed in the bootstrap scripts? I'd be happy to submit a PR but I'm not quite sure where to start.

[jdm]

It sounds like that's the appropriate fix, yeah. We probably want to travel the code from the Mach script in the root directory and see what's different about the bootstrap command that end up in python/servo/bootstrap.py.

[JoshMcguigan]

Tracing the code path from the root ./mach to python/servo/bootstrap.py, I don't see anything that would install distro or six. Perhaps it is because _activate_virtualenv isn't called in bootstrap_command_only (it is only called in bootstrap) within /python/mach_bootstrap.py?

[jdm]

That sounds very likely!