Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spidermonkey_78 fails to build on staging #104483

Closed
FRidh opened this issue Nov 21, 2020 · 9 comments
Closed

spidermonkey_78 fails to build on staging #104483

FRidh opened this issue Nov 21, 2020 · 9 comments
Labels
0.kind: bug 6.topic: python

Comments

@FRidh
Copy link
Member

FRidh commented Nov 21, 2020 •
edited
Loading

Describe the bug
spidermonkey_78 fails to build on staging.

To Reproduce

$ nix-build -A spidermonkey_78
this derivation will be built:
  /nix/store/grqcsv7f65wfxmgbpay79gqr75qchaa0-spidermonkey-78.4.0.drv
building '/nix/store/grqcsv7f65wfxmgbpay79gqr75qchaa0-spidermonkey-78.4.0.drv'...
unpacking sources
unpacking source archive /nix/store/mjy9c61qwf4vna4070c7raxc164y5xcf-firefox-78.4.0esr.source.tar.xz
source root is firefox-78.4.0
setting SOURCE_DATE_EPOCH to timestamp 1602668438 of file firefox-78.4.0/sourcestamp.txt
patching sources
configuring
configure flags: --prefix=/nix/store/35sw6iyxy0jijbcdzh6smna9qwyn2a5l-spidermonkey-78.4.0 --with-system-zlib --with-system-nspr --with-system-icu --with-intl-api --enable-readline --enable-shared-js --disable-jemalloc --enable-optimize --enable-release
Creating Python 3 environment
Using base prefix '/nix/store/6985jid6k9f1bcbngkbr8k5m5cf7rhl5-python3-3.8.6'
New python executable in /build/firefox-78.4.0/obj/_virtualenvs/init_py3/bin/python3
Also creating executable in /build/firefox-78.4.0/obj/_virtualenvs/init_py3/bin/python
Command /build/firefox-78.4....init_py3/bin/python3 -m pip config list had error code 1
Traceback (most recent call last):
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 2634, in <module>
    main()
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 860, in main
    create_environment(
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 1179, in create_environment
    install_wheel(to_install, py_executable, search_dirs, download=download)
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 1023, in install_wheel
    _install_wheel_with_search_dir(download, project_names, py_executable, search_dirs)
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 1050, in _install_wheel_with_search_dir
    config = _pip_config(py_executable, python_path)
  File "/build/firefox-78.4.0/third_party/python/virtualenv/virtualenv.py", line 1134, in _pip_config
    config[key] = ast.literal_eval(value)
  File "/nix/store/6985jid6k9f1bcbngkbr8k5m5cf7rhl5-python3-3.8.6/lib/python3.8/ast.py", line 59, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/nix/store/6985jid6k9f1bcbngkbr8k5m5cf7rhl5-python3-3.8.6/lib/python3.8/ast.py", line 47, in parse
    return compile(source, filename, mode, flags,
  File "<unknown>", line 1
    __import__(name, globals(), locals(), ['build_time_vars'], 0)
    ^
IndentationError: unexpected indent
Traceback (most recent call last):
  File "../js/src/../../configure.py", line 181, in <module>
    sys.exit(main(sys.argv))
  File "../js/src/../../configure.py", line 52, in main
    sandbox.run(os.path.join(os.path.dirname(__file__), 'moz.configure'))
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 445, in run
    self.include_file(path)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 436, in include_file
    exec_(code, self)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/util.py", line 53, in exec_
    exec(object, globals, locals)
  File "/build/firefox-78.4.0/moz.configure", line 7, in <module>
    include('build/moz.configure/init.configure')
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 773, in include_impl
    self.include_file(what)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 436, in include_file
    exec_(code, self)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/util.py", line 53, in exec_
    exec(object, globals, locals)
  File "/build/firefox-78.4.0/build/moz.configure/init.configure", line 254, in <module>
    def virtualenv_python3(env_python, build_env, mozconfig, help):
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 756, in decorator
    depends = DependsFunction(self, func, dependencies, when=when)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 129, in __init__
    sandbox._value_for(self)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 542, in _value_for
    return self._value_for_depends(obj)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/util.py", line 1018, in method_call
    cache[args] = self.func(instance, *args)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 551, in _value_for_depends
    value = obj.result()
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/util.py", line 1018, in method_call
    cache[args] = self.func(instance, *args)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 155, in result
    return self._func(*resolved_args)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/configure/__init__.py", line 1161, in wrapped
    return new_func(*args, **kwargs)
  File "/build/firefox-78.4.0/build/moz.configure/init.configure", line 357, in virtualenv_python3
    manager.build(python)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/virtualenv.py", line 486, in build
    self.create(python)
  File "/build/firefox-78.4.0/python/mozbuild/mozbuild/virtualenv.py", line 249, in create
    raise Exception(
Exception: Failed to create virtualenv: /build/firefox-78.4.0/obj/_virtualenvs/init_py3 (virtualenv.py retcode: 1)

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Notify maintainers

@abbradar @lostnet

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

Nixpkgs staging at 54cb417.

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: `spidermonkey_78`
# a list of nixos modules affected by the problem
module:
@FRidh FRidh changed the title spidermonkey_78 fails to build spidermonkey_78 fails to build on staging Nov 21, 2020
@andir
Copy link
Member

andir commented Nov 22, 2020 •
edited
Loading

For what it is worth I ran a bisect on this:

git bisect start
# bad: [54cb417ba00a7a1463fcd32fc6f58eae09aa5ce8] Merge staging-next into staging
git bisect bad 54cb417ba00a7a1463fcd32fc6f58eae09aa5ce8
# bad: [8d210e2ea42f5a6ef10320800355ecd4b49fb428] python3.pkgs.pipInstallHook: don't warn on script installation
git bisect bad 8d210e2ea42f5a6ef10320800355ecd4b49fb428
# good: [6625284c397b44bc9518a5a1567c1b5aae455c08] Merge pull request #104182 from helsinki-systems/fix/gst-plugins-bad
git bisect good 6625284c397b44bc9518a5a1567c1b5aae455c08
# good: [f7ec01ab6c91b30cc1df54cd49151e401b3fdcb0] mailutils: fix cross-compilation
git bisect good f7ec01ab6c91b30cc1df54cd49151e401b3fdcb0
# good: [26803925aa41f3ba67d70facc5c020aa20cf70a8] ngspice: 32 -> 33
git bisect good 26803925aa41f3ba67d70facc5c020aa20cf70a8
# good: [96fb471fc76ee76604ef63b43bb8e9ef0205f48b] libevdev: 1.9.1 -> 1.10.0
git bisect good 96fb471fc76ee76604ef63b43bb8e9ef0205f48b
# good: [354a3b92b4daba501c99329352a646b627f1c848] gmp: 6.2.0 -> 6.2.1
git bisect good 354a3b92b4daba501c99329352a646b627f1c848
# bad: [346b2212c7c2b4684c95372fec4d34c3c01370cd] pythonPackages.pybind11: 2.5.0 -> 2.6.1
git bisect bad 346b2212c7c2b4684c95372fec4d34c3c01370cd
# good: [5354cd0a162a4a262fa03f4bfaed7de01d589271] ghc: Use a real URL for D5123.diff
git bisect good 5354cd0a162a4a262fa03f4bfaed7de01d589271
# bad: [27d3a18ac7d34b66a14b634cb785e8c8caef024b] buildPython*: don't catch conflicts when cross-compiling
git bisect bad 27d3a18ac7d34b66a14b634cb785e8c8caef024b
# bad: [6501d9880c3f612afd417e7c45fff2f58f19009a] talloc: fix cross-compilation to 32-bit platforms
git bisect bad 6501d9880c3f612afd417e7c45fff2f58f19009a
# bad: [11806b6ede420bbdb980ca1c36918b71e9ddc0a2] cpython: fix finding headers when cross-compiling extension modules
git bisect bad 11806b6ede420bbdb980ca1c36918b71e9ddc0a2
# first bad commit: [11806b6ede420bbdb980ca1c36918b71e9ddc0a2] cpython: fix finding headers when cross-compiling extension modules

Reverting 11806b6 seems to fix it.

@FRidh
Copy link
Member Author

FRidh commented Nov 22, 2020

Thank you. I was afraid it had something to do with a Python change.
cc @lopsided98

@FRidh
Copy link
Member Author

FRidh commented Nov 22, 2020

Oh, the patch is indented with 2 spaces...

@andir
Copy link
Member

andir commented Nov 22, 2020

Fixing the indent on the python 3.7+ patch (see diff below) didn't change the situation. Did you mean something else in the patches?

diff --git a/pkgs/development/interpreters/python/cpython/3.7/fix-finding-headers-when-cross-compiling.patch b/pkgs/development/interpreters/python/cpython/3.7/fix-finding-headers-when-cross-compiling.patch
index 543e267e94b..12020586582 100644
--- a/pkgs/development/interpreters/python/cpython/3.7/fix-finding-headers-when-cross-compiling.patch
+++ b/pkgs/development/interpreters/python/cpython/3.7/fix-finding-headers-when-cross-compiling.patch
@@ -36,16 +36,16 @@ index 37feae5df7..6d4ad06696 100644
 -        python_dir = 'python' + get_python_version() + build_flags
 -        return os.path.join(prefix, "include", python_dir)
 +        if prefix is None:
-+          if plat_specific:
-+            return get_config_var('CONFINCLUDEPY')
-+          else:
-+            return get_config_var('INCLUDEPY')
++            if plat_specific:
++                return get_config_var('CONFINCLUDEPY')
++            else:
++                return get_config_var('INCLUDEPY')
 +        else:
-+          python_dir = 'python' + get_python_version() + build_flags
-+          return os.path.join(prefix, "include", python_dir)
++            python_dir = 'python' + get_python_version() + build_flags
++            return os.path.join(prefix, "include", python_dir)
      elif os.name == "nt":
 +        if prefix is None:
-+          prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
++            prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
          if python_build:
              # Include both the include and PC dir to ensure we can find
              # pyconfig.h

@FRidh
Copy link
Member Author

FRidh commented Nov 22, 2020

My build also finished, and indeed, changing indentation did not fix it.

FRidh added a commit that referenced this issue Nov 22, 2020
@FRidh
Revert "cpython: fix finding headers when cross-compiling extension m…
6a23dde 
…odules"

This breaks virtualenv #104483.
We should probably not move `_sysconfigdata` but just copy it for cross to
another place or output.

This reverts commit 11806b6.
@FRidh
Copy link
Member Author

FRidh commented Nov 22, 2020 •
edited
Loading

Moving _sysconfigdata__linux_x86_64-linux-gnu breaks installing packages in a virtualenv. Reverted in 6a23dde.

We should not move _sysconfigdata but copy or link it instead. cc @lopsided98 Testing this now...

@FRidh
Copy link
Member Author

FRidh commented Nov 23, 2020 •
edited
Loading

reverted the revert and modified it in 6100bc2
spidermonkey_78 builds, cross-compiling numpy and cffi still works, and python3.tests also succeeds.

@FRidh FRidh closed this as completed Nov 23, 2020
@andir
Copy link
Member

andir commented Nov 23, 2020 via email

@FRidh
Copy link
Member Author

FRidh commented Nov 23, 2020

If cross-compilation still works what was the point of the actual patch?
Wasn't that also just added for cross compilation support?

Yes, what I meant is that my revert now fixes the original commit so spidermonkey builds, while not breaking cross-compilation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0.kind: bug 6.topic: python
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@andir @FRidh