Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mirage: 0.5.2 -> 0.6.4 #99121

Merged
merged 4 commits into from Dec 20, 2020
Merged

mirage: 0.5.2 -> 0.6.4 #99121

merged 4 commits into from Dec 20, 2020

Conversation

@vojta001
Copy link
Contributor

@vojta001 vojta001 commented Sep 29, 2020

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Sep 29, 2020

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Sep 30, 2020

The only problematic part is QML cache. Mirage stores its QML caches in .cache/mirage/mirage/qmlcache, however it is not purged on update, causing problems. I suspect it is related to the fact the mirage binary in the Nix store has the creation timestamp 0 (as all files in the store). Can we do anything about it?

@symphorien
Copy link
Member

@symphorien symphorien commented Oct 22, 2020

Works after clearing the cache

@colemickens
Copy link
Member

@colemickens colemickens commented Oct 28, 2020

We could write a wrapper that writes "last-booted-version" to a file in cache near (in?) mirage's dir. If we see it's different on next boot, we wipe the mirage cache dir before starting?

@symphorien
Copy link
Member

@symphorien symphorien commented Oct 29, 2020

Or maybe we can patch the path to the cache to be .cache/mirage/$out ? Is there a XDG_SOMETHING_DIR variable that we can set in the wrapper ?

@mirukana
Copy link

@mirukana mirukana commented Nov 12, 2020

There are some environment variable to disable or relocate the cache: https://doc.qt.io/qt-5/qmldiskcache.html

@symphorien
Copy link
Member

@symphorien symphorien commented Nov 12, 2020

Thanks !
This diff solves the segfault:

diff --git a/pkgs/applications/networking/instant-messengers/mirage/default.nix b/pkgs/applications/networking/instant-messengers/mirage/default.nix
index de84f3a0405..c204ca720e3 100644
--- a/pkgs/applications/networking/instant-messengers/mirage/default.nix
+++ b/pkgs/applications/networking/instant-messengers/mirage/default.nix
@@ -45,6 +45,7 @@ mkDerivation rec {
     buildPythonPath "$out $pythonPath"
     wrapProgram $out/bin/mirage \
       --prefix PYTHONPATH : "$PYTHONPATH" \
+      --run 'export QML_DISK_CACHE_PATH=''${XDG_CACHE_HOME:-$HOME/.cache}/mirage/qmlcache/${placeholder "out"}' \
       "''${qtWrapperArgs[@]}"
     '';

@mirukana
Copy link

@mirukana mirukana commented Nov 13, 2020

Try also mirukana/mirage#90 (comment), if this causes no issues we can use the flag by default for future releases.

@symphorien
Copy link
Member

@symphorien symphorien commented Nov 14, 2020

Indeed with

diff --git a/pkgs/applications/networking/instant-messengers/mirage/default.nix b/pkgs/applications/networking/instant-messengers/mirage/default.nix
index de84f3a0405..c57e57d5d75 100644
--- a/pkgs/applications/networking/instant-messengers/mirage/default.nix
+++ b/pkgs/applications/networking/instant-messengers/mirage/default.nix
@@ -38,7 +38,7 @@ mkDerivation rec {
 
   pythonPath = pypkgs;
 
-  qmakeFlags = [ "PREFIX=${placeholder "out"}" ];
+  qmakeFlags = [ "PREFIX=${placeholder "out"}" "CONFIG+=qtquickcompiler" ];
 
   dontWrapQtApps = true;
   postInstall = ''

I can alternate between stable's mirage and this PR without segfault.

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Nov 24, 2020

@vojta001 Can you update your PR with the cache fixes?

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Nov 24, 2020

@SuperSandro2000 And what solution do we prefer? Is it disabling the cache completely (probably slower, but does not fill the home dir with mess), or adding the derivation hash to the cache name?

@symphorien
Copy link
Member

@symphorien symphorien commented Nov 24, 2020

My preference goes to #99121 (comment)

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Nov 25, 2020

Here you go. Please review @colemickens

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Nov 26, 2020

Result of nixpkgs-review pr 99121 run on x86_64-linux 1

1 package built:
  • mirage-im

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Nov 26, 2020

on darwin:

clarative-5.12.7/lib/libQt5Qml.dylib /nix/store/qafg270zx0ih7rr18gxfvx0b8jb2dnfw-qtbase-5.12.7/lib/libQt5Network.dylib /nix/store/qafg270zx0ih7rr18gxfvx0b8jb2dnfw-qtbase-5.12.7/lib/libQt5Core.dylib -framework OpenGL -framework AGL
@nix { "action": "setPhase", "phase": "glibPreInstallPhase" }
glibPreInstallPhase
@nix { "action": "setPhase", "phase": "installPhase" }
installing
install flags: SHELL=/nix/store/k89nm2jva0qmvd970f84wq2iq1iwm9bs-bash-4.4-p23/bin/bash gsettingsschemadir=/nix/store/dkrwri1an9582pmpbz8c4250qdkzcmgc-mirage-0.5.2/share/gsettings-schemas/mirage-0.5.2/glib-2.0/schemas/ install
Makefile:835: warning: overriding recipe for target 'install_icons256'
Makefile:817: warning: ignoring old recipe for target 'install_icons256'
Makefile:839: warning: overriding recipe for target 'uninstall_icons256'
Makefile:821: warning: ignoring old recipe for target 'uninstall_icons256'
/nix/store/gzxz5xbx5bcvy5lvh8g9wv0871ipi9na-qtbase-5.12.7-dev/bin/qmake -install qinstall /private/tmp/nix-build-mirage-0.5.2.drv-0/source/extra/linux/mirage.png /nix/store/dkrwri1an9582pmpbz8c4250qdkzcmgc-mirage-0.5.2/share/icons/hicolor/256x256/apps/mirage.png
/nix/store/gzxz5xbx5bcvy5lvh8g9wv0871ipi9na-qtbase-5.12.7-dev/bin/qmake -install qinstall /private/tmp/nix-build-mirage-0.5.2.drv-0/source/extra/linux/mirage.desktop /nix/store/dkrwri1an9582pmpbz8c4250qdkzcmgc-mirage-0.5.2/share/applications/mirage.desktop

Builder called die: Cannot wrap '/nix/store/dkrwri1an9582pmpbz8c4250qdkzcmgc-mirage-0.5.2/bin/mirage' because it is not an executable file
Backtrace:
7 assertExecutable /nix/store/g5nqhchcnq9c9ym8ihqj8v5mbbibsa3x-hook/nix-support/setup-hook
136 wrapProgram /nix/store/g5nqhchcnq9c9ym8ihqj8v5mbbibsa3x-hook/nix-support/setup-hook
90 _callImplicitHook /nix/store/m40i7gdarksan1ax23vvwrq17p3pxjb3-stdenv-darwin/setup
106 _eval /nix/store/m40i7gdarksan1ax23vvwrq17p3pxjb3-stdenv-darwin/setup
49 runHook /nix/store/m40i7gdarksan1ax23vvwrq17p3pxjb3-stdenv-darwin/setup
1118 installPhase /nix/store/m40i7gdarksan1ax23vvwrq17p3pxjb3-stdenv-darwin/setup
1309 genericBuild /nix/store/m40i7gdarksan1ax23vvwrq17p3pxjb3-stdenv-darwin/setup
2 main /nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Nov 30, 2020

@SuperSandro2000 What do you suggest? Adding the qtquickcompiler for Linux only?

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 1, 2020

Mirage has never built on darwin so let's remove darwin from meta.platforms.

https://hydra.nixos.org/job/nixpkgs/trunk/mirage-im.x86_64-darwin/all

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Dec 1, 2020

Either mark it broken if upstream has darwin support. If upstream does not mention support for darwin remove it from platforms.

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 8, 2020

@SuperSandro2000 Done. Their support of Darwin isn't the best, but they claim it to be doable and there are unofficial guides for building on Darwin

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Dec 9, 2020

Result of nixpkgs-review pr 99121 run on x86_64-linux 1

1 package built:
  • mirage-im

@SuperSandro2000
Copy link
Member

@SuperSandro2000 SuperSandro2000 commented Dec 9, 2020

Result of nixpkgs-review pr 99121 run on x86_64-darwin 1

1 package marked as broken and skipped:
  • mirage-im

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 10, 2020

can you solve the merge conflict ?

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 11, 2020

Shall I merge master into my feature branch or shall I rebase onto master?

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 11, 2020

please rebase. You may find it easier if you squash your commits before rebasing.

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 13, 2020

Done

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 13, 2020

Could you please test the resulting binary? I have problems with it now and I don't know whether they are related to the state of my .cache folder or whether the build is broken.

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 14, 2020

works for me. How does it fail on your end?

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 14, 2020

The windows launches (therefore QML cache works just fine) and displays

Unexpected error occured: OlmAccountError

load_saved_accounts.<some uid>

Traceback (most recent call last):
  File "qrc:/src/backend/qml_bridge.py", line 83, in on_done
    result = future.result()
  File "/nix/store/cqh3nsk4z8lc960g8yqvz5nf2f6qx9v0-python3-3.8.6/lib/python3.8/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/nix/store/cqh3nsk4z8lc960g8yqvz5nf2f6qx9v0-python3-3.8.6/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
  File "qrc:/src/backend/backend.py", line 295, in load_saved_accounts
    return await asyncio.gather(*(
  File "qrc:/src/backend/backend.py", line 284, in resume
    await self.resume_client(
  File "qrc:/src/backend/backend.py", line 272, in resume_client
    await client.resume(user_id, token, device_id, state, status_msg)
  File "qrc:/src/backend/matrix_client.py", line 311, in resume
    self.restore_login(user_id, device_id, access_token)
  File "/nix/store/qssk7fsipsl6chwfr3kn9fcx9yp0pr7d-python3.8-matrix-nio-0.15.1/lib/python3.8/site-packages/nio/client/base_client.py", line 406, in restore_login
    self.load_store()
  File "/nix/store/qssk7fsipsl6chwfr3kn9fcx9yp0pr7d-python3.8-matrix-nio-0.15.1/lib/python3.8/site-packages/nio/client/base_client.py", line 382, in load_store
    self.olm = Olm(self.user_id, self.device_id, self.store)
  File "/nix/store/qssk7fsipsl6chwfr3kn9fcx9yp0pr7d-python3.8-matrix-nio-0.15.1/lib/python3.8/site-packages/nio/crypto/olm_machine.py", line 213, in __init__
    account = self.store.load_account()  # type: ignore
  File "/nix/store/qssk7fsipsl6chwfr3kn9fcx9yp0pr7d-python3.8-matrix-nio-0.15.1/lib/python3.8/site-packages/nio/store/database.py", line 166, in load_account
    return OlmAccount.from_pickle(
  File "/nix/store/qssk7fsipsl6chwfr3kn9fcx9yp0pr7d-python3.8-matrix-nio-0.15.1/lib/python3.8/site-packages/nio/crypto/sessions.py", line 45, in from_pickle
    account = super().from_pickle(pickle, passphrase)
  File "/nix/store/bxipga1mp2s3vgclma5linvjyn0ailwf-python3.8-python-olm-3.1.5/lib/python3.8/site-packages/olm/account.py", line 166, in from_pickle
    obj._check_error(ret)
  File "/nix/store/bxipga1mp2s3vgclma5linvjyn0ailwf-python3.8-python-olm-3.1.5/lib/python3.8/site-packages/olm/account.py", line 99, in _check_error
    raise OlmAccountError(last_error)
olm.account.OlmAccountError: UNKNOWN_PICKLE_VERSION

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 15, 2020

Wild guess: did you run a version of mirage using a later version of python which has introduced a new version of the pickle format?
@mirukana any idea of the cause?

@vojta001
Copy link
Contributor Author

@vojta001 vojta001 commented Dec 15, 2020

It is possible. I have compiled and run multiple variants during the development of this PR. Interestingly the current v0.5.2 works fine for me.

If the problem is on my machine only, let's just proceed and I'll simply wipe my .cache/mirage folder.

@symphorien
Copy link
Member

@symphorien symphorien commented Dec 20, 2020

@GrahamcOfBorg build mirage-im

@symphorien symphorien merged commit fbefb7e into NixOS:master Dec 20, 2020
19 checks passed
@vojta001 vojta001 deleted the mirage branch Dec 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants