Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hover: init 2020-01-03 #80075

Open
wants to merge 1 commit into
base: master
from
Open

hover: init 2020-01-03 #80075

wants to merge 1 commit into from

Conversation

@ericdallo
Copy link
Contributor

ericdallo commented Feb 14, 2020

Motivation for this change

Add hover command to be able to run flutter mobile apps on desktop.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.
@nixos-discourse

This comment has been minimized.

Copy link

nixos-discourse commented Feb 14, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review-may-2019/3032/116

@dtzWill

This comment has been minimized.

Copy link
Contributor

dtzWill commented Feb 14, 2020

I don't know much about flutter or hover, but gave it a build and noticed that ./result-bin/bin/app wants libflutter_engine.so but can't find it -- does a path needed to be added to its RPATH or something?

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 14, 2020

Hi @dtzWill, thanks for testing it :)
It looks like this fixed this app binary, the correct generated binary is only hover

@ericdallo ericdallo requested a review from jonringer Feb 14, 2020
@ofborg ofborg bot requested a review from kalbasit Feb 14, 2020
pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 14, 2020

the deps.nix is no longer necessary.

and please squash commits

@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from f8b54b2 to 2304dc4 Feb 14, 2020
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 14, 2020

Thanks @jonringer!
Done

@kalbasit

This comment has been minimized.

Copy link
Member

kalbasit commented Feb 14, 2020

@GrahamcOfBorg build hover

pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
pkgs/development/tools/hover/default.nix Outdated Show resolved Hide resolved
@ofborg ofborg bot requested a review from kalbasit Feb 14, 2020
@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from 5f8bd02 to 93a5078 Feb 15, 2020
@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from 93a5078 to 0d81d20 Feb 15, 2020
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 15, 2020

@GrahamcOfBorg build hover

@ericdallo ericdallo requested a review from jonringer Feb 15, 2020
@kalbasit

This comment has been minimized.

Copy link
Member

kalbasit commented Feb 15, 2020

@GrahamcOfBorg build hover

@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from 0d81d20 to 6ee20c5 Feb 15, 2020
@kalbasit

This comment has been minimized.

Copy link
Member

kalbasit commented Feb 16, 2020

@jonringer @kalbasit, I realized that using buildGoModule with subpackage we don't need flutterEngine lib anymore, because hover uses docker to cross-build when running the binary :D
This makes things a lot easier and compatible with darwin!

image

Sorry for realizing that after so many changes :/

Is that how it's supposed to be installed on Mac? It actually requires Docker to be installed first? Docker does not come pre-intalled on Mac, so I wonder if we need to wrap the binary with a check for Docker.

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 16, 2020

Yes @kalbasit, The only way I found to make this derivation works is to use this docker param.
https://github.com/go-flutter-desktop/hover#build-standalone-application

I wrapped the binary with docker

@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from 60cf797 to ff2ada9 Feb 16, 2020
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 16, 2020

@GrahamcOfBorg build hover

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 16, 2020

I almost find that concerning..... that's a lot of implicit assumptions to make about a users system

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 17, 2020

@jonringer, do you disagree in wrapping with docker? I'm open to suggestions :)

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 17, 2020

docker is a big hammer, with a big runtime closure:

$ nix-build -A docker
/nix/store/bikmp0s9z7bkrpj7z0q0bmq124c7wdfc-docker-19.03.6
[18:33:25] jon@jon-desktop /home/jon/projects/nixpkgs (expect-post-patch)
$ nix path-info -Sh ./result
/nix/store/bikmp0s9z7bkrpj7z0q0bmq124c7wdfc-docker-19.03.6	 493.4M

generally go packages are less then 10 MBs

$ nix path-info -Sh ./result-bin
/nix/store/y6hsm0z6w0lpzkf5n46715n26jdawgzn-direnv-2.21.2-bin	   5.9M
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 17, 2020

Sorry @jonringer, I tried to build without docker, but I always stuck in the same... when run hover, It tries to compile go-flutter, which needs libGL and other libs, I already tried to wrap, but it looks like doesn't work :/

Trying to wrap pkgconfig with libs:

  postInstall = ''
    mkdir -p $out/share
    cp -r assets $out/share/assets
    chmod -R a+rx $out/share/assets

    wrapProgram "$out/bin/hover" \
      --prefix LD_LIBRARY_PATH : ${stdenv.lib.makeLibraryPath [ glfw libX11 libXcursor libXinerama libXrandr libXxf86vm libXi libGL]} \
      --prefix PATH : ${pkgconfig}/bin \
      --prefix PKG_CONFIG_PATH : ${glfw}/lib \
      --prefix PKG_CONFIG_PATH : ${libGL}/lib
  '';

Error when `hover run`:

image

That's why I tried with docker and worked, but if you have any idea, please help me 😅

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 18, 2020

oh, you need to propagate build dependencies so they are available during run. You might be better off using buildFHSUserEnv

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 18, 2020

@jonringer I'm trying to use buildFHSUserEnv, but it looks like that when run the generated binary(hover run), it does not find flutter that is on my $PATH. Before buildFHSUserEnv this was working, Do I need to add something more?

Currently Im using flutter in a non nix way exporting on my bash_profile while there's no stable derivation. Can this problem be related to this?

export PATH="$PATH:$HOME/flutter/bin"

Source with buildFHSUserEnv:

{ buildGoModule, buildFHSUserEnv, docker, fetchFromGitHub, glfw, libGL, libX11, libXcursor, libXi,
  libXinerama, libXrandr, libXxf86vm, makeWrapper, stdenv}:
let
  package =  buildGoModule rec {
    pname = "hover";
    version = "unstable-2020-01-23";
    subPackages = [ "." ];

    modSha256 = "019lg3s86965jyqfy83hb52y5rb6g91prsg3ckjxbcnca68lsdja";

    src = fetchFromGitHub {
      rev = "a6f11fe3c4946c9e7cce75360902bd79a077ccf4";
      owner = "go-flutter-desktop";
      repo = pname;
      sha256 = "0pwdzd34zcbyspl7qiwfcrppr6zxp2zliqg9n8s6s3l6k7mffl9j";
    };

    nativeBuildInputs = [ makeWrapper ];

    patches = [
      ./fix-assets-path.patch
    ];
    postPatch = ''
      sed -i 's|@assetsFolder@|'"''${out}/share/assets"'|g' internal/fileutils/assets.go
    '';

    postInstall = ''
      mkdir -p $out/share
      cp -r assets $out/share/assets
      chmod -R a+rx $out/share/assets
    '';

    meta = with stdenv.lib; {
      description = "A build tool to run Flutter applications on desktop";
      homepage = "https://github.com/go-flutter-desktop/hover";
      license = licenses.bsd3;
      maintainers = [ maintainers.ericdallo ];
      platforms = platforms.linux ++ platforms.darwin;
    };
  };
in buildFHSUserEnv {
  name = package.pname;

  targetPkgs = pkgs: [
    libGL
  ];

  runScript = "hover";
}

Error:

hover: Failed to run flutter --version: exit status 
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 19, 2020

@jonringer
Do you think we can merge this PR with docker as the first release?

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 20, 2020

personally, I'm against using docker at runtime as it kind of conflicts with the ethos of nix.

And I'm not familiar enough with flutter/hover to know the right way to move forward here.

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 20, 2020

I see :/
Thanks for your time @jonringer !

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 23, 2020

@jonringer I realized that was my fault, forget it about this error.
Now I'm stuck again, but using buildFHSUserEnv and not using docker.

The derivation builds successfully but fails at runtime when running hover run because hover tries to run a go build when user runs hover run.
So when I hover run, it fails because go can't find libGL dependencies. It seems that the env built by buildFHSUserEnv is ignoring the dependencies from hover derivation.
Am I missing something?

Error
image

Derivation

{ buildGoModule, buildFHSUserEnv, pkgconfig, makeWrapper, fetchFromGitHub,
  libGL, glfw, stdenv, libX11, libXcursor, libXi, libXinerama, libXrandr,
  libXxf86vm}:
let
  pname = "hover";
  version = "unstable-2020-01-23";

  meta = with stdenv.lib; {
    description = "A build tool to run Flutter applications on desktop";
    homepage = "https://github.com/go-flutter-desktop/hover";
    license = licenses.bsd3;
    maintainers = [ maintainers.ericdallo ];
    platforms = platforms.linux ++ platforms.darwin;
  };

  fullPath = stdenv.lib.makeLibraryPath [
    glfw
    libGL
    libX11
    libXcursor
    libXi
    libXinerama
    libXrandr
    libXxf86vm
  ];

  hover = buildGoModule rec {
    inherit pname version meta;

    subPackages = [ "." ];

    modSha256 = "019lg3s86965jyqfy83hb52y5rb6g91prsg3ckjxbcnca68lsdja";

    src = fetchFromGitHub {
      rev = "a6f11fe3c4946c9e7cce75360902bd79a077ccf4";
      owner = "go-flutter-desktop";
      repo = pname;
      sha256 = "0pwdzd34zcbyspl7qiwfcrppr6zxp2zliqg9n8s6s3l6k7mffl9j";
    };

    buildInputs = [ makeWrapper pkgconfig ];

    patches = [
      ./fix-assets-path.patch
    ];

    postInstall = ''
      mkdir -p $out/share
      cp -r assets $out/share/assets
      chmod -R a+rx $out/share/assets

       wrapProgram $out/bin/hover \
        --prefix LD_LIBRARY_PATH : ${fullPath}:$out/lib/hover
    '';
  };

in  buildFHSUserEnv {
  name = "${pname}";
  targetPkgs = pkgs: [
    hover
  ];
  
  runScript = "hover";
@nixos-discourse

This comment has been minimized.

Copy link

nixos-discourse commented Feb 23, 2020

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/help-with-buildfhsuserenv/6014/1

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 23, 2020

using buildUserEnv to create a dev envrionment is getting out of my comfort zone. However, the headers for libX11 are contained in the "dev" output.

[14:08:41] jon@jon-desktop /home/jon/projects/nixpkgs (master)
$ nix-build -A xorg.libX11.dev
/nix/store/q3lrcl2fjcnflv2bxi3v8nwsbpljr2l8-libX11-1.6.8-dev
[14:08:59] jon@jon-desktop /home/jon/projects/nixpkgs (master)
$ tree ./result-dev
./result-dev
├── include
│   └── X11
│       ├── cursorfont.h
│       ├── ImUtil.h
│       ├── Xcms.h
│       ├── XKBlib.h
│       ├── XlibConf.h
│       ├── Xlib.h
│       ├── Xlibint.h
│       ├── Xlib-xcb.h
│       ├── Xlocale.h
│       ├── Xregion.h
│       ├── Xresource.h
│       └── Xutil.h
├── lib
│   └── pkgconfig
│       ├── x11.pc
│       └── x11-xcb.pc
└── nix-support
    └── propagated-build-inputs
@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Feb 23, 2020

this may blow up your closure size, but you might be able to do something like xorg.libX11.all, that should include all runtime, development, and documentation derivations for a given packge

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 24, 2020

@jonringer Thanks for the help! you were right! go build needs the header files that are on the dev derivation of the libs.
I added all the libs required but it fails with a segmentation fault :(
But if I use an empty nix-shell, it works!
I just to understand why this error outside the shell 🤔

Error
image

On a nix shell
image

Current code

{ buildGoModule, buildFHSUserEnv, pkgconfig, fetchFromGitHub,
  stdenv, libXxf86vm, writeScript, xorg, libglvnd}:

let
  pname = "hover";
  version = "unstable-2020-01-23";

  libs = with xorg; [
    libXi
    libXxf86vm
    libglvnd.dev
    libX11.dev
    libXcursor.dev
    libXinerama.dev
    libXrandr.dev
    libXrender.dev
    xorgproto
  ];

  hover = buildGoModule rec {
    inherit pname version;

    meta = with stdenv.lib; {
      description = "A build tool to run Flutter applications on desktop";
      homepage = "https://github.com/go-flutter-desktop/hover";
      license = licenses.bsd3;
      maintainers = [ maintainers.ericdallo ];
      platforms = platforms.linux ++ platforms.darwin;
    };

    subPackages = [ "." ];

    modSha256 = "019lg3s86965jyqfy83hb52y5rb6g91prsg3ckjxbcnca68lsdja";

    src = fetchFromGitHub {
      rev = "a6f11fe3c4946c9e7cce75360902bd79a077ccf4";
      owner = "go-flutter-desktop";
      repo = pname;
      sha256 = "0pwdzd34zcbyspl7qiwfcrppr6zxp2zliqg9n8s6s3l6k7mffl9j";
    };

    buildInputs = [ pkgconfig ] ++ libs;

    patches = [
      ./fix-assets-path.patch
    ];

    postPatch = ''
      sed -i 's|@assetsFolder@|'"''${out}/share/assets"'|g' internal/fileutils/assets.go
    '';

    postInstall = ''
      mkdir -p $out/share
      cp -r assets $out/share/assets
      chmod -R a+rx $out/share/assets
    '';
  };

in buildFHSUserEnv rec {
  name = pname;
  targetPkgs = pkgs: [
    hover
  ] ++ libs;

  runScript = "hover";
}
@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 29, 2020

If I build the derivation in a nix-shell works too:

 nix-shell -I nixpkgs=../nixpkgs/ -p hover

image

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 29, 2020

I realized that this segmentation fault was resulted from cache on $HOME/.cache/go-build, because I added the libglvnd.dev.
I tried to add this lib because I was not finding the GL/glx.h header lib but for some reason this lib leads to a segmentation fault, so I think this is not the right lib to get this GL header.
image

Does anybody knows in which lib from nix I can find this GL/glx.h?
I'm feeling that I'm so close to make this work 😣

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 29, 2020

I think mesa.dev should provide this, but on derivation the mesa/include/GL folder does not contain this header, but if I download manually from the source, it contains the header, it's like mesa derivation is removing the header 🤔

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Feb 29, 2020

Uow, it looks like on a stable branch(nixos-19.09) mesa.dev has this header file...
Can I import a derivation mesa.dev from a stable release 🤦‍♂ ?

@jonringer

This comment has been minimized.

Copy link
Contributor

jonringer commented Mar 1, 2020

Uow, it looks like on a stable branch(nixos-19.09) mesa.dev has this header file...
Can I import a derivation mesa.dev from a stable release man_facepalming ?

no, this is probably a version specific thing or a regression

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Mar 1, 2020

Yeah, I imagined that..
So I'm stuck on how to get this header lib :/

@ericdallo ericdallo force-pushed the ericdallo:hover-flutter branch from ff2ada9 to 324c55b Mar 20, 2020
@zimbatm

This comment has been minimized.

Copy link
Member

zimbatm commented Mar 22, 2020

using nix-locate GL/glx.h, there is one package that stands out: libglvnd.dev.

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Mar 22, 2020

Thanks @zimbatm, but I tried this lib and result in an error as I said it here 😔

@zimbatm

This comment has been minimized.

Copy link
Member

zimbatm commented Mar 22, 2020

My bad, I read this PR too quickly.

I found pkgs/build-support/add-opengl-runpath/setup-hook.sh which might be interesting. A few programs like tensorflow and blender call addOpenGLRunpath.

@ericdallo

This comment has been minimized.

Copy link
Contributor Author

ericdallo commented Mar 22, 2020

@zimbatm, thanks for the tip, I'm trying to use addOpenGLRunpath like blender, but I think I'm doing something wrong:

{ lib, buildGoModule, buildFHSUserEnv, pkgconfig, fetchFromGitHub,
  stdenv, libXxf86vm, writeScript, xorg, libglvnd, addOpenGLRunpath }:

let
  pname = "hover";
  version = "unstable-2020-01-23";

  libs = with xorg; [
    libXi
    libXxf86vm
    libglvnd.dev
    libX11.dev
    libXcursor.dev
    libXinerama.dev
    libXrandr.dev
    libXrender.dev
    xorgproto
  ];

  hover = buildGoModule rec {
    inherit pname version;

    meta = with stdenv.lib; {
      description = "A build tool to run Flutter applications on desktop";
      homepage = "https://github.com/go-flutter-desktop/hover";
      license = licenses.bsd3;
      maintainers = [ maintainers.ericdallo ];
      platforms = platforms.linux ++ platforms.darwin;
    };

    subPackages = [ "." ];

    modSha256 = "019lg3s86965jyqfy83hb52y5rb6g91prsg3ckjxbcnca68lsdja";

    src = fetchFromGitHub {
      rev = "a6f11fe3c4946c9e7cce75360902bd79a077ccf4";
      owner = "go-flutter-desktop";
      repo = pname;
      sha256 = "0pwdzd34zcbyspl7qiwfcrppr6zxp2zliqg9n8s6s3l6k7mffl9j";
    };

    nativeBuildInputs = [ addOpenGLRunpath ];

    buildInputs = [ pkgconfig ] ++ libs;

    patches = [
      ./fix-assets-path.patch
    ];

    postPatch = ''
      sed -i 's|@assetsFolder@|'"''${out}/share/assets"'|g' internal/fileutils/assets.go
    '';

    postInstall = ''
      mkdir -p $out/share
      cp -r assets $out/share/assets
      chmod -R a+rx $out/share/assets
    '';

    postFixup = ''
      addOpenGLRunpath $out/bin/hover
    '';
  };

in buildFHSUserEnv rec {
  name = pname;
  targetPkgs = pkgs: [
    hover
  ] ++ libs;

  runScript = "hover";
}

Output:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants
You can’t perform that action at this time.