New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add `dub install` command #839

Open
ximion opened this Issue May 8, 2016 · 8 comments

Comments

6 participants
@ximion
Contributor

ximion commented May 8, 2016

As per issue #811 it would be nice to have a dub install command to install the built binary/shared|static library and accompanying data files into a standard directory hierarchy used in Linux systems. Ideally, the install command would - for shared/static libraries - also install the needed sources or .di files into a directory hierarchy which can be used as a local repository, as outlined in #838.
This would be of great help for packaging dub-using D code in Linux distributions.

Since there were concerns about users blindly dub-installing files into their filesystem, I propose to make setting a --target= parameter mandatory. Distributors could run dub install --target=debian/tmp without issues, and to protect users dub could ask for a second confirmation when someone passes / as target directory (something along the line of "This will install this software permanently on your system, and might be incompatible with distribution packageing. Do you want to proceed?"). That being said, theoretically a dub uninstall command would also be possible to remove installed stuff from the file system again ^^

But this report is about dub install --target=DIR - would be awesome if we could have this feature.

@schancel

This comment has been minimized.

Show comment
Hide comment
@schancel

schancel Jul 2, 2016

Maybe take a look at go install which will look for a GOBIN environment variable and attempt to place the binaries in the path given by that variable. It makes it rather convenient to just add to your shell environment if you work with go quite a bit.

schancel commented Jul 2, 2016

Maybe take a look at go install which will look for a GOBIN environment variable and attempt to place the binaries in the path given by that variable. It makes it rather convenient to just add to your shell environment if you work with go quite a bit.

@mogud

This comment has been minimized.

Show comment
Hide comment
@mogud

mogud Aug 30, 2016

IMO. dub install is important for developers who write tools in D. For example: D Profile Viewer

mogud commented Aug 30, 2016

IMO. dub install is important for developers who write tools in D. For example: D Profile Viewer

@MartinNowak

This comment has been minimized.

Show comment
Hide comment
@MartinNowak

MartinNowak Jun 17, 2017

Member

Where to install the files? Having them (at least the source) somewhat isolated seems worthwhile.
How to deal with different versions? Leave it to the user (or pkg-config like tools) to declare correct import paths?

/usr/include/d/<package>-<version>/package to be used with -I/usr/include/d/<package>-<version> for packages that have source files in their root folder (see #735).
/usr/lib64/lib<package>-<maj.min>.a
/usr/lib64/lib<package>.so.maj.min.patch (+symlinks)

Library path depends on distribution, e.g. /usr/lib/x86_64-linux-gnu on debian, /usr/lib64/ on redhat...

What about integration with .rpm and .deb building instead of directly installing files?

Member

MartinNowak commented Jun 17, 2017

Where to install the files? Having them (at least the source) somewhat isolated seems worthwhile.
How to deal with different versions? Leave it to the user (or pkg-config like tools) to declare correct import paths?

/usr/include/d/<package>-<version>/package to be used with -I/usr/include/d/<package>-<version> for packages that have source files in their root folder (see #735).
/usr/lib64/lib<package>-<maj.min>.a
/usr/lib64/lib<package>.so.maj.min.patch (+symlinks)

Library path depends on distribution, e.g. /usr/lib/x86_64-linux-gnu on debian, /usr/lib64/ on redhat...

What about integration with .rpm and .deb building instead of directly installing files?

@mogud

This comment has been minimized.

Show comment
Hide comment
@mogud

mogud Jun 26, 2017

Why not install them in a hidden folder of $HOME directory? So many others do like this.

mogud commented Jun 26, 2017

Why not install them in a hidden folder of $HOME directory? So many others do like this.

@MartinNowak

This comment has been minimized.

Show comment
Hide comment
@MartinNowak

MartinNowak Dec 23, 2017

Member

Indeed dub should offer an installation mechanism similar to pip install <pkg> --user, and maybe also some system-wide installation pip install <pkg>, npm install -g <pkg>.
The binaries would just be symlinks, so that packages can access their normal resources (using thisExePath).
Shared libraries would also need to be symlinked.

Maybe we could simply start this as experiment, installing to ~/.local/bin and ~/.local/lib, to figure out where this goes. Should be simple enough to be implemented by anyone interested.

Member

MartinNowak commented Dec 23, 2017

Indeed dub should offer an installation mechanism similar to pip install <pkg> --user, and maybe also some system-wide installation pip install <pkg>, npm install -g <pkg>.
The binaries would just be symlinks, so that packages can access their normal resources (using thisExePath).
Shared libraries would also need to be symlinked.

Maybe we could simply start this as experiment, installing to ~/.local/bin and ~/.local/lib, to figure out where this goes. Should be simple enough to be implemented by anyone interested.

@wilzbach

This comment has been minimized.

Show comment
Hide comment
@wilzbach

wilzbach May 6, 2018

Member

@ximion: @s-ludwig and I briefly discussed this during the DConf hackathon. The summary:

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro.
As far was we could tell for system installations a wrapper around dub or tool that uses dub's API should work fine and would avoid the predictable issue flood if we implement this only for some distros.
Once #1459 gets merged, it should be rather straight-forward to package dub packages on debian?

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

Member

wilzbach commented May 6, 2018

@ximion: @s-ludwig and I briefly discussed this during the DConf hackathon. The summary:

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro.
As far was we could tell for system installations a wrapper around dub or tool that uses dub's API should work fine and would avoid the predictable issue flood if we implement this only for some distros.
Once #1459 gets merged, it should be rather straight-forward to package dub packages on debian?

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

@wilzbach wilzbach added this to To Do in dub sprint II May 6, 2018

@ximion

This comment has been minimized.

Show comment
Hide comment
@ximion

ximion May 7, 2018

Contributor

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro.

It's not that complicated. Actually I think it's the easiest issue to resolve. All distros follow the FHS1 for the most basic directories. Here is a full implementation in Python that Meson uses: https://github.com/mesonbuild/meson/blob/master/mesonbuild/mesonlib.py#L490-L538 It's important though that these default paths can be overridden by the Linux distributor individually.

Once #1459 gets merged, it should be rather straight-forward to package dub packages on debian?

That depends... I can't promise anything, I would actually need to test that, since my last attempt was years ago.
I do remember though that dub was writing to $HOME during a regular compilation process for some caching. That is also a forbidden practice - while not a dealbreaker like dub not being able to use local packages, it would force us to add some annoying hacks.
Some interoperability with other projects would also be neat, via e.g. pkg-config2 (but that's also not mandatory - it would allow projects not using dub to use libraries built with dub easily though).
Fixing #838 will indeed fix the issue that ultimately killed dub usage for package building in Debian back in the days - thank you for that! :-)

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

Those paths are completely non-standard though, although commonly used. Users can't use them without some manual adjustments (which IMHO is a good thing, so users don't accidentally call binaries from their home directories).
We do have some discussion though to specify at least the bin directory in some way3.

Contributor

ximion commented May 7, 2018

@s-ludwig is still a bit wary about complicating dub with the logic of figuring out where to install system binaries as this heavily depends on the distro.

It's not that complicated. Actually I think it's the easiest issue to resolve. All distros follow the FHS1 for the most basic directories. Here is a full implementation in Python that Meson uses: https://github.com/mesonbuild/meson/blob/master/mesonbuild/mesonlib.py#L490-L538 It's important though that these default paths can be overridden by the Linux distributor individually.

Once #1459 gets merged, it should be rather straight-forward to package dub packages on debian?

That depends... I can't promise anything, I would actually need to test that, since my last attempt was years ago.
I do remember though that dub was writing to $HOME during a regular compilation process for some caching. That is also a forbidden practice - while not a dealbreaker like dub not being able to use local packages, it would force us to add some annoying hacks.
Some interoperability with other projects would also be neat, via e.g. pkg-config2 (but that's also not mandatory - it would allow projects not using dub to use libraries built with dub easily though).
Fixing #838 will indeed fix the issue that ultimately killed dub usage for package building in Debian back in the days - thank you for that! :-)

Anyhow, installing binaries to ~./local/bin and ~/.local/lib indeed sounds like sth. we should really try to add support for in dub.

Those paths are completely non-standard though, although commonly used. Users can't use them without some manual adjustments (which IMHO is a good thing, so users don't accidentally call binaries from their home directories).
We do have some discussion though to specify at least the bin directory in some way3.

@Timoses

This comment has been minimized.

Show comment
Hide comment
@Timoses

Timoses Jul 13, 2018

What about a simple configuration field in one of dubs settings.json? E.g. "installPath".

When it is not set and a user runs
dub install
it prints
Specify installPath in settings.json or use --target=...

This would at least allow users to set up their environments so dub does the "right" (user-defined) thing. It could be further split up into binaryInstallPath and libraryInstallPath.

Timoses commented Jul 13, 2018

What about a simple configuration field in one of dubs settings.json? E.g. "installPath".

When it is not set and a user runs
dub install
it prints
Specify installPath in settings.json or use --target=...

This would at least allow users to set up their environments so dub does the "right" (user-defined) thing. It could be further split up into binaryInstallPath and libraryInstallPath.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment