New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

install *ml and *mli files for compiler-libs too #827

Merged
merged 1 commit into from Oct 21, 2016

Conversation

Projects
None yet
10 participants
@hendriktews
Contributor

hendriktews commented Sep 27, 2016

All the other libraries install the *ml and *mli files together with the compiled library code. This patch fixes this for the compiler-libs library.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Sep 27, 2016

Contributor

Partly solves MPR7362.

Contributor

dbuenzli commented Sep 27, 2016

Partly solves MPR7362.

@gasche

This comment has been minimized.

Show comment
Hide comment
@gasche

gasche Sep 27, 2016

Member

I think (but I have not discussed this recently) that the reluctance to expose interface and implementations of the compiler-libs content comes from the fact that these modules are not documented and that no stability guarantee is provided for their users. The OCaml compiler maintainers often have a culture where the good, stable, exposed interface should be documented, and the private, non-stable, dangerous interface is un-documented -- and whether or not there is documentation can be understood as a message to indicate the intent of encouraging use and trying to provide stability.

I would personally agree that even the interfaces that are not meant for wide use and that are unstable could and should be documented. However, there is also a real concern with lack of clear information of what should be considered external/stable and internal/unstable. Finding ways to address this concern could be a useful way to push the proposed changes further.

For example, maybe it would be a good first step to add to the .mli of modules in compiler-libs which are undocumented and unstable an explicit comment reminding their readers that those files are internal to the compiler and that no stability guarantee is offered for them.

Member

gasche commented Sep 27, 2016

I think (but I have not discussed this recently) that the reluctance to expose interface and implementations of the compiler-libs content comes from the fact that these modules are not documented and that no stability guarantee is provided for their users. The OCaml compiler maintainers often have a culture where the good, stable, exposed interface should be documented, and the private, non-stable, dangerous interface is un-documented -- and whether or not there is documentation can be understood as a message to indicate the intent of encouraging use and trying to provide stability.

I would personally agree that even the interfaces that are not meant for wide use and that are unstable could and should be documented. However, there is also a real concern with lack of clear information of what should be considered external/stable and internal/unstable. Finding ways to address this concern could be a useful way to push the proposed changes further.

For example, maybe it would be a good first step to add to the .mli of modules in compiler-libs which are undocumented and unstable an explicit comment reminding their readers that those files are internal to the compiler and that no stability guarantee is offered for them.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Sep 28, 2016

Contributor

@gasche I have no opinion whether the compiler-libs interfaces should be installed or not. But I'd really like that mli and cmti files are at least consistently installed or not installed for all the libs of the distribution, so that odig can provide a consistent view of the libs API to the users whether it is using ocamldoc or odoc.

Contributor

dbuenzli commented Sep 28, 2016

@gasche I have no opinion whether the compiler-libs interfaces should be installed or not. But I'd really like that mli and cmti files are at least consistently installed or not installed for all the libs of the distribution, so that odig can provide a consistent view of the libs API to the users whether it is using ocamldoc or odoc.

@gasche

This comment has been minimized.

Show comment
Hide comment
@gasche

gasche Sep 28, 2016

Member

I understand your request from the point of view of the tool, I was pointing out that there were clear social reasons why the stdlib files and the compiler-libs files are handled differently, this is not merely a matter of an unexplained and unjustified inconsistency.

Member

gasche commented Sep 28, 2016

I understand your request from the point of view of the tool, I was pointing out that there were clear social reasons why the stdlib files and the compiler-libs files are handled differently, this is not merely a matter of an unexplained and unjustified inconsistency.

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Sep 28, 2016

Contributor

I believe that one needs the mli and the ml files of compiler-libs if one really wants to program against this library. Currently users need to download the OCaml sources separately because neither opam nor debian install the those files.

The OCaml reference manual clearly says "the exported front-end interface [...] does not provide any backwards compatibility guarantees." From my point of view, this warning is completely sufficient.

For moving forward, it would be nice to find somebody who is able and willing to decide about what needs to be done to merge this pull request. @gasche could you do this?

I can put a comment

(** This interface follows the evolution of the OCaml language and implementation and is likely to change in the future. There is no guarantee about compatibility with future releases. *)

into those files referenced from the OCaml manual, and a comment

(** This is an OCaml internal interface that is likely to change in the future. Usage is discouraged. *)

into all other files. But before doing so I would like to reach an agreement about these comments and the the other conditions for this pull request.

Contributor

hendriktews commented Sep 28, 2016

I believe that one needs the mli and the ml files of compiler-libs if one really wants to program against this library. Currently users need to download the OCaml sources separately because neither opam nor debian install the those files.

The OCaml reference manual clearly says "the exported front-end interface [...] does not provide any backwards compatibility guarantees." From my point of view, this warning is completely sufficient.

For moving forward, it would be nice to find somebody who is able and willing to decide about what needs to be done to merge this pull request. @gasche could you do this?

I can put a comment

(** This interface follows the evolution of the OCaml language and implementation and is likely to change in the future. There is no guarantee about compatibility with future releases. *)

into those files referenced from the OCaml manual, and a comment

(** This is an OCaml internal interface that is likely to change in the future. Usage is discouraged. *)

into all other files. But before doing so I would like to reach an agreement about these comments and the the other conditions for this pull request.

@lpw25

This comment has been minimized.

Show comment
Hide comment
@lpw25

lpw25 Sep 28, 2016

Contributor

I believe that one needs the mli and the ml files of compiler-libs if one really wants to program against this library.

I can see why this applies to the .mli files since they (theoretically) contain documentation, but why put the entire source of of the compiler in people's lib directory?

Contributor

lpw25 commented Sep 28, 2016

I believe that one needs the mli and the ml files of compiler-libs if one really wants to program against this library.

I can see why this applies to the .mli files since they (theoretically) contain documentation, but why put the entire source of of the compiler in people's lib directory?

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Sep 28, 2016

Contributor

but why put the entire source of of the compiler in people's lib directory?

The ml files are installed for many libraries, see for instance the stdlib or lablgtk2.

The real reason is that the documentation in the mli files is not sufficient. It is, for instance, necessary to call Location.init before Parse.implementation. You can find out about this however only by looking at the sources.

The ml files only add 4.5MB on top of 70MB of the compiled modules and 1.2 MB of the mli files.

Contributor

hendriktews commented Sep 28, 2016

but why put the entire source of of the compiler in people's lib directory?

The ml files are installed for many libraries, see for instance the stdlib or lablgtk2.

The real reason is that the documentation in the mli files is not sufficient. It is, for instance, necessary to call Location.init before Parse.implementation. You can find out about this however only by looking at the sources.

The ml files only add 4.5MB on top of 70MB of the compiled modules and 1.2 MB of the mli files.

@garrigue

This comment has been minimized.

Show comment
Hide comment
@garrigue

garrigue Sep 28, 2016

Contributor

@hendriktews I think this is actually intentional.
As said earlier, as long as we do not commit to an official API for the compiler libraries, we cannot explicitly encourage people to use them directly. The cmi's are there so that one can compile compiler extensions against them, but no to allow people to start writing them without understanding how the compiler works (to some extent). So it seems reasonable to require that people first download the source files, or look them up on github.

This said, as you point out, this situation is not ideal when one wants to use tools to browse the sources. I already had this problem a long time ago when I wrote ocamlbrowser, and used it for the compiler itself. We could maybe add an extra target to install those files, but just for developers who need them. For others I'm afraid this would send the wrong message.

Contributor

garrigue commented Sep 28, 2016

@hendriktews I think this is actually intentional.
As said earlier, as long as we do not commit to an official API for the compiler libraries, we cannot explicitly encourage people to use them directly. The cmi's are there so that one can compile compiler extensions against them, but no to allow people to start writing them without understanding how the compiler works (to some extent). So it seems reasonable to require that people first download the source files, or look them up on github.

This said, as you point out, this situation is not ideal when one wants to use tools to browse the sources. I already had this problem a long time ago when I wrote ocamlbrowser, and used it for the compiler itself. We could maybe add an extra target to install those files, but just for developers who need them. For others I'm afraid this would send the wrong message.

@gasche

This comment has been minimized.

Show comment
Hide comment
@gasche

gasche Sep 28, 2016

Member

For moving forward, it would be nice to find somebody who is able and willing to decide about what needs to be done to merge this pull request. @gasche could you do this?

I would expect either @damiendoligez or @xavierleroy to have strong opinions on this.

Member

gasche commented Sep 28, 2016

For moving forward, it would be nice to find somebody who is able and willing to decide about what needs to be done to merge this pull request. @gasche could you do this?

I would expect either @damiendoligez or @xavierleroy to have strong opinions on this.

@johnwhitington

This comment has been minimized.

Show comment
Hide comment
@johnwhitington

johnwhitington Sep 28, 2016

Contributor

It would be nice to have the .ml files for otherlibs installed too, if that could be considered at the same time.

Contributor

johnwhitington commented Sep 28, 2016

It would be nice to have the .ml files for otherlibs installed too, if that could be considered at the same time.

@lefessan

This comment has been minimized.

Show comment
Hide comment
@lefessan

lefessan Sep 28, 2016

Contributor

With upcoming OPAM 2.0 (compilers as packages), downloading the sources of the compiler is as simple as:

opam source ocaml.4.03.0

and you will find an archive 4.03.0.tar.gz in the directory.
If the reason for installing the sources is to avoid downloading the sources, it is a bit weak now.

Contributor

lefessan commented Sep 28, 2016

With upcoming OPAM 2.0 (compilers as packages), downloading the sources of the compiler is as simple as:

opam source ocaml.4.03.0

and you will find an archive 4.03.0.tar.gz in the directory.
If the reason for installing the sources is to avoid downloading the sources, it is a bit weak now.

@damiendoligez damiendoligez added this to the 4.04 milestone Sep 28, 2016

@damiendoligez

This comment has been minimized.

Show comment
Hide comment
@damiendoligez

damiendoligez Sep 28, 2016

Member

The "downloading the source is hard" argument was always weak, but I find @dbuenzli's remark compelling: whenever we install the .cmi and .cmti we should also install at least the .mli because some tools need it.

I'm much less enthusiastic about installing the .ml files: if you want to look at the sources you should look at them in their natural environment, the whole sourcetree.

Member

damiendoligez commented Sep 28, 2016

The "downloading the source is hard" argument was always weak, but I find @dbuenzli's remark compelling: whenever we install the .cmi and .cmti we should also install at least the .mli because some tools need it.

I'm much less enthusiastic about installing the .ml files: if you want to look at the sources you should look at them in their natural environment, the whole sourcetree.

@lefessan

This comment has been minimized.

Show comment
Hide comment
@lefessan

lefessan Sep 28, 2016

Contributor

I don't think we should expose direct sources (.mli for interfaces) in the installation directory. These files may use syntax extensions, preprocessor directives, etc. that will make them unusable by any tool relying on them. The idea should be for these tools to use the .cmti files, that are supposed to contain both types and comments originating from the source. If these files are not usable by the tools, then we should fix them to provide the information needed by the tools, instead of using a partial solution that is bound to fail at some point.

Contributor

lefessan commented Sep 28, 2016

I don't think we should expose direct sources (.mli for interfaces) in the installation directory. These files may use syntax extensions, preprocessor directives, etc. that will make them unusable by any tool relying on them. The idea should be for these tools to use the .cmti files, that are supposed to contain both types and comments originating from the source. If these files are not usable by the tools, then we should fix them to provide the information needed by the tools, instead of using a partial solution that is bound to fail at some point.

@johnwhitington

This comment has been minimized.

Show comment
Hide comment
@johnwhitington

johnwhitington Sep 28, 2016

Contributor

I admit my use-case is obscure. I have a program which uses compiler-libs to directly interpret the OCaml AST.

feast:ocaml-i john$ ./ocamli -e "1 + 2 * 3" -show-all
    1 + 2 * 3
=>  1 + 6
=>  7
feast:ocaml-i john$ ./ocamli -e "List.fold_left ( + ) 0 [1; 2; 3]" -show
6

Now, this is fine on any computer with OCaml installed, because list.ml is installed, so the definition of fold_left is available. It would be nice to have bigarray.ml, unix.ml etc. also installed, since I want ocamli to run on any computer where OCaml can run. Only genuinely external libraries would need special treatment. I hope one day, of course, to persuade all library authors to install their source...

Contributor

johnwhitington commented Sep 28, 2016

I admit my use-case is obscure. I have a program which uses compiler-libs to directly interpret the OCaml AST.

feast:ocaml-i john$ ./ocamli -e "1 + 2 * 3" -show-all
    1 + 2 * 3
=>  1 + 6
=>  7
feast:ocaml-i john$ ./ocamli -e "List.fold_left ( + ) 0 [1; 2; 3]" -show
6

Now, this is fine on any computer with OCaml installed, because list.ml is installed, so the definition of fold_left is available. It would be nice to have bigarray.ml, unix.ml etc. also installed, since I want ocamli to run on any computer where OCaml can run. Only genuinely external libraries would need special treatment. I hope one day, of course, to persuade all library authors to install their source...

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Sep 28, 2016

Contributor

I don't think we should expose direct sources (.mli for interfaces) in the installation directory.

From an odig perspective, I won't need these once odoc is mature enough, but meanwhile I'd suggest to keep them so that people can have the standard lib in their doc via the ocamldoc generation path.

If I can make MPR7362 more precise from a cmti install perspective here are the points:

  1. All compiler-libs cmti files are currently installed, so their interface is actually exposed. You might want to change that.
  2. otherlibs systematically (IIRC) lack cmti files. This should really be changed.
Contributor

dbuenzli commented Sep 28, 2016

I don't think we should expose direct sources (.mli for interfaces) in the installation directory.

From an odig perspective, I won't need these once odoc is mature enough, but meanwhile I'd suggest to keep them so that people can have the standard lib in their doc via the ocamldoc generation path.

If I can make MPR7362 more precise from a cmti install perspective here are the points:

  1. All compiler-libs cmti files are currently installed, so their interface is actually exposed. You might want to change that.
  2. otherlibs systematically (IIRC) lack cmti files. This should really be changed.
@alainfrisch

This comment has been minimized.

Show comment
Hide comment
@alainfrisch

alainfrisch Sep 28, 2016

Contributor

My 2 cents: I don't buy the arguments that compiler-libs should somehow remains "hidden". It is widely used and is required for a lot of nice and important tools which could not be maintained outside the core distribution otherwise. The lack of API stability guarantee or nice upfront design should not be an excuse to avoid documenting it (and polish it). (Some parts of compiler-libs are actually not too badly documented, such as Parsetree, Ast_mapper, Ast_iterator.) I've nothing against adding more disclaimers, perhaps even in each .mli, about the lack of stability guarantee (and why not some call for helping with improving the doc?).

As for which files to install: I've no strong opinion on it; personally, I'm perfectly fine browsing .mli files from upstream source packages but for easier accessibility, it is rather clear that the current situation is quite bad on the documentation front: no decent way to get cross-referenced documentation for all installed packages. This is an important topic and I'd be tempted to follow the suggestions of people putting efforts into improving this situation (i.e. @dbuenzli). If the community ever produces a better tool than ocamldoc and settles on it, one might always revert later and stop installing .mli files if .cmti are enough to get good documentation, which is not the case today.

Contributor

alainfrisch commented Sep 28, 2016

My 2 cents: I don't buy the arguments that compiler-libs should somehow remains "hidden". It is widely used and is required for a lot of nice and important tools which could not be maintained outside the core distribution otherwise. The lack of API stability guarantee or nice upfront design should not be an excuse to avoid documenting it (and polish it). (Some parts of compiler-libs are actually not too badly documented, such as Parsetree, Ast_mapper, Ast_iterator.) I've nothing against adding more disclaimers, perhaps even in each .mli, about the lack of stability guarantee (and why not some call for helping with improving the doc?).

As for which files to install: I've no strong opinion on it; personally, I'm perfectly fine browsing .mli files from upstream source packages but for easier accessibility, it is rather clear that the current situation is quite bad on the documentation front: no decent way to get cross-referenced documentation for all installed packages. This is an important topic and I'd be tempted to follow the suggestions of people putting efforts into improving this situation (i.e. @dbuenzli). If the community ever produces a better tool than ocamldoc and settles on it, one might always revert later and stop installing .mli files if .cmti are enough to get good documentation, which is not the case today.

@gasche

This comment has been minimized.

Show comment
Hide comment
@gasche

gasche Sep 28, 2016

Member

For the record, I fully agree with Alain's position above.

@johnwhitington I don't think that there is any issue with treating otherlibs/ stuff like stdlib/ is handled today, so I would be comfortable merging a pull request that does that. It is for the compiler-libs part that I think we need a stronger consensus first.

Member

gasche commented Sep 28, 2016

For the record, I fully agree with Alain's position above.

@johnwhitington I don't think that there is any issue with treating otherlibs/ stuff like stdlib/ is handled today, so I would be comfortable merging a pull request that does that. It is for the compiler-libs part that I think we need a stronger consensus first.

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Sep 28, 2016

Contributor

My motivation for this pull request is not that downloading the sources is hard. That has never been hard and the upcoming "opam source" does not really make it simpler IMHO.

The motivation is convenience: For many library functions, when I type their name, the declaration and the documentation in the mli file are a key stroke away. If that leaves any question open or if I am just curious, I can look at the sources with another key stroke. I have not yet seen a convincing argument why this should be different for compiler-libs.

compiler-libs is described in the OCaml manual, there are many tools using it already. Many module interfaces are contained in the documentation in a hidden way (eg, Ast_invariants, Clflags, Identifiable). I don't think that installing the mli or ml files would encourage anybody to do anything on top of what they planed already.

Anyway, I believe the discussion is far too long already for such a simple change. As I wrote before, I am happy to add the disclaimers. I am also happy to create a different install target for compiler-libs and/or the mli and/or the ml files.

Contributor

hendriktews commented Sep 28, 2016

My motivation for this pull request is not that downloading the sources is hard. That has never been hard and the upcoming "opam source" does not really make it simpler IMHO.

The motivation is convenience: For many library functions, when I type their name, the declaration and the documentation in the mli file are a key stroke away. If that leaves any question open or if I am just curious, I can look at the sources with another key stroke. I have not yet seen a convincing argument why this should be different for compiler-libs.

compiler-libs is described in the OCaml manual, there are many tools using it already. Many module interfaces are contained in the documentation in a hidden way (eg, Ast_invariants, Clflags, Identifiable). I don't think that installing the mli or ml files would encourage anybody to do anything on top of what they planed already.

Anyway, I believe the discussion is far too long already for such a simple change. As I wrote before, I am happy to add the disclaimers. I am also happy to create a different install target for compiler-libs and/or the mli and/or the ml files.

@lefessan

This comment has been minimized.

Show comment
Hide comment
@lefessan

lefessan Sep 29, 2016

Contributor

Have you ever tried ocp-browser:
https://www.typerex.org/ocp-index.html#ocp-browser

If we install the sources for the compiler for convenience, then we should do it for all contributed libraries for exactly the same reason. Every OCaml switch is already big (between 200 MB to 1 GB), which makes having a lot of them become very space consuming: a good OCaml dev should make his contributions work on as many OCaml versions as possible, which means having at least all versions since 3.12.1 installed... What will happen if we double the space requirement by adding sources, just for convenience ?

I think a better approach would be for you to create an OPAM package ocaml-sources, with the same sources as the compiler, but which installation procedure would copy the sources in the ocaml sub-directory. That way, your problem would be solved, without impacting other users.

Contributor

lefessan commented Sep 29, 2016

Have you ever tried ocp-browser:
https://www.typerex.org/ocp-index.html#ocp-browser

If we install the sources for the compiler for convenience, then we should do it for all contributed libraries for exactly the same reason. Every OCaml switch is already big (between 200 MB to 1 GB), which makes having a lot of them become very space consuming: a good OCaml dev should make his contributions work on as many OCaml versions as possible, which means having at least all versions since 3.12.1 installed... What will happen if we double the space requirement by adding sources, just for convenience ?

I think a better approach would be for you to create an OPAM package ocaml-sources, with the same sources as the compiler, but which installation procedure would copy the sources in the ocaml sub-directory. That way, your problem would be solved, without impacting other users.

@damiendoligez

This comment has been minimized.

Show comment
Hide comment
@damiendoligez

damiendoligez Sep 29, 2016

Member

Now, this is fine on any computer with OCaml installed, because list.ml is installed, so the definition of fold_left is available. It would be nice to have bigarray.ml, unix.ml etc. also installed, since I want ocamli to run on any computer where OCaml can run.

That's a pretty specific use case and I don't think it justifies installing the .ml files of everything for everyone.

@hendriktews here's my recommendation: install otherlibs .cmti files and add an install target for .ml files but don't install them by default.

Member

damiendoligez commented Sep 29, 2016

Now, this is fine on any computer with OCaml installed, because list.ml is installed, so the definition of fold_left is available. It would be nice to have bigarray.ml, unix.ml etc. also installed, since I want ocamli to run on any computer where OCaml can run.

That's a pretty specific use case and I don't think it justifies installing the .ml files of everything for everyone.

@hendriktews here's my recommendation: install otherlibs .cmti files and add an install target for .ml files but don't install them by default.

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Sep 29, 2016

Contributor

@damiendoligez OK - and the mli files of compiler-libs go into the default installation target?

Contributor

hendriktews commented Sep 29, 2016

@damiendoligez OK - and the mli files of compiler-libs go into the default installation target?

@damiendoligez

This comment has been minimized.

Show comment
Hide comment
@damiendoligez

damiendoligez Oct 4, 2016

Member

@hendriktews

and the mli files of compiler-libs go into the default installation target?

Yes, that makes sense.

Member

damiendoligez commented Oct 4, 2016

@hendriktews

and the mli files of compiler-libs go into the default installation target?

Yes, that makes sense.

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Oct 9, 2016

Contributor

I updated this PR, taking the comments into account. By default it installs the missing mli, cmti and cmt files. To get some of these cmti files, I added option -bin-annot for ocamldoc and tools. There is an additional target for installing compiler-libs ml sources.

In detail: I looked a bit more systematically for missing cmti files and saw that apart from otherlibs, also

  • ocamldoc/odoc_info.cmt
  • ocamldoc/odoc_info.cmti
  • profiling.cmt
  • profiling.cmti
  • topdirs.cmti

are missing. I am quite sure that we want to install odoc_info.cmti but for the others I don't really know. If these files were deliberately omitted, please tell and I'll revert.

By default, this PR additionally installs

  • compiler-lib/*mli
  • topdirs.mli
  • arith_status.cmti big_int.cmti bigarray.cmti dynlink.cmti
    graphics.cmti graphicsX11.cmti nat.cmti num.cmti profiling.cmti
    ratio.cmti str.cmti topdirs.cmti unix.cmti unixLabels.cmti
  • ocamldoc/odoc_info.cmti
  • in threads: condition.cmti event.cmti mutex.cmti thread.cmti threadUnix.cmti
  • in vmthreads: condition.cmti event.cmti mutex.cmti thread.cmti threadUnix.cmti
  • ocamldoc/odoc_info.cmt
  • profiling.cmt

The make target install-compiler-sources installs the ml files for compiler-libs.

@dbuenzli please check if this fixes MPR7362 completely.

@damiendoligez Please check.

Contributor

hendriktews commented Oct 9, 2016

I updated this PR, taking the comments into account. By default it installs the missing mli, cmti and cmt files. To get some of these cmti files, I added option -bin-annot for ocamldoc and tools. There is an additional target for installing compiler-libs ml sources.

In detail: I looked a bit more systematically for missing cmti files and saw that apart from otherlibs, also

  • ocamldoc/odoc_info.cmt
  • ocamldoc/odoc_info.cmti
  • profiling.cmt
  • profiling.cmti
  • topdirs.cmti

are missing. I am quite sure that we want to install odoc_info.cmti but for the others I don't really know. If these files were deliberately omitted, please tell and I'll revert.

By default, this PR additionally installs

  • compiler-lib/*mli
  • topdirs.mli
  • arith_status.cmti big_int.cmti bigarray.cmti dynlink.cmti
    graphics.cmti graphicsX11.cmti nat.cmti num.cmti profiling.cmti
    ratio.cmti str.cmti topdirs.cmti unix.cmti unixLabels.cmti
  • ocamldoc/odoc_info.cmti
  • in threads: condition.cmti event.cmti mutex.cmti thread.cmti threadUnix.cmti
  • in vmthreads: condition.cmti event.cmti mutex.cmti thread.cmti threadUnix.cmti
  • ocamldoc/odoc_info.cmt
  • profiling.cmt

The make target install-compiler-sources installs the ml files for compiler-libs.

@dbuenzli please check if this fixes MPR7362 completely.

@damiendoligez Please check.

@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Oct 9, 2016

Contributor

@lefessan:

Every OCaml switch is already big (between 200 MB to 1 GB), which makes having a lot of them become very space consuming

If opam has a problem or if the way opam is used creates problems, then the people concerned about this should fix this in opam. But these opam problems should neither have a negative impact on people not using opam (and I believe there are quite a lot) nor stop us from taking the right decisions here.

What will happen if we double the space requirement by adding sources

Please argue with facts instead of dubious guesses. This PR adds 1.2MB of mli files and 4.5MB of ml files for the install-compiler-sources target. This is an increase of 0.6% and, respectively, 2.3% for the install-compiler-sources target. I really cannot see any point for creating fear about doubling space requirements.

I think a better approach would be for you to create an OPAM package ocaml-sources, with the same sources as the compiler,

IMHO, the solution would be that the opam maintainers create opam compiler packages that only install the essential stuff for those people that want to install all available OCaml versions in parallel.

Contributor

hendriktews commented Oct 9, 2016

@lefessan:

Every OCaml switch is already big (between 200 MB to 1 GB), which makes having a lot of them become very space consuming

If opam has a problem or if the way opam is used creates problems, then the people concerned about this should fix this in opam. But these opam problems should neither have a negative impact on people not using opam (and I believe there are quite a lot) nor stop us from taking the right decisions here.

What will happen if we double the space requirement by adding sources

Please argue with facts instead of dubious guesses. This PR adds 1.2MB of mli files and 4.5MB of ml files for the install-compiler-sources target. This is an increase of 0.6% and, respectively, 2.3% for the install-compiler-sources target. I really cannot see any point for creating fear about doubling space requirements.

I think a better approach would be for you to create an OPAM package ocaml-sources, with the same sources as the compiler,

IMHO, the solution would be that the opam maintainers create opam compiler packages that only install the essential stuff for those people that want to install all available OCaml versions in parallel.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Oct 9, 2016

Contributor

If opam has a problem or if the way opam is used creates problems, then the people concerned about this should fix this in opam.

I don't think that's a problem specific to OPAM, system package manager might also want to have lean installs.

@dbuenzli please check if this fixes MPR7362 completely.

I guess so, though I don't know if maybe too much .cmti files are installed; only those that are part of the API should be installed --- but this is not for me to judge.

Regarding .cmt files I'm not sure there's an interest in installing those, maybe you want to connect their install to the *-compiler-source targets since they correspond to the .ml files.

Contributor

dbuenzli commented Oct 9, 2016

If opam has a problem or if the way opam is used creates problems, then the people concerned about this should fix this in opam.

I don't think that's a problem specific to OPAM, system package manager might also want to have lean installs.

@dbuenzli please check if this fixes MPR7362 completely.

I guess so, though I don't know if maybe too much .cmti files are installed; only those that are part of the API should be installed --- but this is not for me to judge.

Regarding .cmt files I'm not sure there's an interest in installing those, maybe you want to connect their install to the *-compiler-source targets since they correspond to the .ml files.

@gasche

This comment has been minimized.

Show comment
Hide comment
@gasche

gasche Oct 9, 2016

Member

The "apart from otherlibs" files seem perfectly fine to me, I'm quite sure if they were omitted it was by mistake (and in fact Topdirs is interesting to people playing to the toplevel).

Member

gasche commented Oct 9, 2016

The "apart from otherlibs" files seem perfectly fine to me, I'm quite sure if they were omitted it was by mistake (and in fact Topdirs is interesting to people playing to the toplevel).

Show outdated Hide outdated Makefile
if test -f ocamlopt; then $(MAKE) installopt-compiler-sources; fi
# Installation of the *.ml sources native-code part of compiler-libs
installopt-compiler-sources:

This comment has been minimized.

@lefessan

lefessan Oct 10, 2016

Contributor

Why not merge these two targets ? If somebody wants access to the compiler sources, he probably wants access to both bytecode and native code compilers.

@lefessan

lefessan Oct 10, 2016

Contributor

Why not merge these two targets ? If somebody wants access to the compiler sources, he probably wants access to both bytecode and native code compilers.

This comment has been minimized.

@damiendoligez

damiendoligez Oct 18, 2016

Member

I agree with @lefessan. Just install all compiler sources unconditionally.

@damiendoligez

damiendoligez Oct 18, 2016

Member

I agree with @lefessan. Just install all compiler sources unconditionally.

@mshinwell

This comment has been minimized.

Show comment
Hide comment
@mshinwell

mshinwell Oct 10, 2016

Contributor

The forthcoming gdb support for OCaml needs both .cmt files and source files to be accessible in order to work fully. One way of achieving this is using OPAMKEEPBUILDDIR=true, but that does use more disk space than is really required, since the .o and most of the other .cm* files are not required for debugging. (You also have to remember to actually set the variable...) Instead perhaps it is reasonable to consider putting the source files in the installation directory for compiler-libs. This needs some code changes so they can be located by the debugger, but that should be doable. For people writing code against compiler-libs this would probably improve usability.

Contributor

mshinwell commented Oct 10, 2016

The forthcoming gdb support for OCaml needs both .cmt files and source files to be accessible in order to work fully. One way of achieving this is using OPAMKEEPBUILDDIR=true, but that does use more disk space than is really required, since the .o and most of the other .cm* files are not required for debugging. (You also have to remember to actually set the variable...) Instead perhaps it is reasonable to consider putting the source files in the installation directory for compiler-libs. This needs some code changes so they can be located by the debugger, but that should be doable. For people writing code against compiler-libs this would probably improve usability.

@dbuenzli

This comment has been minimized.

Show comment
Hide comment
@dbuenzli

dbuenzli Oct 10, 2016

Contributor

@mshinwell It would be nice if you could make a summary on the platform mailing list about what you think packages should install by default (compilation flags and build artefacts).

The issue has been raised more than once without being conclusive.

For most topkg packages these changes could happen by simply rereleasing topkg without packages having to be re-released (for now topkg compiles in debug mode and installs the .mli and .cmti files for the modules that you indicate are part of your API).

Contributor

dbuenzli commented Oct 10, 2016

@mshinwell It would be nice if you could make a summary on the platform mailing list about what you think packages should install by default (compilation flags and build artefacts).

The issue has been raised more than once without being conclusive.

For most topkg packages these changes could happen by simply rereleasing topkg without packages having to be re-released (for now topkg compiles in debug mode and installs the .mli and .cmti files for the modules that you indicate are part of your API).

@damiendoligez

Looks good to me, apart from the two notes I added.

Show outdated Hide outdated Makefile
cp compilerlibs/ocamlcommon.cma compilerlibs/ocamlbytecomp.cma \
compilerlibs/ocamltoplevel.cma $(BYTESTART) $(TOPLEVELSTART) \
$(INSTALL_COMPLIBDIR)
cp expunge $(INSTALL_LIBDIR)/expunge$(EXE)
cp toplevel/topdirs.cmi $(INSTALL_LIBDIR)
cp toplevel/topdirs.cmi toplevel/topdirs.cmti toplevel/topdirs.mli \

This comment has been minimized.

@damiendoligez

damiendoligez Oct 18, 2016

Member

Please add toplevel/topdirs.cmt too.

@damiendoligez

damiendoligez Oct 18, 2016

Member

Please add toplevel/topdirs.cmt too.

Show outdated Hide outdated Makefile
if test -f ocamlopt; then $(MAKE) installopt-compiler-sources; fi
# Installation of the *.ml sources native-code part of compiler-libs
installopt-compiler-sources:

This comment has been minimized.

@damiendoligez

damiendoligez Oct 18, 2016

Member

I agree with @lefessan. Just install all compiler sources unconditionally.

@damiendoligez

damiendoligez Oct 18, 2016

Member

I agree with @lefessan. Just install all compiler sources unconditionally.

improve installation of additional material
- install missing mli and cmti files for compiler-libs and otherlibs
- new make target install-compiler-sources to install compiler-libs ml files
@hendriktews

This comment has been minimized.

Show comment
Hide comment
@hendriktews

hendriktews Oct 20, 2016

Contributor

@damiendoligez I rebased and addressed your comments, please check.

@dbuenzli I was not sure about the cmt files either, but make install installs almost all cmt files - I simply followed the pattern.

@mshinwell I am not sure I understand. Do you mean that the gdb support won't work properly, because the sources from many different directories are install flat in compiler-libs? If this is the case, I would suggest to wait until we actually see the problem. It is easy to copy the directory structure into the compiler-libs installation directory, but I would suggest to wait with this.

Contributor

hendriktews commented Oct 20, 2016

@damiendoligez I rebased and addressed your comments, please check.

@dbuenzli I was not sure about the cmt files either, but make install installs almost all cmt files - I simply followed the pattern.

@mshinwell I am not sure I understand. Do you mean that the gdb support won't work properly, because the sources from many different directories are install flat in compiler-libs? If this is the case, I would suggest to wait until we actually see the problem. It is easy to copy the directory structure into the compiler-libs installation directory, but I would suggest to wait with this.

@damiendoligez damiendoligez merged commit 0bb16e0 into ocaml:trunk Oct 21, 2016

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@damiendoligez

This comment has been minimized.

Show comment
Hide comment
@damiendoligez
Member

damiendoligez commented Oct 21, 2016

Thanks @hendriktews !

@hendriktews hendriktews deleted the hendriktews:compiler-libs-mli branch Oct 22, 2016

ollie314 added a commit to ollie314/ocaml that referenced this pull request Oct 22, 2016

Merge pull request #10 from ocaml/trunk
improve installation of additional material (#827)

shindere added a commit to shindere/ocaml that referenced this pull request Dec 28, 2016

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Jan 1, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Jan 3, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Jan 14, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Feb 6, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Feb 7, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

shindere added a commit to shindere/ocaml that referenced this pull request Feb 15, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.

camlspotter pushed a commit to camlspotter/ocaml that referenced this pull request Oct 17, 2017

improve installation of additional material (#827)
- install missing mli and cmti files for compiler-libs and otherlibs
- new make target install-compiler-sources to install compiler-libs ml files

camlspotter pushed a commit to camlspotter/ocaml that referenced this pull request Oct 17, 2017

Move the install-compiler-sources target from Makefile to Makefile.sh…
…ared

This target has been introduced by GPR #827 but added only to the
Unix build system. This commit makes it available to the Windows build
system, too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment